Information interaction method and device, electronic equipment and storage medium

Abstract

The embodiment of the present disclosure provides an information interaction method and device, electronic equipment and storage medium, which displays a terminal interface of an external terminal device in a first target area in an extended reality space; and displays a calling result of a target application in the terminal interface in response to a first user operation on the extended reality space, wherein the target application is an application running on the external terminal device. Since the terminal interface of the external terminal device is displayed in the extended reality space, and the target application running on the external terminal device is controlled through the operation on the extended reality space, the purpose of displaying and controlling the external terminal device in the extended reality space is achieved, efficient interaction between the virtual reality device and the external terminal device is realized, and the user can conveniently control the external terminal device during use of the virtual reality device, thereby improving the use convenience.

Description

Technical Field

[0001] This disclosure relates to the field of virtual reality technology, and in particular to an information interaction method, apparatus, electronic device, and storage medium. Background Technology

[0002] Virtual Reality (VR) is a technology that uses computing and display devices to simulate objects in the real world, enabling users to have an immersive viewing experience. With the continuous improvement of image processing technology and computing power, the realism and immersion that virtual reality technology brings to users are becoming increasingly stronger.

[0003] In existing technologies, virtual reality applications are typically used in conjunction with corresponding virtual reality devices, such as virtual reality headsets, virtual reality controllers, and headphones. However, existing virtual reality devices have a limitation: they cannot display or control external terminal devices, leading to inconvenience for users. Summary of the Invention

[0004] This disclosure provides an information interaction method, apparatus, electronic device, and storage medium to overcome the problem that virtual reality devices cannot display and operate external terminal devices.

[0005] In a first aspect, embodiments of this disclosure provide an information interaction method, including:

[0006] In a first target area within the extended reality space, the terminal interface of an external terminal device is displayed; in response to a first user operation on the extended reality space, the invocation result of a target application is displayed in the terminal interface, wherein the target application is an application running on the external terminal device.

[0007] Secondly, embodiments of this disclosure provide an information interaction device, including:

[0008] The display module is used to display the terminal interface of an external terminal device in the first target area within the extended reality space.

[0009] An interaction module is used to respond to a first user operation on the extended reality space and display the invocation result of a target application within the terminal interface, wherein the target application is an application running on the external terminal device.

[0010] Thirdly, embodiments of this disclosure provide an electronic device, including:

[0011] A processor, and a memory communicatively connected to the processor;

[0012] The memory stores computer-executed instructions;

[0013] The processor executes computer execution instructions stored in the memory to implement the information interaction method described in the first aspect and various possible designs of the first aspect.

[0014] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the information interaction method described in the first aspect and various possible designs of the first aspect.

[0015] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the information interaction method described in the first aspect and various possible designs of the first aspect.

[0016] The information interaction method, apparatus, electronic device, and storage medium provided in this embodiment display the terminal interface of an external terminal device in a first target area within an extended reality space. In response to a first user operation on the extended reality space, the terminal interface displays the invocation result of a target application, wherein the target application is an application running on the external terminal device. By displaying the terminal interface of the external terminal device within the extended reality space and controlling the target application running on the external terminal device through operations on the extended reality space, the purpose of displaying and controlling the external terminal device within the extended reality space is achieved. This realizes efficient interaction between the virtual reality device and the external terminal device, allowing users to conveniently control the external terminal device while using the virtual reality device, thus improving ease of use. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 An application scenario diagram of the information interaction method provided in the embodiments of this disclosure;

[0019] Figure 2 Flowchart of the information interaction method provided in the embodiments of this disclosure Figure 1 ;

[0020] Figure 3 A schematic diagram of an extended reality space provided for an embodiment of this disclosure;

[0021] Figure 4for Figure 2 A flowchart illustrating one implementation of step S101 in the illustrated embodiment;

[0022] Figure 5 A schematic diagram illustrating a process for displaying the terminal interface of an external terminal device within an extended reality space, provided by an embodiment of this disclosure;

[0023] Figure 6 for Figure 2 Flowchart of another implementation of step S101 in the illustrated embodiment;

[0024] Figure 7A This is a schematic diagram of a process for displaying a functional interface provided in an embodiment of the present disclosure;

[0025] Figure 7B This is a schematic diagram illustrating another process for displaying a functional interface provided in an embodiment of the present disclosure;

[0026] Figure 8 This is a schematic diagram illustrating a process for displaying a target terminal interface based on a trigger operation, as provided in an embodiment of this disclosure.

[0027] Figure 9 for Figure 2 A flowchart illustrating one implementation of step S102 in the illustrated embodiment;

[0028] Figure 10 Flowchart of the information interaction method provided in the embodiments of this disclosure Figure 2 ;

[0029] Figure 11 A schematic diagram of a notification interface provided in an embodiment of this disclosure;

[0030] Figure 12 This is a schematic diagram illustrating a process for displaying a notification interface, provided in an embodiment of the present disclosure.

[0031] Figure 13 A schematic diagram of a dynamic first target region provided in an embodiment of this disclosure;

[0032] Figure 14 This is a structural block diagram of an information interaction device provided in an embodiment of the present disclosure;

[0033] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure;

[0034] Figure 16 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0036] The application scenarios of the embodiments of this disclosure are explained below:

[0037] Figure 1 This diagram illustrates an application scenario of the information interaction method provided in this disclosure. The information interaction method provided in this disclosure can be applied to virtual reality (VR) applications and mixed reality (MR) applications. Specifically, as... Figure 1 As shown in the embodiments of this disclosure, the method can be applied to virtual reality devices, such as the virtual reality headset shown in the figure. The virtual reality headset contains a computing unit for running virtual reality applications (e.g., virtual reality games, virtual reality movies). On one hand, by wearing the virtual reality headset, users can immerse themselves in viewing images displayed by the virtual reality application. On the other hand, the virtual reality device can communicate with external terminal devices (shown as smartphones in the figure) and interact with them. The virtual reality device and the external terminal device can communicate directly via Wi-Fi, Bluetooth, etc., or indirectly via network devices such as routers and servers; no specific limitations are imposed.

[0038] Furthermore, exemplarily, the virtual reality device can also be virtual reality glasses, virtual reality controllers, and virtual reality host, etc. The method provided in this embodiment can run on any one or more devices among virtual reality headsets or glasses, virtual reality controllers, and virtual reality host. The method provided in this embodiment can also be applied to a cloud server communicating with the aforementioned virtual reality device. The cloud server receives user instructions obtained through the virtual reality device via network communication and sends image data to the virtual reality device for display. The execution entity of the information interaction method provided in this embodiment can be set according to specific needs, and will not be elaborated here.

[0039] In existing technologies, virtual reality applications typically require corresponding virtual reality devices. When users wear virtual reality headsets, headphones, and other devices, the highly immersive information display isolates them from the external real environment, preventing them from receiving messages from other external devices (such as mobile phones). Some related technologies connect external devices like smartphones to the virtual reality device, allowing notifications from these devices to be displayed in the extended reality space. However, when users see these notifications and need to further interact with the external device (e.g., replying to a text message), they still need to interact with the external device, resulting in the need to repeatedly wear the virtual reality device and causing inconvenience. This disclosure provides an information interaction method to solve the above problems.

[0040] refer to Figure 2 , Figure 2 Flowchart of the information interaction method provided in the embodiments of this disclosure Figure 1 The method described in this embodiment can be applied to virtual reality devices or cloud servers. This embodiment executively uses a virtual reality device as the execution subject. The information interaction method includes:

[0041] Step S101: Display the terminal interface of the external terminal device in the first target area within the extended reality space.

[0042] For example, a virtual reality device (VR device) is a device equipped with a computing unit for running and implementing virtual reality applications. Examples include virtual reality headsets (i.e., all-in-one devices) and virtual reality host units (i.e., standalone devices). Furthermore, the virtual reality host unit can be a dedicated host device or a general-purpose computer device. The virtual reality device runs a virtual reality operating system, through which virtual reality applications are run. The virtual reality device itself has a display unit (all-in-one device) or is communicatively connected to a display unit (standalone device). The display unit is, for example, a virtual reality headset or virtual reality glasses. The display unit has a screen that displays an extended reality space (i.e., virtual reality space or virtual display interface) to display images of the virtual space constructed by the virtual reality device.

[0043] Figure 3 A schematic diagram of an extended reality space provided for an embodiment of this disclosure, such as... Figure 3As shown, exemplarily, the virtual reality device is, for example, a virtual reality headset. After the virtual reality headset is started and initialized, the extended reality space is displayed on the headset's screen. In one possible scenario, the system interface of the virtual operating system is displayed within the extended reality space, including commonly used icons such as application icons and function icons. Clicking on these commonly used icons will run the corresponding virtual reality application and function, which will not be elaborated further. On the other hand, in the first target area of ​​the extended reality space, the terminal interface of the external terminal device is displayed, as shown in the reference... Figure 3 As shown, the external terminal device is, for example, a smartphone, and the corresponding terminal interface is a mobile phone interface. The mobile phone interface displays a running terminal application, such as a communication application. After displaying the smartphone interface in the extended reality space, the user can simultaneously observe the terminal application running in the mobile phone interface while viewing the extended reality space, thereby realizing information communication between the virtual reality space and the terminal device in the external environment.

[0044] Furthermore, in one possible implementation, the terminal interface of the external terminal device can always be displayed within the extended reality space. For example, after the operating system of the virtual reality device is initialized, and after obtaining the corresponding data through communication with the external terminal device, the terminal interface of the external terminal device is fixedly displayed in the first target area of ​​the extended reality space, such as the lower right corner of the extended reality space. This achieves the normalized display of the terminal interface of the external terminal device within the extended reality space, allowing users to view and process information from two terminal devices (the virtual reality device and the external terminal device) simultaneously.

[0045] In another possible implementation, the virtual reality device displays the terminal interface of an external terminal device within the extended reality space after receiving a trigger command. For example, this trigger command may be generated based on push notifications from the external terminal device. In one possible implementation, such as... Figure 4 As shown, the specific implementation steps of step S101 include:

[0046] Step S1011: Obtain push information sent by an external terminal device. The push information represents the application message generated by the target application running on the external terminal device.

[0047] Step S1012: Based on the push information, render the corresponding target terminal interface in the first target area within the extended reality space. The target terminal interface is used to display application messages generated by the target application.

[0048] Application messages generated by the target application are in-application messages generated during the operation of the target application on the external terminal device in response to a certain triggering event. For example, messages generated when an instant messaging application (target application) receives a user message or a video call. Figure 5This is a schematic diagram illustrating a process for displaying the terminal interface of an external terminal device within an extended reality space, as provided in an embodiment of this disclosure. Figure 5 As shown, the external terminal device is, for example, a smartphone. The external terminal device and the virtual reality device are connected via wireless communication methods such as Wi-Fi. Before the instant messaging application (target application) running on the external terminal device receives the user message (the first moment), only content related to the virtual reality application is displayed in the extended reality space. After the instant messaging application running on the external terminal device receives the user message (the second moment), the external terminal device sends a push notification to the virtual reality device. After receiving the push notification, the virtual reality device generates a trigger command and displays the target terminal interface containing the user message in the extended reality space based on the trigger command. Thus, the user message received by the instant messaging application running on the external terminal device can be synchronously displayed in the extended reality space.

[0049] After receiving the push information, the virtual reality device can generate the target terminal interface in the following optional ways. In one possible way, for example, after receiving the push information, the virtual reality device establishes an interface projection data channel with an external terminal device. Then, the external terminal device projects the application interface of the target application (used to display application messages) onto the first target area of ​​the extended reality space for rendering, thereby generating the target terminal interface displayed in the extended reality space.

[0050] In another possible approach, for example, the push message includes application information for the target application. After receiving the push information, the virtual reality device uses the push message to determine the corresponding target application. Then, based on the preset interface description information of the target application, it reconstructs and renders the interface layout of the target application within a first target area. It also combines this with the application information carried in the push message to populate data, thereby generating a target terminal interface displayed in the extended reality space. For example, the target application displays application information within a first interface of the external terminal device; the first interface and the target terminal interface have the same interface layout. Through the interface description information, accurate simulation of the first interface of the target application is achieved, improving the consistency between the terminal interface in the extended reality space and the first interface of the target application running on the external terminal device, thus enhancing visual immersion.

[0051] In this embodiment, the display of the terminal interface in the extended reality space is triggered based on push information, which realizes real-time message push between the external terminal device and the virtual reality device, while avoiding the problem of visual immersion caused by displaying the terminal interface in the extended reality space for a long time, thus improving the user experience.

[0052] In another possible implementation, such as Figure 6 As shown, the specific implementation steps of step S101 include:

[0053] Step S1013: In response to the fourth user operation, a functional interface is displayed in the second target area of ​​the extended reality space. The functional interface is used to display the identifier of at least one alternative application, which is an application running by an external terminal device.

[0054] Figure 7A This is a schematic diagram of a display function interface provided in an embodiment of the present disclosure. Figure 7B This is a schematic diagram of another display function interface provided in an embodiment of the present disclosure. The following is based on... Figure 7A and Figure 7B The above process will be described in detail. For example, in the extended reality space, the first target area is set on the right side of the interface and the second target area is set on the left side of the interface. First, after detecting a fourth user operation, the functional interface is displayed in the second target area of ​​the extended reality space. One possible implementation is as follows... Figure 7A As shown, the fourth user operation can be a trigger operation for a specific functional component in the extended reality space. For example, by using the right virtual reality controller to click on functional component A in the extended reality space, the functional interface can be displayed. Another possible implementation is as follows... Figure 7B As shown, the fourth user operation can also be a gesture for the virtual reality controller, such as raising the left virtual reality controller to display the function interface and the gesture icon corresponding to the raising gesture.

[0055] Furthermore, the function interface displays the identifier of at least one alternative application. These alternative applications are applications running on an external terminal device, such as telephone applications, SMS applications, instant messaging applications, etc., executed on the external terminal device. Clicking the identifier of one of the alternative applications displayed in the function interface allows the application running on the external terminal device to be invoked. The specific implementation will be described in detail in subsequent steps.

[0056] Step S1014: In response to the triggering operation of the identifier of the candidate application, determine the target application and render the corresponding target terminal interface in the first target area in the extended reality space. The target terminal interface is used to display the target application running on the external terminal device.

[0057] For example, the alternative applications displayed in the functional interface can be invoked through corresponding functional components. Specifically, clicking the identifier of an alternative application triggers the corresponding functional component. After the functional component is triggered, a remote invocation method is executed to control the external terminal to execute the corresponding alternative application. In this embodiment, firstly, based on the identifier of the alternative application targeted by the triggering operation, the alternative application selected by the triggering operation is determined as the target application, and the corresponding functional component is triggered. Then, the remote invocation method is executed to control the external terminal to run the target application, and the target terminal interface for displaying the target application is rendered in the first target area within the extended reality space.

[0058] Figure 8 This is a schematic diagram illustrating a process for displaying a target terminal interface based on a trigger operation, as provided in an embodiment of this disclosure. Figure 8 As shown, the functional interface includes application icons for multiple candidate applications arranged in a ring. Each application icon corresponds to a component. When the user selects different application icons using the virtual reality controller, the corresponding target terminal interface will be displayed in the first target area, thus enabling the invocation of applications running on the external terminal from the functional interface of the extended reality space. (Reference) Figure 8 As shown, when application identifier A is clicked, the terminal interface View1 corresponding to application A is displayed in the first target area; when application identifier B is clicked, the terminal interface View2 corresponding to application B is displayed in the first target area.

[0059] There are several ways to render the target terminal interface in the first target area of ​​the extended reality space. One possible way is that after the virtual reality device detects the trigger operation and determines the target application, it establishes an interface projection data channel with the external terminal device. Then, the external terminal device projects the application interface of the target application (used to display application messages) onto the first target area of ​​the extended reality space for rendering, thereby generating the target terminal interface displayed in the extended reality space.

[0060] In another possible approach, for example, after the virtual reality device detects a trigger operation and identifies the target application, it reconstructs and renders the target application's interface layout within a first target area based on preset interface description information of the target application. Then, or simultaneously, it sends a request for the target application to an external terminal device, obtains the application message from the target application through a messaging system, and uses the application message to populate data, thereby generating the target terminal interface displayed in the extended reality space. For example, the target application displays application messages within the first interface of the external terminal device; the first interface and the target terminal interface have the same interface layout. Through the interface description information, accurate simulation of the first interface of the target application is achieved, improving the consistency between the terminal interface in the extended reality space and the first interface of the target application running on the external terminal device, thus enhancing visual immersion.

[0061] In this embodiment, by responding to the first user operation to display the function interface, and based on the trigger operation for the function interface, the target terminal interface is rendered, realizing the external function call based on user needs in the extended reality space, improving the operational flexibility and convenience when calling external terminal device applications in the extended reality space.

[0062] S102: In response to a first user operation on the extended reality space, display the call result of the target application in the terminal interface, wherein the target application is an application running on an external terminal device.

[0063] For example, a terminal interface is displayed in an extended reality space, and the user can further operate the terminal interface based on the content displayed on the terminal interface, thereby realizing interaction with external terminal devices.

[0064] In one possible implementation process, such as Figure 9 As shown, the specific implementation steps of step S102 include:

[0065] Step S1021: In response to a trigger operation targeting a target function identifier within the terminal interface, a call request is sent to an external terminal device. The call request includes input parameters for executing the target function of the target application.

[0066] Step S1022: Receive the call result corresponding to the call request returned by the external terminal device, and display the call result in the first target area.

[0067] Specifically, the terminal interface is used to display the application interface corresponding to the target application. The application interface has at least one function identifier, and each function identifier corresponds to a function component. For example, in an instant messaging application, the "send" identifier corresponds to the function component of sending a message; the "emoji" identifier corresponds to the function component of displaying the emoji page; and the "letter identifiers" in the input method area correspond to the function components of inputting letters in the input method.

[0068] Furthermore, a user, for example, triggers a button on the terminal interface within the extended reality space using a virtual reality controller. In response, the virtual reality device sends a corresponding call request to the external terminal device. This call request includes input parameters for executing the target function of the target application, such as the component identifier of the triggered function and the type of trigger action. Upon receiving the call request, the external terminal device responds by executing the target function of the target application, such as sending a message, displaying an emoji page, or inputting subtitles, as in the example above. The execution result of this target function, i.e., the call result, is shown as a notification of successful message sending or an emoji page. The external terminal device returns this call result to the virtual reality device, which then displays it on its terminal interface to demonstrate the result of the external terminal device responding to the trigger operation. Visually, the external terminal device is projected into the extended reality space. The user can control the external terminal device based on operations performed on the terminal interface within the extended reality space, thus achieving interaction between the virtual reality device and the external terminal device.

[0069] Depending on the implementation method of the terminal interface, the call result displayed on the terminal interface can be the actual content displayed on the display interface of the target application running on the external terminal device. This method can be achieved by projecting the display interface of the target application running on the external terminal device. Alternatively, it can be simulated content generated based on the content displayed on the display interface of the target application running on the external terminal device. This method can be achieved by reconstructing the display interface of the target application based on the application layout of the target application and the application messages sent by the external terminal device. For a detailed description of the implementation process, please refer to the detailed steps of rendering the target terminal interface in the above embodiments, which will not be repeated here.

[0070] In this embodiment, the terminal interface of the external terminal device is displayed in a first target area within the extended reality space. In response to a first user operation on the extended reality space, the call result of the target application is displayed on the terminal interface, where the target application is an application running on the external terminal device. By displaying the terminal interface of the external terminal device within the extended reality space and controlling the target application running on the external terminal device through operations on the extended reality space, the purpose of displaying and controlling the external terminal device within the extended reality space is achieved. This realizes efficient interaction between the virtual reality device and the external terminal device, allowing users to conveniently control the external terminal device while using the virtual reality device, thus improving ease of use.

[0071] refer to Figure 10 , Figure 10 Flowchart of the information interaction method provided in the embodiments of this disclosure Figure 2 This embodiment is in Figure 2 Based on the illustrated embodiment, step S101 is further refined, and a step of determining the first target region is added. This information interaction method includes:

[0072] Step S201: Obtain push information sent by an external terminal device. The push information represents the application message generated by the target application running on the external terminal device.

[0073] Step S202: In the second target area of ​​the extended reality space, a notification interface is displayed. The notification interface is used to display the abbreviated message and / or receipt identifier corresponding to the push information.

[0074] Figure 11 This is a schematic diagram of a notification interface provided in an embodiment of this disclosure. Figure 11 As shown, the second target area is located on the left side of the extended reality space. After receiving a push notification from an external terminal device, the virtual reality device displays an interface within the second target area based on the content of the push message. This interface displays a shortened message and / or a receipt identifier corresponding to the push message; that is, a notification interface. The area of ​​the second target area is smaller than the area of ​​the first target area. More specifically, in one implementation, the notification interface displays a shortened message corresponding to the push message. The shortened message is part of the application message represented by the push message. For example, if the application message includes the Chinese string "I'll go find you this afternoon to get back what belongs to me," then the corresponding shortened message is the string "I'll go find you this afternoon…".

[0075] In another implementation, the notification interface displays a receiving identifier corresponding to the push message. The receiving identifier indicates that a notification of a push message sent by an external terminal device has been received. For example, the receiving identifier is a combination of the application identifier of the target application and a specific identifier. For example, the receiving identifier is a pattern composed of a square icon of the target application and a circle located in the upper right corner of the square icon.

[0076] In this embodiment, by displaying a notification interface in the second target area of ​​the extended reality space, the user is notified that the target application on the external terminal device has sent an application message, preventing the user from missing application messages from the external terminal device. At the same time, displaying the notification message in a smaller second target area avoids directly displaying the terminal interface, which would visually affect the immersion and improve the visual effect.

[0077] In one possible implementation, step S202 is specifically implemented as follows:

[0078] Step S2021: In the second target area of ​​the extended reality space, display the first notification interface, which is used to display the receiving identifier corresponding to the push information.

[0079] Step S2021: In response to the third user's operation, the second notification interface is simultaneously displayed in the third target area of ​​the extended reality space.

[0080] For example, the first notification interface and the second notification interface are two implementations of the notification interface. The area of ​​the first notification interface is smaller than that of the second notification interface. The first notification interface is used to display a receipt identifier, and the second notification interface is used to display a thumbnail message. The display of the second notification interface needs to be triggered based on a third user operation, such as detecting a lift gesture towards the virtual reality controller, then displaying the notification interface and simultaneously displaying the gesture identifier corresponding to the lift gesture in a second target area. Furthermore, the gesture identifier is displayed near the second target area, which is located in the extended reality space on the side closer to the lift gesture.

[0081] Figure 12 This is a schematic diagram of a process for displaying a notification interface provided in an embodiment of the present disclosure, such as... Figure 12As shown, firstly, after the virtual reality device receives a push notification from an external terminal device, it displays a first notification interface in the second target area. This first notification interface displays a receiving identifier, which is the name of the target application, "app_1," indicating that app_1 sent the push message. Next, when the virtual reality device detects a lift-up gesture towards the virtual reality controller, it displays a second notification interface in the third target area. The second notification interface is larger than the first notification interface and displays more information. For example, as shown in the figure, the second notification interface displays a shortened message corresponding to the push message sent by the target application app_1: "I'll come see you this afternoon...". Optionally, when displaying the second notification interface, a gesture identifier can be displayed near the second notification interface to indicate to the user that a lift-up gesture has been applied.

[0082] Furthermore, the gesture icon can be, for example, a wrist icon of a wristwatch, and the second notification interface is a matching watch interface. By displaying matching gesture icons and the second notification interface, the complexity of user operation is reduced and the ease of use is improved.

[0083] In this embodiment, after the virtual reality device receives the push information, it first displays a smaller first notification interface, and then displays a second notification interface based on the user's operation. This achieves hierarchical display of push information sent by external terminal devices. In the context of virtual reality applications, this further reduces the impact of external information access on the visual immersion within the virtual space interface and improves the flexibility and convenience of displaying external information.

[0084] In another possible implementation, step S201 can be implemented by directly displaying the second notification interface without displaying the first notification interface. Specifically, step S201 includes: in response to a third user operation, synchronously displaying the notification interface in the second target area of ​​the extended reality space, and displaying the gesture identifier corresponding to the lift gesture. The third user operation is a lift gesture on the virtual reality controller, and the second target area is located in the extended reality space near the lift gesture. The specific implementation method is similar to the process of displaying the second notification interface in the above steps, and will not be repeated here. Furthermore, in this implementation, the notification interface can be the same as the functional interface described in the previous embodiments. That is, the notification interface is used to display the identifier of at least one alternative application, which is an application running by an external terminal device. For specific implementation methods, please refer to the description of the notification interface in the above embodiments, and will not be repeated here.

[0085] Optionally, after step S202, the method further includes:

[0086] Step S203: Obtain the target image currently displayed in the extended reality space, the target image including at least one display object.

[0087] Step S204: Determine the first target region based on the position of at least one display object in the target image.

[0088] For example, based on the operating system or application currently running on the virtual reality device, a corresponding target image is displayed in the extended reality space. The target image includes at least one display object. Based on the specific content displayed in the target image, the display object can be, for example, a virtual object, a virtual character, or system controls, application icons, etc., displayed on the operating system interface. By parsing and recognizing the target image, the position of each display object can be obtained. Specifically, the position of the display object is, for example, an array in a two-dimensional plane composed of the coordinates of the four corners of the quadrilateral border used to locate the display object; or, an array composed of the coordinates of the center point of the display object and the area of ​​the display object. After obtaining the position of each display object in the target image, an area for displaying the terminal interface, i.e., the first target area, can be determined based on different purposes.

[0089] In one possible implementation, step S204 includes the following steps:

[0090] Step S2041: Obtain the interval distance between each display object, and determine at least one candidate area based on the interval distance between each display object. The candidate area is an area that can accommodate the terminal interface.

[0091] Step S2042: Determine a first target region from at least one candidate region.

[0092] For example, to avoid the terminal interface displayed on the upper layer from obscuring the display objects in the target image, the spacing between each display object is determined based on its position. The larger the spacing, the larger the non-obscuring area between the display objects. If the spacing is greater than a preset spacing threshold, it is considered that the non-obscuring area between two display objects can accommodate the terminal interface, and this non-obscuring area is determined as a candidate area. Further, after determining at least one candidate area, a first target area is selected from the at least one candidate area according to a preset selection rule. The preset selection rule, for example, is to divide the target image into multiple image regions, preferentially select candidate areas located within the edge regions of the target image, and if there are no candidate areas within the edge regions, then select candidate areas located within the center regions of the target image, thereby making the first target area as close as possible to the edge of the extended reality space, thus improving the display effect of the extended reality space.

[0093] In this embodiment, the distance between each displayed object is obtained, and based on this distance, candidate areas that will not cause occlusion are determined. Then, the edge areas of the extended reality space are searched sequentially to determine the first target area. This avoids the terminal interface from obscuring effective information in the target image and improves the visual display effect.

[0094] In another possible implementation, step S204 includes the following steps:

[0095] Step S2043: Perform target recognition on each displayed object to obtain the target recognition object;

[0096] Step S2044: Determine the first target region based on the contour of the target object.

[0097] For example, in this embodiment, image recognition is first performed on each displayed object in the target image to determine the objects belonging to the target object category, i.e., the target recognition objects, such as mobile phones, tablets, tables, photo frames, etc. Then, the outline of the target recognition object is determined as the first target area, realizing the mapping between the target recognition object and the terminal interface. The outline of the target recognition object can be its external shape outline, positioning frame outline, or display screen outline, and so on. For example, application messages sent by a target application on an external terminal device are displayed on a television (target recognition object) in the target image displayed by the virtual reality device, realizing the transmission of external information into the virtual reality space. In this embodiment, by setting the first target area carrying the terminal interface on the target recognition object, smooth interaction between the virtual reality space and the external real space is achieved, improving visual immersion and visual experience.

[0098] Furthermore, the object category of the target recognition object is determined by the external terminal device. More specifically, the object category of the target recognition object is consistent with the external terminal device. The push information sent by the external terminal device and received by the virtual reality device contains the device identifier of the external terminal device. The virtual reality device determines the target object category based on this device identifier, and then determines the target recognition object based on the target object category. For example, if the device identifier indicates that the external terminal device is a smartphone, then the corresponding target object category is determined to be category P, where category P represents mobile phones and tablets. When a target image includes a smartphone or tablet, the first target region is selected and set within the outline of the smartphone or tablet unit. If the device identifier indicates that the external terminal device is a smart TV, then the corresponding target object category is determined to be category T, where category T represents a TV. When a target image includes a TV, the first target region is set within the outline of the TV. Figure 13 This is a schematic diagram of a dynamic first target region provided in an embodiment of the present disclosure, such as... Figure 13 As shown, exemplarily, the virtual reality device communicates with both a smart TV and a smartphone. When the virtual reality device receives a push notification from the smart TV, it identifies the currently displayed target image, determines the outline of the "TV" in the target image as the first target area Zoom1, and displays the smart TV's terminal interface in the extended reality space in the first target area Zoom1 after the trigger condition is met. When the virtual reality device receives a push notification from the smartphone, it identifies the currently displayed target image, determines the outline of the "phone" in the target image as the first target area Zoom2, and displays the smart TV's terminal interface in the extended reality space in the first target area Zoom2 after the trigger condition is met.

[0099] In this embodiment, by identifying the target object that corresponds to the type of the external terminal device, the purpose of matching and displaying the external terminal device with the virtual device in the virtual space is achieved, realizing seamless interaction between the external device and the virtual device in the virtual space, and further improving the visual immersion.

[0100] Furthermore, the method provided in this embodiment can also be applied to mixed reality (MR) scenarios. Mixed reality technology combines virtual reality technology with augmented reality (AR) to integrate images from the real environment into the virtual reality space. Its specific implementation principle is existing technology and will not be elaborated here. Regarding the steps in this embodiment, the target image displayed in the extended reality space includes both real image objects and virtual objects; that is, the display objects include virtual reality objects and real display objects. In one possible implementation, the specific steps for determining the first target area based on the position of at least one display object in the target image include: performing target recognition on each real display object to obtain the target real display object, where the target real display object is the real image of the external terminal device; and determining the first target area based on the outline of the target real display object.

[0101] In this embodiment, for mixed reality application scenarios, during the process of recognizing the target image, image recognition is performed only on the real display object to obtain the target real display object corresponding to the external terminal device. This allows the first target area carrying the target terminal page to be displayed on the outline of the target real display object, thereby realizing further interaction and integration between the external terminal device and the virtual reality device in mixed reality application scenarios and further improving the visual performance of the extended reality space in mixed reality.

[0102] Step S205: In response to a second user operation on the notification interface, render the corresponding target terminal interface in the first target area within the extended reality space.

[0103] For example, after determining the first target area, in response to a second user operation on the notification interface, such as when the user selects a thumbnail message or a receiving identifier in the notification interface using a virtual reality controller, the corresponding target terminal interface will be displayed within the first target area to show the detailed information corresponding to the thumbnail message and the receiving identifier, i.e., the application interface of the target application. This enables the invocation of applications running on external terminals from the functional interface of the extended reality space.

[0104] There are several ways to render the target terminal interface in the first target area of ​​the extended reality space. One possible way is that after the virtual reality device detects the second user's operation, it determines the corresponding target application and establishes an interface projection data channel with the external terminal device. Then, the external terminal device projects the application interface of the target application (used to display application messages) onto the first target area of ​​the extended reality space for rendering, thereby generating the target terminal interface displayed in the extended reality space.

[0105] In another possible approach, for example, after the virtual reality device detects a second user's operation, it identifies the corresponding target application and, based on preset interface description information of the target application, reconstructs and renders the interface layout of the target application within a first target area. Then, or simultaneously, it sends a request for the target application to an external terminal device, obtains the application message from the target application through a messaging system, and uses the application message to populate data, thereby generating a target terminal interface displayed in the extended reality space. For example, the target application displays application messages within a first interface on the external terminal device; the first interface and the target terminal interface have the same interface layout. Through interface description information, accurate simulation of the first interface of the target application is achieved, improving the consistency between the terminal interface in the extended reality space and the first interface of the target application running on the external terminal device, thus enhancing visual immersion.

[0106] Step S206: In response to the first user operation on the extended reality space, display the call result of the target application in the terminal interface, wherein the target application is an application running on an external terminal device.

[0107] In this embodiment, the specific implementation of steps S201 and S206 is as follows: Figure 2 The corresponding steps in the illustrated embodiments have been described in detail and will not be repeated here.

[0108] Corresponding to the information interaction method in the above embodiments, Figure 14 This is a structural block diagram of the information interaction device 3 provided in an embodiment of this disclosure. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. (Refer to...) Figure 14 The information interaction device 3 includes:

[0109] Display module 31 is used to display the terminal interface of an external terminal device in the first target area within the extended reality space;

[0110] The interaction module 32 is used to respond to a first user operation on the extended reality space and display the call result of the target application in the terminal interface, wherein the target application is an application running on an external terminal device.

[0111] In one embodiment of this disclosure, the display module 31 is specifically used to: acquire push information sent by an external terminal device, the push information representing application messages generated by a target application running on the external terminal device; and, based on the push information, render a corresponding target terminal interface in a first target area within the extended reality space, the target terminal interface being used to display the application messages generated by the target application.

[0112] In one embodiment of this disclosure, when the display module 31 renders the corresponding target terminal interface in the first target area within the extended reality space based on push information, it is specifically used to: display a notification interface in the second target area of ​​the extended reality space, the notification interface being used to display the abbreviated message and / or receiving identifier corresponding to the push information, the area of ​​the second target area being smaller than the area of ​​the first target area; and in response to a second user operation on the notification interface, render the corresponding target terminal interface in the first target area within the extended reality space.

[0113] In one embodiment of this disclosure, when the display module 31 displays a notification interface in the second target area of ​​the extended reality space, it is specifically used to: in response to a third user operation, synchronously display the notification interface in the second target area of ​​the extended reality space, wherein the third user operation is a gesture to lift the virtual reality controller, and the second target area is located on the side of the extended reality space closer to the gesture; the display module 31 further includes: displaying a gesture identifier corresponding to the gesture.

[0114] In one embodiment of this disclosure, the target application displays application messages on a first interface of an external terminal device; the first interface has the same interface layout as the target terminal interface.

[0115] In one embodiment of this disclosure, the display module 31 is specifically configured to: in response to a fourth user operation, display a functional interface in a second target area of ​​the extended reality space, the functional interface being used to display the identifier of at least one candidate application, the candidate application being an application run by an external terminal device; in response to a trigger operation for the identifier of the candidate application, determine a target application, and render a corresponding target terminal interface in a first target area within the extended reality space, the target terminal interface being used to display the target application run by the external terminal device.

[0116] In one embodiment of this disclosure, the interaction module 32 is specifically configured to: in response to a trigger operation targeting a target function identifier within the terminal interface, send a call request to an external terminal device, the call request including input parameters for executing the target function of the target application; receive the call result corresponding to the call request returned by the external terminal device, and display the call result in a first target area.

[0117] In one embodiment of this disclosure, before displaying the terminal interface of an external terminal device in a first target area within the extended reality space, the display module 31 is further configured to: acquire a target image currently displayed in the extended reality space, the target image including at least one display object; and determine the first target area based on the position of the at least one display object in the target image.

[0118] In one embodiment of this disclosure, when the display module 31 determines the first target area based on the position of at least one display object in the target image, it is specifically configured to: obtain the interval distance between each display object; determine at least one candidate area based on the interval distance between each display object, wherein the candidate area is an area capable of accommodating the terminal interface; and determine the first target area from the at least one candidate area.

[0119] In one embodiment of this disclosure, when the display module 31 determines the first target region based on the position of at least one display object in the target image, it is specifically used to: perform target recognition on each display object to obtain a target recognition object, wherein the object category of the target recognition object is determined based on the terminal category of the external terminal device; and determine the first target region based on the outline of the target recognition object.

[0120] In one embodiment of this disclosure, the display objects include virtual reality objects and real display objects. When the display module 31 determines the first target area based on the position of at least one display object in the target image, it is specifically used to: perform target recognition on each real display object to obtain a target real display object, wherein the target real display object is a real image of an external terminal device; and determine the first target area based on the outline of the target real display object.

[0121] The apparatus provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0122] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as... Figure 15 As shown, the electronic device 4 includes:

[0123] Processor 41, and memory 42 communicatively connected to processor 41;

[0124] Memory 42 stores instructions executed by the computer;

[0125] The processor 41 executes computer execution instructions stored in the memory 42 to achieve, for example, Figures 2-13 The information interaction method in the illustrated embodiment.

[0126] Optionally, the processor 41 and the memory 42 are connected via a bus 43.

[0127] For relevant instructions, please refer to the corresponding text. Figures 2-13 The relevant descriptions and effects of the steps in the corresponding embodiments are understood, and will not be elaborated on here.

[0128] refer to Figure 16 The diagram illustrates a structural schematic of an electronic device 900 suitable for implementing embodiments of the present disclosure. The electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 16 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0129] like Figure 16 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0130] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 16 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0131] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0132] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0133] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0134] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0135] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0137] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0138] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0139] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0140] In a first aspect, according to one or more embodiments of this disclosure, an information interaction method is provided, comprising:

[0141] In a first target area within the extended reality space, the terminal interface of an external terminal device is displayed; in response to a first user operation on the extended reality space, the invocation result of a target application is displayed in the terminal interface, wherein the target application is an application running on the external terminal device.

[0142] According to one or more embodiments of this disclosure, displaying the terminal interface of an external terminal device in a first target area within an extended reality space includes: acquiring push information sent by the external terminal device, the push information representing application messages generated by a target application running on the external terminal device; and rendering a corresponding target terminal interface in the first target area within the extended reality space based on the push information, the target terminal interface being used to display the application messages generated by the target application.

[0143] According to one or more embodiments of this disclosure, rendering a corresponding target terminal interface in a first target area within the extended reality space based on the push information includes: displaying a notification interface in a second target area within the extended reality space, the notification interface being used to display a thumbnail message and / or a receiving identifier corresponding to the push information, the area of ​​the second target area being smaller than the area of ​​the first target area; and rendering a corresponding target terminal interface in the first target area within the extended reality space in response to a second user operation on the notification interface.

[0144] According to one or more embodiments of this disclosure, displaying a notification interface in the second target area of ​​the extended reality space includes: synchronously displaying the notification interface in the second target area of ​​the extended reality space in response to a third user operation, wherein the third user operation is a raise gesture for a virtual reality controller, and the second target area is located on the side of the extended reality space closer to the raise gesture; the method further includes: displaying a gesture identifier corresponding to the raise gesture.

[0145] According to one or more embodiments of this disclosure, the target application displays the application message within a first interface of the external terminal device; the first interface has the same interface layout as the target terminal interface.

[0146] According to one or more embodiments of this disclosure, displaying the terminal interface of an external terminal device in a first target area within an extended reality space includes: in response to a fourth user operation, displaying a functional interface in a second target area within the extended reality space, the functional interface being used to display the identifier of at least one alternative application, the alternative application being an application run by the external terminal device; in response to a trigger operation for the identifier of the alternative application, determining the target application, and rendering a corresponding target terminal interface in the first target area within the extended reality space, the target terminal interface being used to display the target application run by the external terminal device.

[0147] According to one or more embodiments of this disclosure, displaying the invocation result of a target application in the terminal interface in response to a first user operation on the extended reality space includes: sending an invocation request to the external terminal device in response to a trigger operation on a target function identifier in the terminal interface, the invocation request including input parameters for executing the target function of the target application; receiving the invocation result corresponding to the invocation request returned by the external terminal device, and displaying the invocation result in the first target area.

[0148] According to one or more embodiments of this disclosure, before displaying the terminal interface of an external terminal device in the first target area within the extended reality space, the method further includes: acquiring a target image currently displayed within the extended reality space, the target image including at least one display object; and determining the first target area based on the position of at least one display object in the target image.

[0149] According to one or more embodiments of this disclosure, determining the first target region based on the position of at least one of the display objects in the target image includes: obtaining the interval distance between each of the display objects; determining at least one candidate region based on the interval distance between each of the display objects, the candidate region being a region capable of accommodating the terminal interface; and determining the first target region from the at least one candidate region.

[0150] According to one or more embodiments of this disclosure, determining the first target region based on the position of at least one of the display objects in the target image includes: performing target recognition on each of the display objects to obtain target recognition objects, wherein the object category of the target recognition objects is determined based on the terminal category of the external terminal device; and determining the first target region based on the outline of the target recognition objects.

[0151] According to one or more embodiments of this disclosure, the display object includes a virtual reality object and a real display object. Determining the first target region based on the position of at least one of the display objects in the target image includes: performing target recognition on each of the real display objects to obtain a target real display object, wherein the target real display object is a real image of the external terminal device; and determining the first target region based on the outline of the target real display object.

[0152] Secondly, according to one or more embodiments of this disclosure, an information interaction device is provided, comprising:

[0153] The display module is used to display the terminal interface of an external terminal device in the first target area within the extended reality space.

[0154] An interaction module is used to respond to a first user operation on the extended reality space and display the invocation result of a target application within the terminal interface, wherein the target application is an application running on the external terminal device.

[0155] According to one or more embodiments of this disclosure, the display module is specifically configured to: acquire push information sent by the external terminal device, the push information representing application messages generated by a target application running on the external terminal device; and render a corresponding target terminal interface in a first target area within the extended reality space based on the push information, the target terminal interface being used to display the application messages generated by the target application.

[0156] According to one or more embodiments of this disclosure, when the display module renders the corresponding target terminal interface in the first target area within the extended reality space based on the push information, it is specifically configured to: display a notification interface in the second target area of ​​the extended reality space, the notification interface being used to display a thumbnail message and / or a receiving identifier corresponding to the push information, the area of ​​the second target area being smaller than the area of ​​the first target area; and render the corresponding target terminal interface in the first target area within the extended reality space in response to a second user operation on the notification interface.

[0157] According to one or more embodiments of this disclosure, when the display module displays a notification interface in the second target area of ​​the extended reality space, it is specifically configured to: in response to a third user operation, synchronously display the notification interface in the second target area of ​​the extended reality space, wherein the third user operation is a raise gesture for a virtual reality controller, and the second target area is located in the extended reality space on the side closer to the raise gesture; the display module further includes: displaying a gesture identifier corresponding to the raise gesture.

[0158] According to one or more embodiments of this disclosure, the target application displays the application message within a first interface of the external terminal device; the first interface has the same interface layout as the target terminal interface.

[0159] According to one or more embodiments of this disclosure, the display module is specifically configured to: in response to a fourth user operation, display a functional interface in a second target area of ​​the extended reality space, the functional interface being used to display the identifier of at least one alternative application, the alternative application being an application run by the external terminal device; in response to a trigger operation for the identifier of the alternative application, determine the target application, and render a corresponding target terminal interface in a first target area within the extended reality space, the target terminal interface being used to display the target application run by the external terminal device.

[0160] According to one or more embodiments of this disclosure, the interaction module is specifically configured to: in response to a trigger operation on a target function identifier within the terminal interface, send a call request to the external terminal device, the call request including input parameters for executing the target function of the target application; receive a call result corresponding to the call request returned by the external terminal device, and display the call result in the first target area.

[0161] According to one or more embodiments of this disclosure, before displaying the terminal interface of an external terminal device in a first target area within an extended reality space, the display module is further configured to: acquire a currently displayed target image within the extended reality space, the target image including at least one display object; and determine the first target area based on the position of at least one display object in the target image.

[0162] According to one or more embodiments of this disclosure, when the display module determines the first target area based on the position of at least one of the display objects in the target image, it is specifically configured to: obtain the interval distance between each of the display objects; determine at least one candidate area based on the interval distance between each of the display objects, the candidate area being an area capable of accommodating the terminal interface; and determine the first target area from the at least one candidate area.

[0163] According to one or more embodiments of this disclosure, when the display module determines the first target region based on the position of at least one of the display objects in the target image, it is specifically configured to: perform target recognition on each of the display objects to obtain target recognition objects, wherein the object category of the target recognition objects is determined based on the terminal category of the external terminal device; and determine the first target region based on the outline of the target recognition objects.

[0164] According to one or more embodiments of this disclosure, the display object includes a virtual reality object and a real display object. When the display module determines the first target region based on the position of at least one of the display objects in the target image, it is specifically used to: perform target recognition on each of the real display objects to obtain a target real display object, wherein the target real display object is a real image of the external terminal device; and determine the first target region based on the outline of the target real display object.

[0165] Thirdly, according to one or more embodiments of the present disclosure, an electronic device is provided, including: a processor, and a memory communicatively connected to the processor;

[0166] The memory stores computer-executed instructions;

[0167] The processor executes computer execution instructions stored in the memory to implement the information interaction method described in the first aspect and various possible designs of the first aspect.

[0168] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, the information interaction method described in the first aspect and various possible designs of the first aspect is implemented.

[0169] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the information interaction method described in the first aspect and various possible designs of the first aspect.

[0170] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0171] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0172] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An information exchange method, characterized in that, include: In the first target area within the extended reality space, the terminal interface of the external terminal device is displayed; In response to a first user operation on the extended reality space, the result of the call to the target application is displayed in the terminal interface, wherein the target application is an application running on the external terminal device. The first target area within the extended reality space displays the terminal interface of an external terminal device, including: In response to a fourth user action, a functional interface is displayed in the second target area of ​​the extended reality space. The functional interface is used to display the identifier of at least one alternative application, which is an application running by the external terminal device. In response to a trigger operation for the identifier of the candidate application, the target application is determined, and the corresponding target terminal interface is rendered in a first target area within the extended reality space. The target terminal interface is used to display the target application running on the external terminal device.

2. The method according to claim 1, characterized in that, The first target area within the extended reality space displays the terminal interface of an external terminal device, including: Obtain push information sent by the external terminal device, wherein the push information represents application messages generated by the target application running on the external terminal device; Based on the push information, a corresponding target terminal interface is rendered in the first target area within the extended reality space. The target terminal interface is used to display application messages generated by the target application.

3. The method according to claim 2, characterized in that, The step of rendering the corresponding target terminal interface in the first target area within the extended reality space based on the push information includes: In the second target area of ​​the extended reality space, a notification interface is displayed. The notification interface is used to display the abbreviated message and / or the receiving identifier corresponding to the push information. The area of ​​the second target area is smaller than the area of ​​the first target area. In response to a second user action on the notification interface, a corresponding target terminal interface is rendered in the first target area within the extended reality space.

4. The method according to claim 3, characterized in that, In the second target area of ​​the extended reality space, a notification interface is displayed, including: In response to a third user action, the notification interface is simultaneously displayed in a second target area of ​​the extended reality space, wherein the third user action is a gesture to lift the virtual reality controller, and the second target area is located in the extended reality space on the side closer to the gesture. The method further includes: Display the gesture icon corresponding to the raised gesture.

5. The method according to claim 2, characterized in that, The target application displays the application message on the first interface of the external terminal device; the first interface has the same interface layout as the target terminal interface.

6. The method according to claim 1, characterized in that, The step of displaying the invocation result of the target application within the terminal interface in response to a first user operation on the extended reality space includes: In response to a trigger operation targeting a target function identifier within the terminal interface, a call request is sent to the external terminal device, the call request including input parameters for executing the target function of the target application; Receive the call result corresponding to the call request returned by the external terminal device, and display the call result in the first target area.

7. The method according to any one of claims 1-5, characterized in that, Before displaying the terminal interface of the external terminal device in the first target area within the extended reality space, the method further includes: Obtain the target image currently displayed within the extended reality space, wherein the target image includes at least one display object; The first target region is determined based on the position of at least one of the display objects in the target image.

8. The method according to claim 7, characterized in that, Determining the first target region based on the position of at least one of the display objects in the target image includes: Obtain the interval distance between each of the displayed objects; Based on the spacing between each of the display objects, at least one candidate area is determined, wherein the candidate area is an area that can accommodate the terminal interface; The first target region is determined from the at least one candidate region.

9. The method according to claim 7, characterized in that, Determining the first target region based on the position of at least one of the display objects in the target image includes: Target recognition is performed on each of the displayed objects to obtain target recognition objects, and the object category of the target recognition objects is determined based on the terminal category of the external terminal device; The first target region is determined based on the outline of the target object.

10. The method according to claim 7, characterized in that, The display objects include virtual reality objects and real display objects. Determining the first target region based on the position of at least one of the display objects in the target image includes: Target recognition is performed on each of the real display objects to obtain the target real display object, wherein the target real display object is the real image of the external terminal device; The first target region is determined based on the outline of the target real-world object.

11. An information interaction device, characterized in that, include: The display module is used to display the terminal interface of an external terminal device in the first target area within the extended reality space. An interaction module is used to respond to a first user operation on the extended reality space and display the invocation result of a target application within the terminal interface, wherein the target application is an application running on the external terminal device. The display module is specifically used to respond to a fourth user operation and display a functional interface in the second target area of ​​the extended reality space. The functional interface is used to display the identifier of at least one alternative application, which is an application running by the external terminal device. In response to a trigger operation for the identifier of the candidate application, the target application is determined, and the corresponding target terminal interface is rendered in a first target area within the extended reality space. The target terminal interface is used to display the target application running on the external terminal device.

12. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the information interaction method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the information interaction method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the information interaction method according to any one of claims 1 to 10.

Images