Split-screen display method, and head-mounted display device and computer-readable medium

Abstract

Disclosed in the embodiments of the present disclosure are a split-screen display method, and a head-mounted display device and a computer-readable medium. A specific implementation of the method comprises: in response to having detected starting information of a corresponding two-dimensional application, displaying a main application page of the two-dimensional application; and in response to having detected a page navigation selection operation on the main application page, and when the page navigation selection operation meets a preset split-screen condition, executing the following split-screen steps: monitoring the page navigation selection operation; changing attribute information of an application page corresponding to the page navigation selection operation, so as to obtain an application page that has been subjected to change processing; and displaying, in a virtual sub-screen in a three-dimensional virtual space, the application page that has been subjected to change processing. In the implementation, a plurality of application pages of one two-dimensional application can be simultaneously displayed, and operation steps for displaying the application pages can be simplified, thereby shortening the time consumed for displaying the application pages and reducing the consumed computing resources.

Description

Split-screen display methods, head-mounted display devices, and computer-readable media

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411580377.9, filed on November 6, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The embodiments disclosed herein relate to the field of virtual vision technology, and more specifically to split-screen display methods, head-mounted display devices, and computer-readable media. Background Technology

[0004] Using two-dimensional (2D) applications in an Augmented Reality (AR) spatial operating system can enhance the user experience of 2D applications. Currently, the common approach to displaying 2D applications in an AR spatial operating system is to use a page-chaining design. First, the main application page of the 2D application is opened in the AR spatial operating system. Then, when the user needs to open other application pages of the 2D application, they need to exit the currently displayed application page before the other application page is displayed. Therefore, multiple application pages of the same 2D application cannot be displayed simultaneously in the AR spatial operating system.

[0005] However, in practice, it has been found that when displaying the application pages of 2D applications using the above method, the following technical problems often occur:

[0006] When displaying application pages of a 2D application using a page-chain design, it is impossible to display multiple application pages of a single 2D application simultaneously, resulting in a poor user experience. Furthermore, when a user needs to open other application pages of a 2D application, they must exit the currently displayed application page before the other application page can be displayed, making the process of displaying application pages cumbersome. This leads to longer display times and higher computational resource consumption for displaying application pages of a 2D application.

[0007] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not form prior art known to those skilled in the art. Summary of the Invention

[0008] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion later. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0009] Some embodiments of this disclosure provide a split-screen display method, a head-mounted display device, and a computer-readable medium to address one or more of the technical problems mentioned in the background section above.

[0010] In a first aspect, some embodiments of this disclosure provide a split-screen display method applied to a head-mounted display device, comprising: in response to detecting startup information of a corresponding two-dimensional application, displaying the main application page of the two-dimensional application in a main virtual screen within a three-dimensional virtual space, wherein the three-dimensional virtual space is deployed in the head-mounted display device; in response to detecting a page jump selection operation corresponding to the main application page, and the page jump selection operation satisfies preset split-screen conditions, performing the following split-screen steps: listening to the page jump selection operation; modifying the attribute information of the application page corresponding to the page jump selection operation to obtain a modified application page, wherein the attribute information includes a source window identifier attribute and an activity identifier attribute; displaying the modified application page in a sub-virtual screen within the three-dimensional virtual space, wherein multiple application pages of the two-dimensional application can be displayed simultaneously on various virtual screens.

[0011] Optionally, before displaying the main application page of the two-dimensional application on the main virtual screen within the three-dimensional virtual space, the method further includes: updating the three-dimensional virtual space deployed in the head-mounted display device to obtain an updated three-dimensional virtual space as the three-dimensional virtual space, wherein a preset number of virtual screens are created within the updated three-dimensional virtual space; and determining the virtual screen corresponding to the main application page among the preset number of virtual screens in the updated three-dimensional virtual space as the main virtual screen.

[0012] Optionally, updating the three-dimensional virtual space deployed in the head-mounted display device to obtain the updated three-dimensional virtual space includes: creating a preset number of texture images in the three-dimensional virtual space; and creating a preset number of two-dimensional rectangular canvases based on the preset number of texture images, wherein the texture images in the preset number of texture images correspond to the two-dimensional rectangular canvases in the preset number of two-dimensional rectangular canvases.

[0013] Optionally, after creating a preset number of two-dimensional rectangular canvases based on the preset number of texture images, the method further includes: creating a preset number of virtual screens in the three-dimensional virtual space based on a preset set of application pages to be displayed corresponding to the two-dimensional application; and determining the preset number of two-dimensional rectangular canvases as rendering targets corresponding to the preset number of virtual screens, wherein the two-dimensional rectangular canvases in the preset number of two-dimensional rectangular canvases correspond to the virtual screens in the preset number of virtual screens.

[0014] Optionally, before displaying the modified application page in the sub-virtual screen within the three-dimensional virtual space, the method further includes: determining the virtual screen corresponding to the modified application page among a preset number of virtual screens within the three-dimensional virtual space as a sub-virtual screen.

[0015] Optionally, the step of modifying the attribute information of the application page corresponding to the page jump selection operation to obtain the modified application page includes: setting the attribute value of the source window identifier attribute included in the attribute information to empty; and setting the attribute value of the activity identifier attribute included in the attribute information to the new task activity attribute value.

[0016] In a second aspect, some embodiments of this disclosure provide a head-mounted display device, including: one or more processors; a storage device for storing one or more programs; a display screen for imaging in front of a user; and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect.

[0017] Thirdly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any implementation of the first aspect.

[0018] The above-described embodiments of this disclosure have the following beneficial effects: Through a split-screen display method according to some embodiments of this disclosure, multiple application pages of a 2D application can be displayed simultaneously, thereby improving the user experience and simplifying the operation steps for displaying application pages, thus shortening the time spent displaying application pages of a 2D application and reducing the computational resources consumed. The reason why it is impossible to display multiple application pages of a 2D application simultaneously, resulting in a poor user experience, cumbersome operation steps for displaying application pages, long display time for 2D application pages, and high computational resource consumption, is that: when displaying application pages of a 2D application according to a page chain design, it is impossible to display multiple application pages of a 2D application simultaneously, resulting in a poor user experience; and when the user needs to open other application pages of the 2D application, they must exit the currently displayed application page of the 2D application before displaying the other application page, making the operation steps for displaying application pages cumbersome, leading to long display time for 2D application pages and high computational resource consumption. Based on this, some embodiments of the split-screen display method disclosed herein firstly, in response to detecting the launch information of a corresponding two-dimensional application, displays the main application page of the two-dimensional application in a main virtual screen within a three-dimensional virtual space, wherein the three-dimensional virtual space is deployed on the head-mounted display device. Thus, the main application page of the two-dimensional application can be displayed on the virtual screen. Then, in response to detecting a page jump selection operation corresponding to the main application page, and the page jump selection operation satisfies preset split-screen conditions, the following split-screen steps are executed: First, the page jump selection operation is monitored. Thus, the callback of the page jump selection operation being opened can be monitored. Then, the attribute information of the application page corresponding to the page jump selection operation is modified to obtain the modified application page. The attribute information may include a source window identifier attribute and an activity identifier attribute. Thus, by modifying the source window identifier attribute and activity identifier attribute of the application page, the association between the page to be displayed (the application page corresponding to the page jump selection operation) and the source page of the page to be displayed (the starting page of the page to be displayed) can be severed, thereby further enabling the simultaneous display of multiple application pages of the two-dimensional application. Finally, the modified application page is displayed on the sub-virtual screens within the three-dimensional virtual space. Multiple application pages of the two-dimensional application can be displayed simultaneously on various virtual screens. This allows for the simultaneous display of various application pages corresponding to the two-dimensional application on different virtual screens.Because the display method does not follow a chained approach for the application pages of the 2D application, but instead severs the association between the application page to be displayed and its source page by modifying the attribute values ​​of the source window identifier and the activity identifier of the page to be displayed, the various application pages of the 2D application can be displayed simultaneously on various virtual screens. Furthermore, when displaying other application pages of the 2D application, it is not necessary to exit the currently displayed application page before displaying other application pages; instead, other application pages of the 2D application are displayed directly on other virtual screens. This simplifies the operation steps for displaying application pages, shortens the time spent displaying application pages, and reduces the consumption of computing resources. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0020] Figure 1 is an exemplary system architecture diagram of a split-screen display method applying some embodiments of the present disclosure.

[0021] Figure 2 is a flowchart of some embodiments of the split-screen display method according to the present disclosure.

[0022] Figure 3 is a flowchart of some other embodiments of the split-screen display method according to the present disclosure.

[0023] Figure 4 is a schematic diagram of an application scenario of the split-screen display method according to this disclosure.

[0024] Figure 5 is a schematic diagram of the hardware structure of a head-mounted display device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0027] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0028] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0029] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0030] Figure 1 illustrates an exemplary system architecture 100 to which the split-screen display method of this disclosure for head-mounted display devices can be applied.

[0031] As shown in Figure 1, the exemplary system architecture 100 may include a head-mounted display device 11 and a terminal device 12.

[0032] The head-mounted display device 11 may include one or two display screens 111. The display screens 111 are used to display a virtual interface. Additionally, the head-mounted display device 11 includes a frame 112. In some embodiments, the sensors, processing unit, memory, and battery of the head-mounted display device 11 may be housed inside the frame 112. In some optional implementations of some embodiments, one or more components of the sensors, processing unit, memory, and battery may be integrated into another separate accessory (not shown), connected to the frame 112 via a data cable 13. In some optional implementations of some embodiments, the head-mounted display device 11 may only have display functionality and some sensors, while the terminal device 12 provides data processing, data storage, power supply, and other capabilities.

[0033] Terminal device 12 may include a touch screen 121, which can be used to display content. In some embodiments, head-mounted display device 11 and terminal device 12 can be connected via data cable 13.

[0034] It should be understood that the number of head-mounted display devices 11 and terminal devices 12 in Figure 1 is merely illustrative. Any suitable number of head-mounted display devices and terminal devices can be used depending on the implementation requirements.

[0035] Referring further to FIG2, a flow 200 of some embodiments of the split-screen display method according to the present disclosure is shown. The split-screen display method includes the following steps:

[0036] Step 201: In response to detecting the launch information of the corresponding two-dimensional application, the main application page of the two-dimensional application is displayed in the main virtual screen in the three-dimensional virtual space.

[0037] In some embodiments, in response to detecting startup information of a corresponding 2D application, the executing entity of the split-screen display method (e.g., a head-mounted display device) can display the main application page of the 2D application in a main virtual screen within a 3D virtual space. The 3D virtual space can be deployed on the head-mounted display device. The 2D application can be any 2D application. The main application page can be the homepage of the 2D application. The 3D virtual space can be a Unity 3D space. The startup information indicates that the 2D application has been launched. The main virtual screen can represent a virtual screen used to display the main application page of the 2D application. The main virtual screen can be a virtual screen created in the 3D virtual space. The head-mounted display device can have a Unity application deployed on it. A preset number of virtual screens corresponding to the 2D application are created within the 3D virtual space. All of the preset number of virtual screens created within the 3D virtual space can be used to display the application page of the 2D application. Here, the specific value of the preset number is not limited; for example, the preset number can be 3.

[0038] Optionally, before step 201, the executing entity may also determine the virtual screen corresponding to the main application page among a preset number of virtual screens in the three-dimensional virtual space as the main virtual screen.

[0039] Optionally, after step 201, the executing entity may also synchronize the main application page of the two-dimensional application displayed in the main virtual screen to the three-dimensional virtual space.

[0040] Step 202: In response to detecting a page navigation selection operation on the corresponding main application page, and the page navigation selection operation meets the preset split-screen conditions, the following split-screen steps are executed:

[0041] Step 2021: Listen for and process page navigation selection operations.

[0042] In some embodiments, the executing entity can listen for and process page navigation selection operations. The page navigation selection operation corresponding to the main application page can represent an operation of navigating from the currently displayed application page to any application page derived from the main application page. The preset split-screen condition can be that there is a virtual screen within the three-dimensional virtual space for displaying the application page corresponding to the page navigation selection operation. In practice, the executing entity can listen for callbacks that trigger the page navigation selection operation within the framework of the terminal device's operating system (e.g., Android system).

[0043] It should be noted that each application page navigated from the main application page (the source page of each application page is the main application page), and each target application page navigated from any of the application pages (the source page of the target application page can be any of the application pages), can be an application page derived from the main application page. For example, if the application page navigated from the main application page is a chat window page, and the application page navigated from the chat window page is a video call page, then both the chat window page and the video call page can be application pages derived from the main application page.

[0044] Step 2022: Modify the attribute information of the application page corresponding to the page jump selection operation to obtain the modified application page.

[0045] In some embodiments, the executing entity may modify the attribute information of the application page corresponding to the page jump selection operation to obtain a modified application page. The attribute information may include a source window identifier attribute and an activity identifier attribute. The attribute information can characterize the attributes of the application page. The source window identifier attribute can characterize the SourceRecord attribute of the Activity launched in the terminal device's operating system (e.g., Android system). The activity identifier attribute can characterize the LaunchFlag attribute.

[0046] In some optional implementations of certain embodiments, the executing entity may modify the attribute information of the application page corresponding to the page jump selection operation through the following steps to obtain the modified application page:

[0047] The first step is to set the attribute value of the source window identifier attribute, which is included in the attribute information, to empty.

[0048] The second step is to determine the attribute value of the activity identifier attribute included in the attribute information as the new task activity attribute value. The new task activity attribute value can represent FLAG_ACTIVITY_NEW_TASK.

[0049] It should be noted that an empty value for the SourceRecord attribute of the launched Activity indicates that the application page corresponding to the page navigation selection operation is the first page opened, and the application page corresponding to the page navigation selection operation has no source page. This severs the association between the application page corresponding to the page navigation selection operation and its source page. A value of FLAG_ACTIVITY_NEW_TASK for the LaunchFlag attribute indicates that the application page corresponding to the page navigation selection operation is a newly launched activity, thus enabling the simultaneous display of multiple application pages of the same two-dimensional application.

[0050] Step 2023: Display the modified application page in the sub-virtual screen within the three-dimensional virtual space.

[0051] In some embodiments, the executing entity can display the modified application page in sub-virtual screens within the three-dimensional virtual space. Multiple application pages of the two-dimensional application can be displayed simultaneously on various virtual screens. The sub-virtual screens can represent the virtual screens displayed in the three-dimensional virtual space.

[0052] Optionally, before step 2023, the executing entity may determine the virtual screen corresponding to the modified application page among a preset number of virtual screens in the three-dimensional virtual space as a sub-virtual screen. It should be noted that the virtual screen corresponding to the modified application page refers to the virtual screen used to display the application page (the application page before modification) corresponding to the page jump selection operation.

[0053] Optionally, after step 2023, the executing entity can synchronize the modified application page displayed in the sub-virtual screen to the three-dimensional virtual space.

[0054] The above-described embodiments of this disclosure have the following beneficial effects: Through a split-screen display method according to some embodiments of this disclosure, multiple application pages of a 2D application can be displayed simultaneously, thereby improving the user experience and simplifying the operation steps for displaying application pages, thus shortening the time spent displaying application pages of a 2D application and reducing the computational resources consumed. The reason why it is impossible to display multiple application pages of a 2D application simultaneously, resulting in a poor user experience, cumbersome operation steps for displaying application pages, long display time for 2D application pages, and high computational resource consumption, is that: when displaying application pages of a 2D application according to a page chain design, it is impossible to display multiple application pages of a 2D application simultaneously, resulting in a poor user experience; and when the user needs to open other application pages of the 2D application, they must exit the currently displayed application page of the 2D application before displaying the other application page, making the operation steps for displaying application pages cumbersome, leading to long display time for 2D application pages and high computational resource consumption. Based on this, some embodiments of the split-screen display method disclosed herein firstly, in response to detecting the launch information of a corresponding two-dimensional application, displays the main application page of the two-dimensional application in a main virtual screen within a three-dimensional virtual space, wherein the three-dimensional virtual space is deployed on the head-mounted display device. Thus, the main application page of the two-dimensional application can be displayed on the virtual screen. Then, in response to detecting a page jump selection operation corresponding to the main application page, and the page jump selection operation satisfies preset split-screen conditions, the following split-screen steps are executed: First, the page jump selection operation is monitored. Thus, the callback of the page jump selection operation being opened can be monitored. Then, the attribute information of the application page corresponding to the page jump selection operation is modified to obtain the modified application page. The attribute information may include a source window identifier attribute and an activity identifier attribute. Thus, by modifying the source window identifier attribute and activity identifier attribute of the application page, the association between the page to be displayed (the application page corresponding to the page jump selection operation) and the source page of the page to be displayed (the starting page of the page to be displayed) can be severed, thereby further enabling the simultaneous display of multiple application pages of the two-dimensional application. Finally, the modified application page is displayed on the sub-virtual screens within the three-dimensional virtual space. Multiple application pages of the two-dimensional application can be displayed simultaneously on various virtual screens. This allows for the simultaneous display of various application pages corresponding to the two-dimensional application on different virtual screens.Because the display method does not follow a chained approach for the application pages of the 2D application, but instead severs the association between the application page to be displayed and its source page by modifying the attribute values ​​of the source window identifier and the activity identifier of the page to be displayed, the various application pages of the 2D application can be displayed simultaneously on various virtual screens. Furthermore, when displaying other application pages of the 2D application, it is not necessary to exit the currently displayed application page before displaying other application pages; instead, other application pages of the 2D application are displayed directly on other virtual screens. This simplifies the operation steps for displaying application pages, shortens the time spent displaying application pages, and reduces the consumption of computing resources.

[0055] Referring further to Figure 3, a flow 300 of another embodiment of the split-screen display method is shown. Flow 300 of this split-screen display method includes the following steps:

[0056] Step 301: In response to detecting the startup information of the corresponding two-dimensional application, update the three-dimensional virtual space deployed in the head-mounted display device to obtain the updated three-dimensional virtual space as the three-dimensional virtual space.

[0057] In some embodiments, in response to detecting the launch information of a corresponding two-dimensional application, the executing entity of the split-screen display method (e.g., a head-mounted display device) can update the three-dimensional virtual space deployed in the head-mounted display device to obtain the updated three-dimensional virtual space. A preset number of virtual screens can be created within the updated three-dimensional virtual space. The specific value of this preset number is not limited here.

[0058] In some optional implementations of certain embodiments, in response to detecting startup information of a corresponding two-dimensional application, the executing entity can update the three-dimensional virtual space deployed in the head-mounted display device through the following steps to obtain the updated three-dimensional virtual space as the three-dimensional virtual space:

[0059] The first step is to create a predetermined number of texture images within the three-dimensional virtual space. These texture images can represent OpenGL textures. The specific value of the predetermined number is not limited here; for example, the predetermined number could be 3. In practice, the executing entity can create the predetermined number of texture images using OpenGL ES functions.

[0060] The second step involves creating a predetermined number of two-dimensional rectangular canvases based on the predetermined number of texture images. Each texture image in the predetermined number of texture images corresponds to a two-dimensional rectangular canvas within the predetermined number of two-dimensional rectangular canvases. There can be a one-to-one correspondence between the texture images and the two-dimensional rectangular canvases. Each two-dimensional rectangular canvas can represent a Surface object in the terminal device's operating system (e.g., Android). In practice, firstly, for each texture image in the predetermined number of texture images, the executing entity can create a corresponding two-dimensional rectangular canvas using the SurfaceTexture class and OpenGL ES functions under the terminal device's operating system (e.g., Android). This results in the creation of the predetermined number of two-dimensional rectangular canvases.

[0061] The third step involves creating a preset number of virtual screens within the three-dimensional virtual space based on the preset set of application pages to be displayed for the corresponding two-dimensional application. These virtual screens can represent the VirtualDisplay function in the terminal device's operating system. The preset application pages to be displayed in the preset set of application pages to be displayed can represent the application pages of the two-dimensional application that the user has preset to create corresponding virtual screens for, and that are displayed using the created virtual screens. For example, the preset set of application pages to be displayed may include the main application page, the chat application page, and the contacts application page. It should be noted that the preset application pages to be displayed in the preset set of application pages to be displayed can be set according to user needs, without specific limitations here, and can further enhance the user experience.

[0062] The fourth step involves determining the preset number of two-dimensional rectangular canvases as rendering targets corresponding to the preset number of virtual screens. Specifically, the two-dimensional rectangular canvases within the preset number of two-dimensional rectangular canvases can correspond to virtual screens within the preset number of virtual screens. There can be a one-to-one correspondence between the two-dimensional rectangular canvases within the preset number of two-dimensional rectangular canvases and the virtual screens within the preset number of virtual screens. In practice, the executing entity can use Unity's built-in renderer to determine the preset number of two-dimensional rectangular canvases as rendering targets corresponding to the preset number of virtual screens.

[0063] In addressing the first technical problem mentioned above, a second technical problem often arises: creating identical virtual screens to display various application pages of a 2D application results in either smaller virtual screens for frequently used and time-consuming application pages, leading to a poor user experience, or larger virtual screens for infrequently used and short-duration application pages, resulting in wasted computing resources. A common solution to this second technical problem is to determine the size of the virtual screen for each application page based on the ranking of how often users access the page. However, considering the drawbacks of this approach, and leveraging the company's strengths in virtual screen creation, the inventors have decided to adopt the following solution:

[0064] In some optional implementations of certain embodiments, the executing entity may create a preset number of virtual screens in the three-dimensional virtual space according to a preset set of application pages to be displayed corresponding to the two-dimensional application through the following steps:

[0065] The first step is to obtain a set of historical application page usage information for the user corresponding to the two-dimensional application. The user corresponding to the two-dimensional application represents the currently logged-in user of the two-dimensional application. The historical application page usage information in the set represents the user's records of using application pages in the two-dimensional application. For example, historical application page usage information could represent that the user used the chat box page of the two-dimensional application for 1 hour and 5 times. The historical application page usage information in the set can correspond one-to-one with the preset application pages to be displayed in the preset set of application pages to be displayed. The preset application pages to be displayed in the preset set of application pages to be displayed can also correspond one-to-one with the virtual screens in the preset number of virtual screens. In practice, the executing entity can obtain the set of historical application page usage information for the user corresponding to the two-dimensional application from a database storing the historical application page usage information.

[0066] The second step is to perform data cleaning on the historical application page usage information set to obtain the cleaned historical application page usage information set as the target historical application page usage information set.

[0067] The third step involves determining the page display attribute information for each preset application page to be displayed in the preset application page set, based on the target historical application page usage information set. This results in the page display attribute information for each page. The page display attribute information can characterize the size of the virtual screen used to display the preset application page, the angle between the virtual screen and the field of view (FOV), and the distance between the virtual screen and the user in the head-mounted display device. For example, the page display attribute information could be the size of the virtual screen (1X = 1920 × 1080), the angle between the virtual screen and FOV (90 degrees), and the distance between the virtual screen and the target user in the head-mounted display device (1.5m). In practice, firstly, for each target historical application page usage information in the target historical application page usage information set, the executing entity can use the preset application page to be displayed corresponding to the target historical application page usage information as the target preset application page to be displayed. Then, the page display attribute information included in the preset page attribute information set corresponding to the target historical application page usage information is determined as the page display attribute information corresponding to the target preset application page to be displayed. The preset page attribute information in the preset page attribute information set can represent the correspondence between the target historical application usage information and the page attribute information. For example, the preset page attribute information could be: "Target historical application page usage information: The user used the chat box page of the two-dimensional application for 1 hour, and the number of times it was used was 5. Page display attribute information: Virtual screen size: 2X (1X = 1920 × 1080), the angle between the virtual screen and the FOV can be 85 degrees, and the distance between the virtual screen and the target user in the head-mounted display device can be 1.5m."

[0068] The fourth step is to create the preset number of virtual screens based on the page display attribute information. In practice, for the page display attribute information in the page display attribute information, the executing entity can call the DisplayManagerService service in the terminal device operating system (e.g., Android system) through the multi-window service to create virtual screens corresponding to the page display attribute information. Thus, the preset number of virtual screens can be obtained. For example, as shown in Figure 4, 401, 402, and 403 in Figure 4 can be the created virtual screens. Among them, the virtual screen represented by 401 corresponds to the preset application page to be displayed whose target historical application page usage information includes the fewest usage times and the shortest usage time, therefore the size of the virtual screen represented by 401 can be the smallest; the virtual screen represented by 403 corresponds to the preset application page to be displayed whose target historical application page usage information includes the most usage times and the longest usage time, therefore the size of the virtual screen represented by 403 can be the largest. The virtual screen represented by 402 corresponds to the preset application page to be displayed whose target historical application page usage information includes fewer usage times and shorter usage time, therefore the size of the virtual screen represented by 402 is smaller.

[0069] In some optional implementations of certain embodiments, the executing entity may perform data cleaning processing on the historical application page usage information set through the following steps to obtain the data-cleaned historical application page usage information set as the target historical application page usage information set:

[0070] The first step is to perform missing value detection on the historical application page usage information set to obtain detection results. These results indicate whether the historical application page usage information set contains missing values ​​or not. In practice, the executing entity can use Pandas' `isna()` or `isnull()` methods combined with string processing functions to perform missing value detection on the historical application page usage information set and obtain the detection results.

[0071] The second step is to determine, based on the detection results, whether the set of historical application page usage information meets the preset missing value filling condition. The preset missing value filling condition can be the detection results information corresponding to the set of historical application page usage information indicating the presence of missing values ​​in the set.

[0072] The third step involves, in response to determining that the set of historical application page usage information meets the preset missing value imputation condition, performing missing value imputation processing on the set of historical application page usage information to obtain a missing value imputation-processed set of historical application page usage information. In practice, the executing entity can use a fixed value imputation algorithm to perform missing value imputation processing on the set of historical application page usage information using specific text values ​​to obtain a missing value imputation-processed set of historical application page usage information. For example, the specific text value could be "unknown".

[0073] The fourth step involves performing outlier detection on the historical application page usage information set after missing value imputation, obtaining outlier detection results. These results indicate whether the historical application page usage information set after missing value imputation contains outliers or not. In practice, the executing entity can use a clustering algorithm to perform outlier detection on the historical application page usage information set after missing value imputation to obtain outlier detection results. For example, the clustering algorithm could be the DBSCAN algorithm.

[0074] Fifth, based on the outlier detection results, determine whether the set of historical application page usage information meets the preset outlier conditions. The preset outlier conditions can be the outlier detection results of the corresponding historical application page usage information set, indicating that outliers exist in the historical application page usage information set after missing value imputation.

[0075] Step 6: In response to determining that the historical application page usage information set meets the preset outlier condition, outlier deletion processing is performed on the historical application page usage information set to obtain the outlier-deleted historical application page usage information set as the target historical application page usage information set after data cleaning. In practice, the executing entity can delete outliers from the historical application page usage information set to obtain the outlier-deleted historical application page usage information set as the target historical application page usage information set.

[0076] The above-described technical solution, as an inventive point of this disclosure, addresses the second technical problem: "The virtual screens for application pages used frequently and for extended periods are small, resulting in a poor user experience; or the virtual screens for application pages used infrequently and for short periods are large, resulting in a significant waste of computing resources." The solution involves creating identical virtual screens to display the various application pages of a two-dimensional application. By solving these factors, the user experience can be improved, and the waste of computing resources can be reduced. To achieve this, when creating virtual screens, this disclosure first obtains a set of historical application page usage information for the currently logged-in user of the two-dimensional application. Then, the obtained historical application page usage information set is cleaned. Therefore, determining the size and other attributes of the virtual screen based on the cleaned historical application page usage information set improves the accuracy of setting the virtual screen size and other attributes, thus enhancing the user experience. Next, the display attribute information for each page is determined based on the cleaned historical application page usage information set. Finally, virtual screens are created based on the display attribute information for each page. Since the historical application page usage information in the acquired set can include the number of times and the time spent by the currently logged-in user using the application page, the size and other attributes of the virtual screen can be set according to the number of times and the time spent using the application page. This allows for the creation of larger virtual screens to display application pages that have been used frequently and for a long time, or smaller virtual screens to display application pages that have been used infrequently and for a short time. This can improve the user experience and reduce the waste of computing resources.

[0077] Step 302: Determine the virtual screen corresponding to the main application page from the preset number of virtual screens in the three-dimensional virtual space as the main virtual screen.

[0078] Step 303: Display the main application page of the two-dimensional application in the main virtual screen within the three-dimensional virtual space.

[0079] Step 304: In response to detecting a page navigation selection operation on the corresponding main application page, and the page navigation selection operation meets the preset split-screen conditions, the following split-screen steps are executed:

[0080] Step 3041 involves listening to and processing page navigation selection operations.

[0081] Step 3042: Modify the attribute information of the application page corresponding to the page jump selection operation to obtain the modified application page.

[0082] Step 3043: Display the modified application page in the sub-virtual screen within the three-dimensional virtual space.

[0083] In some embodiments, the specific implementation of steps 304 and 303 and the resulting technical effects can be referred to steps 201-202 in the embodiments corresponding to Figure 2, and will not be repeated here.

[0084] As can be seen from Figure 3, compared with the description of some embodiments corresponding to Figure 2, the process 300 of the split-screen display method in some embodiments corresponding to Figure 3 reflects that the size and other attributes of the virtual screen to be created can be determined according to the set of historical application page usage information after data cleaning, thereby improving the accuracy of setting virtual screen attributes. Furthermore, the size and other attributes of the virtual screen to be created can be determined according to the time and number of times the user uses the application page. The size of the virtual screen used to display the user's application page for a longer time and more times can be set to a larger size, while the size of the virtual screen used to display the user's application page for a shorter time and fewer times can be set to a smaller size, further improving the user experience and reducing the waste of computing resources.

[0085] Referring to Figure 5 below, a schematic diagram of the hardware structure of a head-mounted display device 500 with display functionality is shown.

[0086] As shown in Figure 5, the head-mounted display device 500 includes a processing unit (CPU) 501, a memory (ROM) 502, an input unit 503, and an output unit 504, wherein the processing unit 501, memory 502, input unit 503, and output unit 504 are interconnected via a bus 505. Here, the methods according to some embodiments of this disclosure can be implemented as a computer program and stored in the memory 502. The processing unit 501 in the head-mounted display device 500 implements the split-screen display function defined in some embodiments of this disclosure by calling the computer program stored in the memory 502. In some implementations, the input unit 503 may include devices such as a camera, microphone, gyroscope, accelerometer, and magnetometer, and the output unit 504 may be a display screen or other device that can display content. The display screen may be a microdisplay. Thus, when the processing unit 501 calls the computer program to execute the split-screen display function, it can control the input unit 503 to acquire user gestures, voice commands, and other operation instructions, and control the output unit 504 to display the content.

[0087] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0088] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0089] The computer-readable medium may be included in the head-mounted display device or may exist independently without being assembled into the head-mounted display device. The computer-readable medium carries one or more programs that, when executed by the head-mounted display device, cause the head-mounted display device to: in response to detecting startup information of a corresponding two-dimensional application, display the main application page of the two-dimensional application in a main virtual screen within a three-dimensional virtual space, wherein the three-dimensional virtual space is deployed on the head-mounted display device; in response to detecting a page jump selection operation corresponding to the main application page, and the page jump selection operation satisfies preset split-screen conditions, perform the following split-screen steps: listen to the page jump selection operation; modify the attribute information of the application page corresponding to the page jump selection operation to obtain a modified application page, wherein the attribute information includes a source window identifier attribute and an activity identifier attribute; display the modified application page in a sub-virtual screen within the three-dimensional virtual space, wherein multiple application pages of the two-dimensional application can be displayed simultaneously on various virtual screens.

[0090] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0091] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0092] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0093] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A split-screen display method, applied to a head-mounted display device, comprising: In response to detecting the launch information of the corresponding two-dimensional application, the main application page of the two-dimensional application is displayed in the main virtual screen in the three-dimensional virtual space, wherein the three-dimensional virtual space is deployed in the head-mounted display device; In response to detecting a page navigation selection operation corresponding to the main application page, and the page navigation selection operation meets the preset split-screen conditions, the following split-screen steps are executed: Listen for and process page navigation selection operations; The attribute information of the application page corresponding to the page jump selection operation is modified to obtain the modified application page, wherein the attribute information includes the source window identifier attribute and the activity identifier attribute; The modified application page is displayed on a sub-virtual screen within the three-dimensional virtual space, wherein multiple application pages of the two-dimensional application can be displayed simultaneously on various virtual screens.

2. The split-screen display method according to claim 1, wherein, Before displaying the main application page of the two-dimensional application on the main virtual screen within the three-dimensional virtual space, the method further includes: The three-dimensional virtual space deployed in the head-mounted display device is updated to obtain the updated three-dimensional virtual space as the three-dimensional virtual space, wherein a preset number of virtual screens are created in the updated three-dimensional virtual space. The virtual screen corresponding to the main application page among a preset number of virtual screens in the updated three-dimensional virtual space is determined as the main virtual screen.

3. The split-screen display method according to claim 2, wherein, The step of updating the three-dimensional virtual space deployed in the head-mounted display device to obtain the updated three-dimensional virtual space includes: A preset number of texture images are created within the three-dimensional virtual space; Based on the preset number of texture images, a preset number of two-dimensional rectangular canvases are created, wherein the texture images in the preset number of texture images correspond to the two-dimensional rectangular canvases in the preset number of two-dimensional rectangular canvases.

4. The split-screen display method according to claim 3, wherein, After creating a preset number of two-dimensional rectangular canvases based on the preset number of texture images, the method further includes: Based on the preset set of application pages to be displayed corresponding to the two-dimensional application, a preset number of virtual screens are created in the three-dimensional virtual space; The preset number of two-dimensional rectangular canvases are determined as rendering targets corresponding to the preset number of virtual screens, wherein the two-dimensional rectangular canvases in the preset number of two-dimensional rectangular canvases correspond to the virtual screens in the preset number of virtual screens.

5. The split-screen display method according to claim 2, wherein, Before displaying the modified application page in the sub-virtual screen within the three-dimensional virtual space, the method further includes: The virtual screen corresponding to the modified application page among a preset number of virtual screens in the three-dimensional virtual space is determined as a sub-virtual screen.

6. The split-screen display method according to claim 1, wherein, The step of modifying the attribute information of the application page corresponding to the page jump selection operation to obtain the modified application page includes: The attribute value of the source window identifier attribute included in the attribute information is set to empty; The attribute value of the activity identifier attribute included in the attribute information is determined as the new task activity attribute value.

7. A head-mounted display device, comprising: One or more processors; Storage device for storing one or more programs; A display screen used to project an image in front of the user; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1 to 6.

8. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images