Split-screen display method, head-mounted display device and computer readable medium

Abstract

The embodiment of the invention discloses a split-screen display method, head-mounted display equipment and a computer readable medium. A specific embodiment of the method comprises the steps of displaying a main application page of a two-dimensional application in response to detection of starting information corresponding to the two-dimensional application; in response to the detected page jump selection operation of the main application page and the page jump selection operation meeting a preset screen splitting condition, executing the following screen splitting steps: monitoring the page jump selection operation; changing attribute information of an application page corresponding to the page jump selection operation to obtain a changed application page; and displaying the changed application page in a sub-virtual screen in the three-dimensional virtual space. According to the implementation mode, the multiple application pages of one two-dimensional application can be displayed at the same time, the operation steps for displaying the application pages can be simplified, time consumed for displaying the application pages is shortened, and consumed computing resources are reduced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of virtual vision technology, and in particular to a split-screen display method, a head-mounted display device, and a computer-readable medium. Background Art

[0002] Using 2D applications in the AR space operating system can enhance the user experience of using 2D applications. At present, when displaying 2D applications in the AR space operating system, the usual method is: using a page chain design display method to display the application page of the 2D application. First, open the main application page of the 2D application in the AR space operating system. Then, when the user needs to open other application pages of the 2D application, it is necessary to exit the currently displayed application page and then display the other application pages that the user needs to open. Therefore, multiple application pages of the same 2D application cannot be displayed in the AR space operating system at the same time.

[0003] However, it is found in practice that when the above method is used to display the application page of a 2D application, the following technical problems often occur:

[0004] When the application page of a 2D application is displayed according to the display method of page chain design, it is impossible to display multiple application pages of a 2D application at the same time, resulting in a poor user experience; and when the user needs to open other application pages of the 2D application, it is necessary to exit the application page of the currently displayed 2D application before displaying the other application pages that the user needs to open, resulting in cumbersome operation steps for displaying the application page, resulting in a long time-consuming and more computing resources consumed in displaying the application page of the 2D application.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a split-screen display method, a head-mounted display device, and a computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a split-screen display method, which is applied to a head-mounted display device, comprising: in response to detecting startup information corresponding to a two-dimensional application, displaying a main application page of the two-dimensional application in a main virtual screen within a three-dimensional virtual space, wherein the three-dimensional virtual space is deployed on the head-mounted display device; in response to detecting a page jump selection operation corresponding to the main application page, and the page jump selection operation meets a preset split-screen condition, performing the following split-screen steps: monitoring the page jump selection operation; changing the attribute information of the application page corresponding to the page jump selection operation to obtain the changed application page, wherein the attribute information includes a source window identification attribute and an activity identification attribute; displaying the changed application page in a sub-virtual screen within the three-dimensional virtual space, wherein multiple application pages of the two-dimensional application can be displayed on each virtual screen at the same time.

[0009] Optionally, the method also includes: updating the three-dimensional virtual space deployed in the above-mentioned head-mounted display device to obtain the updated three-dimensional virtual space as the three-dimensional virtual space, wherein a preset number of virtual screens are created in the updated three-dimensional virtual space; and determining the virtual screen corresponding to the above-mentioned main application page among the preset number of virtual screens in the above-mentioned three-dimensional virtual space as the main virtual screen.

[0010] Optionally, the method also includes: creating a preset number of texture images in the above-mentioned three-dimensional virtual space; creating the above-mentioned preset number of two-dimensional rectangular canvases according to the above-mentioned preset number of texture images, wherein the texture images in the above-mentioned preset number of texture images correspond to the two-dimensional rectangular canvases in the above-mentioned preset number of two-dimensional rectangular canvases.

[0011] Optionally, the method also includes: creating a preset number of virtual screens in the above-mentioned three-dimensional virtual space according to a preset set of application pages to be displayed corresponding to the above-mentioned two-dimensional application; determining the above-mentioned preset number of two-dimensional rectangular canvases as respective rendering targets corresponding to the above-mentioned preset number of virtual screens, wherein the two-dimensional rectangular canvases in the above-mentioned preset number of two-dimensional rectangular canvases correspond to the virtual screens in the above-mentioned preset number of virtual screens.

[0012] Optionally, the method further includes: determining a virtual screen corresponding to the changed application page among a preset number of virtual screens in the three-dimensional virtual space as a sub-virtual screen.

[0013] Optionally, the method further includes: determining the attribute value of the source window identification attribute included in the attribute information to be empty; and determining the attribute value of the activity identification attribute included in the attribute information to be the new task activity attribute value.

[0014] In a second aspect, some embodiments of the present disclosure provide a head-mounted display device, comprising: one or more processors; a storage device for storing one or more programs; a display screen for imaging in front of the user's eyes; when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the implementation methods in the first aspect.

[0015] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in any one of the implementations in the first aspect.

[0016] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through a split-screen display method of some embodiments of the present disclosure, it is possible to realize the simultaneous display of multiple application pages of a 2D application, thereby improving the user experience, and simplifying the operation steps of displaying the application page, thereby shortening the time consumption of displaying the application page of the 2D application and reducing the consumed computing resources. The reasons for not being able to realize the simultaneous display of multiple application pages of a 2D application, the poor user experience, the cumbersome operation steps of displaying the application page, the long time consumption of displaying the application page of the 2D application and the large amount of consumed computing resources are: when the application page of the 2D application is displayed according to the display method of the page chain design, it is impossible to realize the simultaneous display of multiple application pages of a 2D application, resulting in a poor user experience; and when the user needs to open other application pages of the 2D application, it is necessary to exit the application page of the currently displayed 2D application before displaying the other application pages that the user needs to open, resulting in the cumbersome operation steps of displaying the application page, resulting in a long time consumption of displaying the application page of the 2D application and the large amount of consumed computing resources. Based on this, the split-screen display method of some embodiments of the present disclosure, first, in response to detecting the startup information of the corresponding two-dimensional application, the main application page of the above-mentioned two-dimensional application is displayed in the main virtual screen in the three-dimensional virtual space, wherein the above-mentioned three-dimensional virtual space is deployed in the above-mentioned head-mounted display device. Thus, the main application page of the above-mentioned two-dimensional application can be displayed on the virtual screen. Then, in response to detecting the page jump selection operation corresponding to the above-mentioned main application page, and the page jump selection operation meets the preset split-screen condition, the following split-screen steps are performed: First, the page jump selection operation is monitored. Thus, the callback opened by the above-mentioned page jump selection operation can be monitored. Afterwards, the attribute information of the application page corresponding to the page jump selection operation is changed to obtain the application page after the change processing. Among them, the above-mentioned attribute information may include a source window identification attribute and an activity identification attribute. Thus, the association relationship between the page to be displayed (the application page corresponding to the page jump selection operation) and the source page of the page to be displayed (the starting page of the page to be displayed) can be cut off by modifying the source window identification attribute and the activity identification attribute of the application page, so that the multiple application pages of the above-mentioned two-dimensional application can be further displayed simultaneously. Finally, the changed application page is displayed in the sub-virtual screen in the three-dimensional virtual space. The multiple application pages of the two-dimensional application can be displayed on each virtual screen at the same time. Thus, it is possible to display each application page corresponding to the two-dimensional application on each virtual screen at the same time.Also, because the application pages of the above two-dimensional application are not displayed in a chain method according to a display method designed by page chain, but the association between the application page to be displayed and the source page to be displayed is cut off by modifying the attribute value of the source window identification attribute and the attribute value of the activity identification attribute of the page to be displayed, so that each application page of the above two-dimensional application can be displayed on each virtual screen at the same time. Also, because when displaying other application pages of the two-dimensional application, it is not necessary to exit the currently displayed application page before displaying other application pages, but directly display other application pages of the two-dimensional application on other virtual screens, so that the operation steps for displaying the application page can be simplified, the time consumption for displaying the application page can be shortened, and the consumed computing resources can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0018] Figure 1 is an exemplary system architecture diagram of a split-screen display method applying some embodiments of the present disclosure;

[0019] Figure 2 is a flow chart of some embodiments of the split-screen display method according to the present disclosure;

[0020] Figure 3 is a flow chart of other embodiments of the split-screen display method according to the present disclosure;

[0021] Figure 4 is a schematic diagram of an application scenario of the split-screen display method disclosed in the present invention;

[0022] Figure 5 It is a schematic diagram of the hardware structure of a head-mounted display device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0024] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0025] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0026] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0027] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0028] Figure 1 An exemplary system architecture 100 that can be applied to an embodiment of the split-screen display method applied to a head-mounted display device of the present disclosure is shown.

[0029] like Figure 1 As shown, the exemplary system architecture 100 may include a head mounted display device 11 and a terminal device 12 .

[0030] The head-mounted display device 11 may include one or two display screens 111. The above-mentioned display screens are used to display a virtual interface. In addition, the head-mounted display device 11 also includes a frame 112. In some embodiments, the sensor, processing unit, memory and battery of the head-mounted display device 11 may be placed inside the frame 112. In some optional implementations of some embodiments, one or more components of the sensor, processing unit, memory and battery may also be integrated into another independent accessory (not shown), and connected to the frame 112 via a data cable 13. In some optional implementations of some embodiments, the head-mounted display device 11 may only have a display function and some sensors, and the terminal device 12 provides data processing, data storage, power supply and other capabilities.

[0031] The terminal device 12 may include a touch screen 121 , which may be used to display content. In some embodiments, the head mounted display device 11 and the terminal device 12 may be connected via a data cable 13 .

[0032] It should be understood that Figure 1 The number of head mounted display devices and terminal devices in the embodiment is only for illustration. According to the implementation requirements, any suitable number of head mounted display devices and terminal devices may be provided.

[0033] Further references Figure 2 , shows a process 200 of some embodiments of the split-screen display method according to the present disclosure. The split-screen display method comprises the following steps:

[0034] Step 201 : in response to detecting startup information of a corresponding two-dimensional application, displaying a main application page of the two-dimensional application in a main virtual screen in a three-dimensional virtual space.

[0035] In some embodiments, in response to detecting the startup information of the corresponding two-dimensional application, the execution subject (e.g., head-mounted display device) of the split-screen display method may display the main application page of the two-dimensional application in the main virtual screen in the three-dimensional virtual space. Among them, the three-dimensional virtual space may be deployed in the head-mounted display device. The two-dimensional application may be any two-dimensional application program. The main application page may be the homepage of the two-dimensional application. The three-dimensional virtual space may be Unity 3D space. The startup information may indicate that the two-dimensional application has been turned on. The main virtual screen may indicate a virtual screen for displaying the main application page of the two-dimensional application. The main virtual screen may be a virtual screen created in the three-dimensional virtual space. The head-mounted display device may be deployed with Unity applications. A preset number of virtual screens corresponding to the two-dimensional application are created in the three-dimensional virtual space. The preset number of virtual screens created in the three-dimensional virtual space may all be used to display the application page of the two-dimensional application. Here, the specific value of the preset number is not limited, for example, the preset number may be 3.

[0036] Optionally, before the step 201, the execution entity may also determine a virtual screen corresponding to the main application page among a preset number of virtual screens in the three-dimensional virtual space as a main virtual screen.

[0037] Optionally, after the above step 201, the above execution entity may also synchronize the main application page of the two-dimensional application displayed in the above main virtual screen to the above three-dimensional virtual space.

[0038] Step 202: In response to detecting a page jump selection operation corresponding to the main application page, and the page jump selection operation meets a preset split screen condition, the following split screen steps are performed:

[0039] Step 2021, monitor and process the page jump selection operation.

[0040] In some embodiments, the execution subject may monitor and process the page jump selection operation. Among them, the page jump selection operation corresponding to the main application page may represent an operation of jumping from the currently displayed application page to any application page among the application pages derived from the main application page. The preset split-screen condition may be a virtual screen in the three-dimensional virtual space for displaying the application page corresponding to the page jump selection operation. In practice, the execution subject may monitor the callback of the page jump selection operation being opened in the Framework of the terminal device operating system (e.g., Android system).

[0041] It should be noted that each application page jumped from the above main application page (the source page of each application page is the above main application page), and the target application page jumped from any application page of the above application pages (the source page of the target application page can be an application page) can be each application page derived from the above main application page. For example, the application page jumped from the main application page is a chat box page, and the application page jumped from the chat box page is a video call page, then the chat box page and the video call page can both be application pages derived from the above main application page.

[0042] Step 2022: Change the attribute information of the application page corresponding to the page jump selection operation to obtain the changed application page.

[0043] In some embodiments, the execution subject may change the attribute information of the application page corresponding to the page jump selection operation to obtain the changed application page. The attribute information may include a source window identification attribute and an activity identification attribute. The attribute information may represent the attributes of the application page. The source window identification attribute may represent the SourceRecord attribute of the Activity started in the terminal device operating system (e.g., Android system). The activity identification attribute may represent the LaunchFlag attribute.

[0044] In some optional implementations of some embodiments, the execution subject may change the attribute information of the application page corresponding to the page jump selection operation through the following steps to obtain the changed application page:

[0045] The first step is to determine that the attribute value of the source window identification attribute included in the above attribute information is empty.

[0046] In the second step, the attribute value of the activity identification attribute included in the attribute information is determined as the new task activity attribute value. The new task activity attribute value may represent FLAG_ACTIVITY_NEW_TASK.

[0047] It should be noted that the attribute value of the SourceRecord attribute of the above-mentioned launch Activity is empty, which can indicate that the application page corresponding to the page jump selection operation is the first page opened, and the application page corresponding to the page jump selection operation has no source page. Therefore, the association between the application page corresponding to the page jump selection operation and the source page of the application page corresponding to the page jump selection operation can be cut off; the attribute value of the above-mentioned LaunchFlag attribute is FLAG_ACTIVITY_NEW_TASK, which can indicate that the application page corresponding to the page jump selection operation is a newly launched activity, thereby realizing the simultaneous display of multiple application pages of the same two-dimensional application.

[0048] Step 2023: Display the changed application page on the sub-virtual screen in the three-dimensional virtual space.

[0049] In some embodiments, the execution subject may display the changed application page in a sub-virtual screen in the three-dimensional virtual space. The multiple application pages of the two-dimensional application may be displayed on each virtual screen at the same time. The sub-virtual screen may represent a virtual screen displayed in the three-dimensional virtual space.

[0050] Optionally, before the above step 2023, the above execution subject may determine the virtual screen corresponding to the application page after the change process among the preset number of virtual screens in the above three-dimensional virtual space as the sub-virtual screen. It should be noted that the virtual screen corresponding to the application page after the change process refers to the virtual screen used to display the application page corresponding to the page jump selection operation (the application page before the change process).

[0051] Optionally, after the step 2023, the execution entity may synchronize the changed application page displayed in the sub-virtual screen to the three-dimensional virtual space.

[0052] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through a split-screen display method of some embodiments of the present disclosure, it is possible to realize the simultaneous display of multiple application pages of a 2D application, thereby improving the user experience, and simplifying the operation steps of displaying the application page, thereby shortening the time consumption of displaying the application page of the 2D application and reducing the consumed computing resources. The reasons for not being able to realize the simultaneous display of multiple application pages of a 2D application, the poor user experience, the cumbersome operation steps of displaying the application page, the long time consumption of displaying the application page of the 2D application and the large amount of consumed computing resources are: when the application page of the 2D application is displayed according to the display method of the page chain design, it is impossible to realize the simultaneous display of multiple application pages of a 2D application, resulting in a poor user experience; and when the user needs to open other application pages of the 2D application, it is necessary to exit the application page of the currently displayed 2D application before displaying the other application pages that the user needs to open, resulting in the cumbersome operation steps of displaying the application page, resulting in a long time consumption of displaying the application page of the 2D application and the large amount of consumed computing resources. Based on this, the split-screen display method of some embodiments of the present disclosure, first, in response to detecting the startup information of the corresponding two-dimensional application, the main application page of the above-mentioned two-dimensional application is displayed in the main virtual screen in the three-dimensional virtual space, wherein the above-mentioned three-dimensional virtual space is deployed in the above-mentioned head-mounted display device. Thus, the main application page of the above-mentioned two-dimensional application can be displayed on the virtual screen. Then, in response to detecting the page jump selection operation corresponding to the above-mentioned main application page, and the page jump selection operation meets the preset split-screen condition, the following split-screen steps are performed: First, the page jump selection operation is monitored. Thus, the callback opened by the above-mentioned page jump selection operation can be monitored. Afterwards, the attribute information of the application page corresponding to the page jump selection operation is changed to obtain the application page after the change processing. Among them, the above-mentioned attribute information may include a source window identification attribute and an activity identification attribute. Thus, the association relationship between the page to be displayed (the application page corresponding to the page jump selection operation) and the source page of the page to be displayed (the starting page of the page to be displayed) can be cut off by modifying the source window identification attribute and the activity identification attribute of the application page, so that the multiple application pages of the above-mentioned two-dimensional application can be further displayed simultaneously. Finally, the changed application page is displayed in the sub-virtual screen in the three-dimensional virtual space. The multiple application pages of the two-dimensional application can be displayed on each virtual screen at the same time. Thus, it is possible to display each application page corresponding to the two-dimensional application on each virtual screen at the same time.Also, because the application pages of the above two-dimensional application are not displayed in a chain method according to a display method designed by page chain, but the association between the application page to be displayed and the source page to be displayed is cut off by modifying the attribute value of the source window identification attribute and the attribute value of the activity identification attribute of the page to be displayed, so that each application page of the above two-dimensional application can be displayed on each virtual screen at the same time. Also, because when displaying other application pages of the two-dimensional application, it is not necessary to exit the currently displayed application page before displaying other application pages, but directly display other application pages of the two-dimensional application on other virtual screens, so that the operation steps for displaying the application page can be simplified, the time consumption for displaying the application page can be shortened, and the consumed computing resources can be reduced.

[0053] Further references Figure 3 , which shows a process 300 of another embodiment of the split-screen display method. The process 300 of the split-screen display method includes the following steps:

[0054] Step 301 , in response to detecting startup information of a corresponding two-dimensional application, updating the three-dimensional virtual space deployed in the head mounted display device to obtain the updated three-dimensional virtual space as the three-dimensional virtual space.

[0055] In some embodiments, in response to detecting the startup information of the corresponding two-dimensional application, the execution subject of the split-screen display method (e.g., a head-mounted display device) can update the three-dimensional virtual space deployed in the head-mounted display device to obtain the updated three-dimensional virtual space as the three-dimensional virtual space. Among them, a preset number of virtual screens can be created in the updated three-dimensional virtual space. Here, the specific value of the above preset number is not limited.

[0056] In some optional implementations of some embodiments, in response to detecting startup information of a corresponding two-dimensional application, the execution subject may update the three-dimensional virtual space deployed in the head mounted display device through the following steps to obtain the updated three-dimensional virtual space as the three-dimensional virtual space:

[0057] The first step is to create a preset number of texture images in the three-dimensional virtual space. The texture images may represent the texture of OpenGL. The specific value corresponding to the preset number is not specifically limited. For example, the preset number may be 3. In practice, the execution subject may create the preset number of texture images through the OpenGL ES function of OpenGL.

[0058] The second step is to create the preset number of two-dimensional rectangular canvases according to the preset number of texture images. Among them, the texture images in the preset number of texture images can correspond to the two-dimensional rectangular canvases in the preset number of two-dimensional rectangular canvases. The texture images in the preset number of texture images can correspond to the two-dimensional rectangular canvases in the preset number of two-dimensional rectangular canvases. Each of the preset number of two-dimensional rectangular canvases can represent a Surface object in the terminal device operating system (for example, Android system). In practice, first, for each texture image in the preset number of texture images, the execution subject can create a two-dimensional rectangular canvas corresponding to the texture image through the SurfaceTexture class and OpenGL ES function under the terminal device operating system (for example, Android system). Thus, the preset number of two-dimensional rectangular canvases created can be obtained.

[0059] The third step is to create a preset number of virtual screens in the three-dimensional virtual space according to the preset set of application pages to be displayed corresponding to the above-mentioned two-dimensional application. Among them, the virtual screens in the above-mentioned preset number of virtual screens can represent the VirtualDisplay in the above-mentioned terminal device operating system. The preset application pages to be displayed in the above-mentioned preset application page set can represent the user's preset need to create a corresponding virtual screen, and the application page of the above-mentioned two-dimensional application displayed using the created corresponding virtual screen. For example, the preset application page set to be displayed may include the main application page, the chat box application page, and the contact application page. It should be noted that the preset application pages to be displayed in the preset application page set to be displayed can be set according to the needs of the user, which is not specifically limited here, and can also further enhance the user experience.

[0060] The fourth step is to determine the preset number of two-dimensional rectangular canvases as the rendering targets corresponding to the preset number of virtual screens. Among them, the two-dimensional rectangular canvases in the preset number of two-dimensional rectangular canvases can correspond to the virtual screens in the preset number of virtual screens. The two-dimensional rectangular canvases in the preset number of two-dimensional rectangular canvases and the virtual screens in the preset number of virtual screens can be one-to-one corresponding. In practice, the execution entity can use Unity's built-in renderer to determine the preset number of two-dimensional rectangular canvases as the rendering targets corresponding to the preset number of virtual screens.

[0061] In the process of adopting technical solutions to solve the above-mentioned technical problem one, the following technical problem two is often accompanied: creating identical virtual screens for displaying each application page of a two-dimensional application, resulting in smaller virtual screens for application pages that are used more frequently and for a longer period of time, leading to a poor user experience, or resulting in larger virtual screens for application pages that are used very seldom and for a shorter period of time, resulting in more wasted computing resources. In response to the above-mentioned technical problem two, the conventional solution is generally to determine the size of the virtual screen corresponding to the application page based on the ranking of the number of times most users use the application page. The inventor took into account the shortcomings of determining the size of the virtual screen corresponding to the application page based on the ranking of the number of times most users use the application page. In combination with the research and development advantages of the inventor's company in creating virtual screens, we decided to adopt the following solution:

[0062] In some optional implementations of some embodiments, the execution subject may create a preset number of virtual screens in the three-dimensional virtual space according to a preset set of application pages to be displayed corresponding to the two-dimensional application through the following steps:

[0063] The first step is to obtain a set of historical application page usage information of the user corresponding to the above-mentioned two-dimensional application. Among them, the user corresponding to the above-mentioned two-dimensional application can represent the user currently logged in to the above-mentioned two-dimensional application. The historical application page usage information in the above-mentioned historical application page usage information set can represent the record of the above-mentioned user using the application page of the above-mentioned two-dimensional application. For example, the historical application page usage information can represent that the user used the chat box page of the above-mentioned two-dimensional application for 1 hour and the number of times of use was 5 times. The historical application page usage information in the above-mentioned historical application page usage information set can correspond one-to-one with the preset application page to be displayed in the above-mentioned preset application page to be displayed set. The preset application page to be displayed in the above-mentioned preset application page to be displayed set can correspond one-to-one with the virtual screen in the above-mentioned preset number of virtual screens. In practice, the above-mentioned execution entity can obtain the set of historical application page usage information of the user corresponding to the above-mentioned two-dimensional application from the database storing the set of historical application page usage information.

[0064] In the second step, data cleaning is performed on the above historical application page usage information set to obtain the historical application page usage information set after data cleaning as the target historical application page usage information set.

[0065] The third step is to determine the page display attribute information corresponding to each preset application page to be displayed in the preset application page to be displayed set according to the above-mentioned target historical application page usage information set, and obtain each page display attribute information. Among them, the page display attribute information in the above-mentioned each page display attribute information can characterize the size of the virtual screen used to display the preset application page to be displayed, the angle between the virtual screen and the fov, and the distance between the virtual screen and the user in the above-mentioned head-mounted display device. For example, the page display attribute information can be the size of the virtual screen: 1X (1X = 1920 × 1080), the angle between the virtual screen and the fov can be 90 degrees, and the distance between the virtual screen and the above-mentioned target user in the above-mentioned head-mounted display device can be 1.5m. In practice, first, for each target historical application page usage information in the above-mentioned target historical application page usage information set, the above-mentioned execution subject can use the preset application page to be displayed corresponding to the above-mentioned target historical application page usage information as the target preset application page to be displayed. Then, the page display attribute information included in the preset page attribute information corresponding to the above-mentioned target historical application page usage information in the preset page attribute information set is determined as the page display attribute information corresponding to the above-mentioned target preset application page to be displayed. Among them, the preset page attribute information in the above-mentioned preset page attribute information set can characterize the correspondence between the target historical application usage information and the page attribute information. For example, the preset page attribute information can be: "Target historical application page usage information: the user used the chat box page of the above-mentioned two-dimensional application for 1 hour, the number of times used was 5 times, page display attribute information: virtual screen size: 2X (1X = 1920 × 1080), the angle between the virtual screen and the fov can be 85 degrees, and the distance between the virtual screen and the above-mentioned target user in the above-mentioned head-mounted display device can be 1.5m".

[0066] The fourth step is to create the preset number of virtual screens according to the above-mentioned page display attribute information. In practice, for the page display attribute information in the above-mentioned page display attribute information, the above-mentioned execution subject can call the DisplayManagerService service in the above-mentioned terminal device operating system (for example, Android system) through the multi-window service to create a virtual screen corresponding to the above-mentioned page display attribute information according to the above-mentioned page display attribute information. In this way, the preset number of virtual screens created can be obtained. For example, Figure 4 As shown, Figure 4401, 402, and 403 may be the created virtual screens, respectively. The target historical application page usage information of the preset application page to be displayed corresponding to the virtual screen represented by 401 includes the least number of uses and the shortest usage time, so the size of the virtual screen represented by 401 may be the smallest; the target historical application page usage information of the preset application page to be displayed corresponding to the virtual screen represented by 403 includes the most number of uses and the longest usage time, so the size of the virtual screen represented by 403 may be the largest. The target historical application page usage information of the preset application page to be displayed corresponding to the virtual screen represented by 402 includes fewer number of uses and shorter usage time, so the size of the virtual screen represented by 402 is smaller.

[0067] In some optional implementations of some embodiments, the execution subject may perform data cleaning on the historical application page usage information set through the following steps to obtain the historical application page usage information set after data cleaning as the target historical application page usage information set:

[0068] The first step is to perform missing value detection on the target historical application page usage information set to obtain detection result information. The detection result information can indicate that there are missing values ​​in the target historical application page usage information set or that there are no missing values ​​in the target historical application page usage information set. In practice, the execution subject can perform missing value detection on the target historical application page usage information set through Pandas' isna() or isnull() method combined with string processing function to obtain detection result information.

[0069] The second step is to determine whether the target historical application page usage information set meets the preset missing value filling condition based on the detection result information. The preset missing value filling condition may be that the detection result information corresponding to the target historical application page usage information set indicates that the target historical application page usage information set has missing values.

[0070] The third step is, in response to determining that the target historical application page usage information set satisfies the preset missing value filling condition, performing missing value filling processing on the target historical application page usage information set to obtain the target historical application page usage information set after the missing value filling processing. In practice, the execution subject may perform missing value filling processing on the target historical application page usage information set using a specific text value through a fixed value filling algorithm to obtain the target historical application page usage information set after the missing value filling processing. For example, the specific text value may be "unknown".

[0071] The fourth step is to perform outlier detection processing on the target historical application page usage information set after the missing value filling processing to obtain outlier detection result information. Among them, the above-mentioned outlier detection result information can indicate that there are outliers in the target historical application page usage information set after the missing value filling processing or that there are no outliers in the target historical application page usage information set after the missing value filling processing. In practice, the above-mentioned execution entity can perform outlier detection processing on the target historical application page usage information set after the missing value filling processing through a clustering algorithm to obtain outlier detection result information. For example, the clustering algorithm can be the DBSCAN algorithm.

[0072] Step 5: Determine whether the target historical application page usage information set meets a preset abnormal value condition based on the abnormal value detection result information. The preset abnormal value condition may be that the abnormal value detection result information of the corresponding target historical application page usage information set indicates that the target historical application page usage information set after the missing value filling process has an abnormal value.

[0073] Step 6: In response to determining that the target historical application page usage information set meets the preset abnormal value condition, the target historical application page usage information set is subjected to abnormal value deletion processing to obtain the target historical application page usage information set after the abnormal value deletion processing as the target historical application page usage information set after data cleaning processing. In practice, the execution subject may delete the abnormal values ​​in the target historical application page usage information set to obtain the target historical application page usage information set after the abnormal value deletion processing as the target historical application page usage information set.

[0074] The above technical solution, as an inventive point of an embodiment of the present disclosure, solves the second technical problem "the virtual screen of the application page that is used more times and for a longer time by the user is smaller, resulting in a poor user experience or the virtual screen corresponding to the application page that is used less times and for a shorter time by the user is larger, resulting in more wasted computing resources" because of the following reasons: each identical virtual screen is created to display each application page of the two-dimensional application. If the above factors are solved, the effect of improving the user experience and reducing the waste of computing resources can be achieved. In order to achieve this effect, when creating a virtual screen, the present disclosure first obtains the historical application page usage information set of the user who is currently logged in to the two-dimensional application, and then performs data cleaning on the obtained historical application page usage information set. Therefore, determining the size and other attributes of the virtual screen according to the historical application page usage information set after data cleaning can improve the accuracy of setting the virtual screen size and other attributes of the virtual screen, and improve the user experience. Afterwards, the display attribute information of each page is determined according to the historical application page usage information set after data cleaning. Then, a virtual screen is created according to the display attribute information of each page. Since the historical application page usage information in the acquired historical application page usage information set may include the number of times and the time consumed by the currently logged-in user using the application page, the size of the virtual screen and other attributes may be set according to the number of times and the time consumed by the user using the application page. A larger virtual screen may be created to display application pages that are used more frequently and for a longer period of time, and a smaller virtual screen may be created to display application pages that are used very seldom and for a shorter period of time, thereby improving the user experience and reducing the waste of computing resources.

[0075] Step 302: determine a virtual screen corresponding to the main application page among a preset number of virtual screens in the three-dimensional virtual space as a main virtual screen.

[0076] Step 303: Display the main application page of the two-dimensional application on the main virtual screen in the three-dimensional virtual space.

[0077] Step 304: in response to detecting a page jump selection operation corresponding to the main application page, and the page jump selection operation meets a preset split screen condition, the following split screen steps are performed:

[0078] Step 3041 monitors and processes the page jump selection operation.

[0079] Step 3042: Change the attribute information of the application page corresponding to the page jump selection operation to obtain the changed application page.

[0080] Step 3043: Display the changed application page on the sub-virtual screen in the three-dimensional virtual space.

[0081] In some embodiments, the specific implementation of steps 302 and 303 and the technical effects brought about can be referred to Figure 2 The corresponding steps 201-202 in the embodiments are not described in detail here.

[0082] from Figure 3 It can be seen that Figure 2 Compared with the description of some corresponding embodiments, Figure 3 The process 300 of the split-screen display method in some corresponding embodiments reflects that the size and other attributes of the virtual screen to be created can be determined based on the historical application page usage information set after data cleaning, thereby improving the accuracy of setting the virtual screen attributes, and the size and other attributes of the virtual screen to be created can be determined based on the time and number of times the user uses the application page. The size of the virtual screen used to display the user's use of the application page for a longer time and a larger number of times can be set to a larger size, and the size of the virtual screen used to display the user's use of the application page for a shorter time and a smaller number of times can be set to a smaller size, further improving the user experience and reducing the waste of computing resources.

[0083] Reference below Figure 5 , which shows a schematic diagram of the hardware structure of a head-mounted display device 500 with display function.

[0084] like Figure 5 As shown, the head mounted display device 500 includes a processing device (CPU) 501, a memory (ROM) 502, an input unit 503 and an output unit 504, wherein the processing device 501, the memory 502, the input unit 503 and the output unit 504 are connected to each other through a bus 505. Here, the method according to some embodiments of the present disclosure may be implemented as a computer program and stored in the memory 502. The processing device 501 in the head mounted display device 500 specifically implements the split screen display function defined in the method of some embodiments of the present disclosure by calling the above computer program stored in the memory 502. In some implementations, the input unit 503 may include devices such as a camera, a microphone, a gyroscope, an accelerometer, a magnetometer, etc., and the output unit 504 may be a device that can be used to display content, such as a display screen. The above display screen may be a micro display screen. Therefore, when the processing device 501 calls the above computer program to execute the split screen display function, it may control the input unit 503 to obtain the user's gesture, voice and other operation instructions, and control the output unit 504 to display the display content.

[0085] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0086] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0087] The computer-readable medium may be included in the head-mounted display device; or may exist independently without being assembled into the head-mounted display device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the head-mounted display device, the head-mounted display device: in response to detecting startup information corresponding to the two-dimensional application, display the main application page of the two-dimensional application in the main virtual screen in the three-dimensional virtual space, wherein the three-dimensional virtual space is deployed on the head-mounted display device; in response to detecting startup information corresponding to the two-dimensional application, display the main application page of the two-dimensional application in the main virtual screen in the three-dimensional virtual space, wherein the three-dimensional virtual space is deployed on the head-mounted display device; in response to detecting a page jump selection operation corresponding to the main application page, and the page jump selection operation meets a preset split-screen condition, perform the following split-screen steps: monitor the page jump selection operation; change the attribute information of the application page corresponding to the page jump selection operation to obtain the changed application page, wherein the attribute information includes a source window identification attribute and an activity identification attribute; display the changed application page in the sub-virtual screen in the three-dimensional virtual space, wherein multiple application pages of the two-dimensional application can be displayed on each virtual screen at the same time.

[0088] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0089] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0090] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0091] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A split-screen display method, applied to a head-mounted display device, comprising: In response to detecting startup information of a corresponding two-dimensional application, displaying a main application page of the two-dimensional application in a main virtual screen in a three-dimensional virtual space, wherein the three-dimensional virtual space is deployed in the head mounted display device; In response to detecting a page jump selection operation corresponding to the main application page, and the page jump selection operation meets a preset split-screen condition, performing the following split-screen steps: Monitor and process the page jump selection operation; Performing a change process on the attribute information of the application page corresponding to the page jump selection operation to obtain the changed application page, wherein the attribute information includes a source window identification attribute and an activity identification attribute; The changed application page is displayed in a sub-virtual screen in the three-dimensional virtual space, wherein a plurality of application pages of the two-dimensional application can be displayed on each virtual screen at the same time.

2. The method according to claim 1, wherein: Before displaying the main application page of the two-dimensional application on the main virtual screen in the three-dimensional virtual space, the method further includes: Updating the three-dimensional virtual space deployed in the head mounted display device to obtain the updated three-dimensional virtual space as the three-dimensional virtual space, wherein a preset number of virtual screens are created in the updated three-dimensional virtual space; A virtual screen corresponding to the main application page among a preset number of virtual screens in the three-dimensional virtual space is determined as a main virtual screen.

3. The method according to claim 1, wherein: The updating process of the three-dimensional virtual space deployed in the head mounted display device to obtain the updated three-dimensional virtual space as the three-dimensional virtual space includes: Creating a preset number of texture images in the three-dimensional virtual space; According to the preset number of texture images, the preset number of two-dimensional rectangular canvases are created, wherein the texture images in the preset number of texture images correspond to the two-dimensional rectangular canvases in the preset number of two-dimensional rectangular canvases.

4. The method according to claim 3, wherein: After creating the preset number of two-dimensional rectangular canvases according to the preset number of texture images, the method further includes: Creating a preset number of virtual screens in the three-dimensional virtual space according to a preset set of application pages to be displayed corresponding to the two-dimensional application; The preset number of two-dimensional rectangular canvases are determined as rendering targets corresponding to the preset number of virtual screens, wherein a two-dimensional rectangular canvas in the preset number of two-dimensional rectangular canvases corresponds to a virtual screen in the preset number of virtual screens.

5. The method according to claim 2, wherein: Before displaying the changed application page on the sub-virtual screen in the three-dimensional virtual space, the method further includes: A virtual screen corresponding to the changed application page among a preset number of virtual screens in the three-dimensional virtual space is determined as a sub-virtual screen.

6. The method according to claim 1, wherein: The changing the attribute information of the application page corresponding to the page jump selection operation to obtain the changed application page includes: Determine the attribute value of the source window identification attribute included in the attribute information to be empty; The attribute value of the activity identification attribute included in the attribute information is determined as the new task activity attribute value.

7. A head mounted display device, comprising: one or more processors; A storage device for storing one or more programs; A display screen for forming an image in front of the user's eyes; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images