Page processing method, electronic device, storage medium and program product

By displaying three-dimensional content in the two-dimensional user interaction interface and performing linkage adjustments, the problem of poor display of three-dimensional content in the existing technology is solved, visual effects and user experience are improved, and the good integration of three-dimensional content and two-dimensional interface is achieved.

CN118444816BActive Publication Date: 2024-12-06TAOBAO CHINA SOFTWARE
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410900035.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-05
Publication Date
2024-12-06
Estimated Expiration
2044-07-05

AI Technical Summary

Technical Problem

The prior art is difficult to effectively display and adjust three-dimensional content in a two-dimensional user interaction interface, resulting in poor visual effects and insufficient user experience.

Method used

By providing a three-dimensional display area on the two-dimensional user interaction interface and mapping the texture information of the three-dimensional content to the adapted three-dimensional model, the display and linkage adjustment of the three-dimensional content is realized to adapt to user operations and improve visual effects and user experience.

Benefits of technology

It realizes the display of attractive three-dimensional content in the two-dimensional user interaction interface, improves the user's viewing experience, and solves the adaptation problem between the interface and the content through linkage adjustments, enhancing the naturalness and intuitiveness of user interaction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118444816B_ABST
    Figure CN118444816B_ABST
Patent Text Reader

Abstract

The embodiments of the present application provide a page processing method, an electronic device, a storage medium, and a program product. In the embodiments of the present application, in an extended reality scenario, a three-dimensional display area is provided on a two-dimensional user interaction interface, and a three-dimensional model adapted to the three-dimensional display area is provided. By mapping the texture information of the three-dimensional content on the three-dimensional model, and then displaying the three-dimensional model with texture information within the spatial range corresponding to the three-dimensional display area, the effect of displaying the three-dimensional content in the two-dimensional user interaction interface can be finally achieved, creating a more attractive visual effect and improving the user's viewing experience. Furthermore, in the case of initiating an interactive operation for the two-dimensional user interaction interface, the two-dimensional user interaction interface and the three-dimensional model can be linked and adjusted in response to the interactive operation, so that the three-dimensional content carried by the three-dimensional model is adapted to the two-dimensional user interaction interface in terms of size, position, content, etc.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of machine vision and intelligent systems, and in particular to a page processing method, electronic equipment, storage medium and program product. Background Art

[0002] Spatial computing devices include a range of devices that enable humans to interact with computers in three-dimensional space. For example, Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). In order to create more attractive visual effects, there is a need to display three-dimensional (3D) video in the two-dimensional user interface (2D UI) of the application on spatial computing devices. Summary of the invention

[0003] Multiple aspects of the present application provide a page processing method, device, system and storage medium for displaying three-dimensional content in a two-dimensional user interaction interface. Furthermore, when a user performs an interactive operation on the two-dimensional user interaction interface, the two-dimensional user interaction interface and the three-dimensional content are adjusted in a linkage manner.

[0004] An embodiment of the present application provides a page processing method, which is applied to an extended reality XR device, on which an XR application is running. The method includes: in response to an interface display operation for the XR application, displaying a two-dimensional user interaction interface, the two-dimensional user interaction interface including a three-dimensional display area, and the three-dimensional display area is associated with an initial three-dimensional model adapted to its shape; mapping texture information of three-dimensional content associated with the two-dimensional user interaction interface to the initial three-dimensional model to obtain a target three-dimensional model; and displaying the target three-dimensional model within a spatial range corresponding to the three-dimensional display area to display the three-dimensional content on the two-dimensional user interaction interface.

[0005] An embodiment of the present application provides a page processing method, which is applied to an extended reality XR device, on which an XR application is running. The method includes: responding to a first operation initiated by a user for the XR application, displaying a two-dimensional user interaction interface in a real three-dimensional space, wherein the two-dimensional user interaction interface includes a three-dimensional display area; displaying a three-dimensional content picture in a real space area corresponding to the three-dimensional display area, and aligning a projection position of the three-dimensional content picture on the two-dimensional user interaction interface with the three-dimensional display area; responding to a second operation initiated by the user for the two-dimensional user interaction interface, jointly adjusting visualization properties of the two-dimensional user interaction interface and the three-dimensional content picture, and keeping the projection position aligned with the three-dimensional display area.

[0006] An embodiment of the present application also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement each step in the page processing method provided in the embodiment of the present application.

[0007] The embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the steps in the page processing method provided in the embodiment of the present application.

[0008] The embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is caused to implement each step of the page processing method provided in the embodiment of the present application.

[0009] In an embodiment of the present application, in an extended reality scenario, a three-dimensional display area is provided on a two-dimensional user interaction interface, and a three-dimensional model adapted to the three-dimensional display area is provided. By mapping the texture information of the three-dimensional content on the three-dimensional model, and then displaying the three-dimensional model with texture information within the spatial range corresponding to the three-dimensional display area, it is ultimately possible to achieve the effect of displaying three-dimensional content in a two-dimensional user interaction interface, creating a more attractive visual effect and enhancing the user viewing experience.

[0010] Furthermore, when an interactive operation is initiated for the two-dimensional user interaction interface, the two-dimensional user interaction interface and the three-dimensional model can be linked and adjusted in response to the interactive operation so that the three-dimensional content carried by the three-dimensional model is adapted to the two-dimensional user interaction interface in terms of size, position, content, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1a A flow chart of a page processing method provided for an exemplary embodiment of the present application;

[0013] Figure 1b A flowchart of another page processing method provided for an exemplary embodiment of the present application;

[0014] Figure 2a to Figure 2d A schematic diagram of an interface of an XR application provided for an exemplary embodiment of the present application;

[0015] Figure 3 A schematic diagram of the hierarchical relationship of views of a window of an XR application provided by an exemplary embodiment of the present application;

[0016] Figure 4 A schematic diagram of the internal structure of a three-dimensional content player for an XR application provided by an exemplary embodiment of the present application;

[0017] Figure 5 A schematic diagram of the structure of a page processing device provided for an exemplary embodiment of the present application;

[0018] Figure 6 A schematic structural diagram of an electronic device provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0020] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0021] It should be noted that the drawings in the specification use gray, black and white or color as examples for illustration, but the colors of the elements in the drawings are not actually limited. Color can be used to distinguish between a two-dimensional user interaction interface and the real world, or to distinguish between different display areas (such as a three-dimensional display area or a two-dimensional display area), etc., and is not limited to the colors in the drawings.

[0022] In view of the above technical problems, in an embodiment of the present application, in an extended reality scenario, a three-dimensional display area is provided on a two-dimensional user interaction interface, and a three-dimensional model adapted to the three-dimensional display area is provided. By mapping the texture information of the three-dimensional content on the three-dimensional model, and then displaying the three-dimensional model with texture information within the spatial range corresponding to the three-dimensional display area, the effect of displaying the three-dimensional content in the two-dimensional user interaction interface can be finally achieved, creating a more attractive visual effect and improving the user's viewing experience. Furthermore, in the case of initiating an interactive operation on the two-dimensional user interaction interface, the two-dimensional user interaction interface and the three-dimensional model can be linked and adjusted in response to the interactive operation, so that the three-dimensional content carried by the three-dimensional model is adapted to the two-dimensional user interaction interface in terms of size, position, content, etc.

[0023] The solution provided by the embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0024] Figure 1a The following is a flow chart of a page processing method provided by an exemplary embodiment of the present application. Figure 1a As shown, the method includes:

[0025] 101a. In response to an interface display operation for an XR application, display a two-dimensional user interaction interface, where the two-dimensional user interaction interface includes a three-dimensional display area, and the three-dimensional display area is associated with an initial three-dimensional model adapted to its shape;

[0026] 102a, mapping texture information of the three-dimensional content associated with the two-dimensional user interaction interface onto the initial three-dimensional model to obtain a target three-dimensional model;

[0027] 103a. Display the target three-dimensional model in the space range corresponding to the three-dimensional display area, so as to display the three-dimensional content on the two-dimensional user interaction interface.

[0028] In this embodiment, XR (Extended Reality) technology refers to combining the real world with the virtual world through computers to create a virtual environment that can realize human-computer interaction. This is also a general term for various technologies such as AR, VR, and MR. Devices that use XR technology are called XR devices, which are a type of spatial computing device. The implementation form of XR devices is not limited. For example, XR devices can include but are not limited to: glasses, helmets and other head-mounted devices.

[0029] In this embodiment, the structure of the XR device is not limited. For example, the XR device includes: a display screen, various sensors, a camera, etc., and further includes: an audio device (such as headphones or a microphone), a controller (such as a handle or a touchpad), and a battery and charging system. Depending on the implementation form of the XR device, the various components included will also be different, and the above are only examples. For example, a head-mounted device may not include components such as a handle or a touchpad. Among them, the XR device may include the following components: 1. Display. It can provide a high-resolution visual experience for the XR device and provide a co-immersive experience for the user. 2. Sensors. The sensors in the XR device are crucial for capturing the user's movements, position, and posture. The sensors in the XR device may include but are not limited to: optical sensors, motion sensors, depth sensors, environmental sensors, etc. Among them, optical sensors are often used to track the user's eye movements to achieve eye tracking functions. Eye tracking contributes to more natural and intuitive user interactions, for example, controlling a virtual interface or selecting an object through the user's line of sight. The optical sensor also includes: a camera that can capture images of the real world and accurately superimpose virtual elements on these images. Cameras can also be used for environmental recognition, helping devices understand the user's surroundings in order to provide a more intelligent and personalized experience. Motion sensors are used to capture the user's head and hand movements in real time and convert them into corresponding operations in the virtual environment, thereby providing a basis for the interaction between spatial computing devices and users. For example, motion sensors capture whether the user initiates a one-hand pinch operation or a one-hand release operation. Depth sensors can measure the distance between the user and objects in the real world, which is critical for accurately superimposing virtual objects into the real world. Depth sensors help spatial computing devices understand the user's surroundings and provide a more realistic and natural experience by measuring the depth information of the surface of an object. Environmental sensors include temperature sensors, humidity sensors, and light sensors, which are used to monitor the environmental conditions around spatial computing devices. Environmental sensors help devices automatically adjust parameters such as display brightness and temperature control to provide a more comfortable user experience.

[0030] In this embodiment, the XR device can display a virtual screen, and the virtual screen includes virtual objects. The virtual screen can be the screen presented by the XR device running various XR applications. The XR application can be various application programs installed on the XR device (such as various 3D games, 3D shopping software, 3D smart home software, etc.), or it can be a desktop program or other system program of the XR device, etc., without limitation. In this embodiment, the XR application has a two-dimensional user interaction (2D UI) interface and supports embedding three-dimensional content on the two-dimensional user interaction interface. XR applications are different from 2D applications on traditional terminal devices. XR applications can be 3D applications or new forms of 2D applications embedded with 3D content.

[0031] In this embodiment, the user can initiate an interface display operation for the XR application on the XR device. For example, when the XR device displays a virtual desktop, the virtual desktop includes icons of various XR applications. The user can initiate an interface display operation for the XR application being watched by looking at the icon of the XR application. Alternatively, when the XR device displays a page in the XR application, the user can initiate an interface display operation for the XR application by clicking or double-clicking the page control. It should be noted that in addition to being triggered by the user, the interface display operation can also be triggered by an application event corresponding to the XR application or by a system event corresponding to the XR device. State changes or behaviors within the application, for example, completion of data loading within the application, completion of a certain operation, etc. System events are related to behaviors or state changes at the operating system level. System events include changes in the file system, changes in network status, execution status of other programs, and some behaviors of the system itself. It should be noted that the above introduction to application events and system events also applies to the following embodiments.

[0032] In this embodiment, the XR device can display a two-dimensional user interaction interface in response to an interface display operation for an XR application. The two-dimensional user interaction interface may include but is not limited to: the home page, list page, content page or detail page of the XR application. The two-dimensional user interaction interface includes a three-dimensional display area. Although it is called a three-dimensional display area, the three-dimensional display area is a partial area on the two-dimensional user interaction interface, not a three-dimensional display space. The three-dimensional display area is a display window for three-dimensional content on the two-dimensional user interaction interface, which is responsible for creating a visual effect of displaying three-dimensional content on the two-dimensional user interaction interface, so that the user has a feeling of watching three-dimensional content through a window, so it is called a three-dimensional display area. It is explained here that the three-dimensional content is not directly displayed in the three-dimensional display area, but is displayed in the real space area corresponding to the three-dimensional display area. The three-dimensional content can be a three-dimensional image or a three-dimensional video. Further, the two-dimensional user interaction interface also includes: a two-dimensional display area, which is also a partial area on the two-dimensional user interaction interface, used to display various two-dimensional contents such as two-dimensional images, two-dimensional videos, text information and two-dimensional components. For example, the two-dimensional user interaction interface is implemented as a video waterfall list page, which includes multiple small cards, and a small card can be a three-dimensional display area or a two-dimensional display area.

[0033] In this embodiment, the three-dimensional display area is associated with an initial three-dimensional model that is adapted to the shape of the three-dimensional display area. The initial three-dimensional model is a carrier of three-dimensional content, and is responsible for cooperating with the three-dimensional display area to display the three-dimensional content, thereby creating a visual effect of displaying the three-dimensional content on the two-dimensional user interaction interface. Depending on the two-dimensional shape of the three-dimensional display area, the three-dimensional form of the initial three-dimensional model is also different. For example, if the shape of the three-dimensional display area is a rounded rectangle, the shape of the initial three-dimensional model can be a rounded cube that matches the rounded rectangle, or it can also be other three-dimensional forms that can be fully displayed by the rounded rectangle, such as an ellipsoid or a sphere; if the shape of the three-dimensional display area is a semicircle, the shape of the initial three-dimensional model is a hemispherical shape that matches the semicircle, or it can also be other three-dimensional forms that can be fully displayed by the semicircle, such as a cube; if the shape of the three-dimensional display area is a circle, the shape of the initial three-dimensional model is a sphere that matches the circle, or it can also be other three-dimensional forms that can be fully displayed by the circle, such as a cube, a cube, etc.

[0034] In this embodiment, the texture information of the three-dimensional content associated with the two-dimensional user interaction interface is mapped to the initial three-dimensional model to obtain a target three-dimensional model. The target three-dimensional model is a three-dimensional model that carries three-dimensional content. By mapping the texture information of different three-dimensional content to the initial three-dimensional model, the target three-dimensional model will carry different three-dimensional content. Optionally, relatively independent three-dimensional content can be displayed in the three-dimensional display area, that is, it has nothing to do with the two-dimensional content in other two-dimensional display areas; three-dimensional content related to the two-dimensional content in the two-dimensional display area can also be displayed. For example, when displaying product information in a two-dimensional display area, the three-dimensional display area can be associated with the three-dimensional content corresponding to the product information. The three-dimensional content can be a three-dimensional model of the product information to display the product in three dimensions, or a three-dimensional scene image where the product information is located to display the three-dimensional scene where the product information is located, or a three-dimensional video that introduces and explains the product information, etc.

[0035] Mapping the texture information of the three-dimensional content onto the initial three-dimensional model includes: obtaining texture information from the three-dimensional content, and mapping the texture information onto the initial three-dimensional model. The three-dimensional content may include a three-dimensional video or a three-dimensional image. The implementation methods for extracting texture information for the three-dimensional video or the three-dimensional image are the same or similar. The following is an example of extracting texture information for the three-dimensional video. 1. Decoding. Convert the compressed three-dimensional video into a displayable video frame. The decoded video frame is usually an uncompressed red, green, and blue (RGB) image, which contains color information and brightness information of the three-dimensional video frame. 2. Image preprocessing. Before extracting texture information, the video frame usually needs to be preprocessed. This may include converting the video frame into a grayscale image or a color image, as well as operations such as scaling, cropping, and rotating. 3. Image filtering. Image filtering is the process of suppressing the noise of the video frame while retaining the image detail features as much as possible. Before extracting texture information, using a filter (such as a Gaussian filter or a median filter, etc.) can reduce the noise in the video frame and improve the accuracy of subsequent feature extraction. 4. Edge detection. Edge detection is the process of identifying pixels with obvious brightness changes in video frames, which helps to extract contours and boundary information in video frames. In the process of texture information extraction, edge detection is used to identify the boundaries and patterns of textures. 5. Feature extraction. Feature extraction is the process of abstracting information in video frames into feature vectors, which usually involves color analysis, texture analysis, etc. Color feature extraction: extract the value of the RGB color channel of each pixel, and then calculate its statistical features, such as color histogram, color mean or variance. Texture feature extraction: calculate the relationship between pixels in the local area of ​​the video frame to represent texture features. For example, the local binary pattern (LBP) algorithm is used to calculate the texture features of each pixel, and then the texture feature vector is constructed by counting the frequency of occurrence of different textures. 6. Feature representation and normalization. The extracted color features and texture features need to be combined into a feature vector, and usually need to be normalized to ensure that different features have equal influence on the final classification result.

[0036] The implementation method of mapping the texture information of the three-dimensional content onto the initial three-dimensional model is not limited. For example, the texture information can be mapped onto the surface of the initial three-dimensional model using UV mapping technology. UV mapping is also called UV mapping, and UV mapping creates a two-dimensional coordinate system for the surface of the initial three-dimensional model that is adapted to the surface morphology of the initial three-dimensional model, so that the texture information can be "wrapped" around the initial three-dimensional model. The two-dimensional coordinate system can be a spherical, ellipsoidal or other curved surface coordinate system, or a plane coordinate system, depending on the surface morphology of the initial three-dimensional model. In this way, each point on the texture information corresponds to a point on the surface of the initial three-dimensional model, thereby achieving accurate mapping of the texture. For example, when drawing a brick wall, a realistic brick wall image can be used as a texture, and the texture corresponding to the brick wall can be mapped to the rectangular surface corresponding to the initial three-dimensional model, thereby quickly creating a realistic brick wall effect.

[0037] In this embodiment, the target 3D model is displayed in the space range (i.e., the real space area) corresponding to the 3D display area, so as to achieve the purpose of displaying 3D content on the 2D user interaction interface. Figure 2a In the 3D display area (e.g., the upper left corner area) of the 2D user interface, a 3D video with depth of field is placed, so that the user can feel the feeling of looking at the scenery through the window. Figure 2a Also displayed are products C1 to C5, and product description information D1 to D5 corresponding to each product, wherein product C1 is displayed through a three-dimensional video, and products C2 to C5 are displayed through a two-dimensional video. In addition, optionally, the two-dimensional user interaction interface may also include interactive controls, which are used to switch or jump the two-dimensional user interaction interface. For example, the two-dimensional user interaction interface is a video list page, and the interactive control points to the home page. In response to a trigger operation on the interactive control, it can jump from the video list page to the home page. For example, in Figure 2a In the example, the two-dimensional user interaction interface also includes four interactive controls, namely E1 to E5. For example, the interactive control E1 points to the home page of the XR application, and the XR device can display the home page of the XR application in response to the user's click operation on the interactive control E1; the interactive control E2 is a product search control, and the XR device can display the search control in response to the user's click operation on the interactive control E2. When the user enters product information in the search control, the XR device displays the product list page in response to the search operation; the interactive control E3 is a list page, and the XR device can display the default list page in response to the user's click operation on the interactive control E3; the interactive control E3 points to the detailed information of the XR application, such as the name, version number, etc., and the XR device can display the detailed information of the XR application in response to the user's click operation on the interactive control E4.

[0038] For example, in Figure 2b In the video list page, the two-dimensional user interaction interface is implemented as a video list page, also known as a video waterfall page. When the user browses the page, one of the three-dimensional contents is played, and the cover information of other unplayed three-dimensional contents or two-dimensional contents is displayed. The three-dimensional content selected for playback can be the three-dimensional content at the center of the page, the three-dimensional content that the user is looking at, or a three-dimensional content randomly selected from the current display window for display. Figure 2b In the example, product F1, product F2, ..., product F5 are displayed, wherein product F3 is a 3D video being played, and product F1, product F2, product F4, and product F5 display covers for illustration purposes, but are not limited thereto. Figure 2b In the example, the subviews corresponding to product F1 and product F5 are partially displayed. Figure 2b and Figure 2c The two-dimensional user interface displayed is the same, the difference is that Figure 2c The two-dimensional user interaction interface is displayed in the real world. The two-dimensional user interaction interface can be moved and scaled as a whole, and the interface content in the user interaction interface can also be slid up and down.

[0039] For example, in Figure 2d In the process of displaying the 3D content, the interface can also overlay a subview, which can be a transparent view, including product description information, the merchant nickname and interactive controls. The XR device can jump to the product details page in response to the trigger operation on the interactive control.

[0040] Among them, the detailed introduction of the moving operation, zooming operation, sliding operation and displaying sub-views can be found in the aforementioned embodiments, which will not be repeated here.

[0041] In an embodiment of the present application, a three-dimensional display area is provided on a two-dimensional user interaction interface, and a three-dimensional model adapted to the three-dimensional display area is provided. By mapping the texture information of the three-dimensional content on the three-dimensional model, and then displaying the three-dimensional model with texture information within the spatial range corresponding to the three-dimensional display area, it is ultimately possible to achieve the effect of displaying three-dimensional content in a two-dimensional user interaction interface, creating a more attractive visual effect and enhancing the user viewing experience.

[0042] In an optional embodiment, considering that the two-dimensional user interaction interface and the target three-dimensional model are rendered by different rendering engines, an adjustment operation is initiated for the two-dimensional user interaction interface, the two-dimensional user interaction interface changes, while the target three-dimensional model does not change, which may cause the two-dimensional user interaction interface and the target three-dimensional model to be inconsistent and mismatched. For example, the user performs a zoom operation on the two-dimensional user interaction interface, and the two-dimensional user interaction interface is reduced or enlarged. However, the size of the target three-dimensional model is not reduced or enlarged, and the original size is maintained. In this way, the target three-dimensional model and the three-dimensional display area on the two-dimensional user interaction interface are not compatible. For another example, the user initiates a sliding operation on the two-dimensional user interaction interface, and the two-dimensional user interaction interface is slid upward. Accordingly, the three-dimensional display area slides upward with the two-dimensional user interaction interface, and the position of the target three-dimensional model remains unchanged, which will cause the target three-dimensional model to overlap with the two-dimensional user interaction interface, and the target three-dimensional model will block the content on the two-dimensional user interaction interface.

[0043] In response to the above problems, the XR device can respond to adjustments to the two-dimensional user interaction interface, adjust the visualization attributes of the two-dimensional user interaction interface, follow the changes in the visualization attributes of the two-dimensional user interaction interface, and make linkage adjustments to the visualization attributes of the target three-dimensional model so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area. Visual attributes include: at least one of interface size, interface posture, interface brightness, and interface content. Among them, the visualization attributes can be implemented entirely at the application layer, entirely at the system layer, partially at the application layer, and partially at the system layer. For example, the adjustment of the interface size, interface brightness, and interface content is implemented at the application layer, and the adjustment of the interface posture is implemented at the system layer.

[0044] The adjustment initiated for the two-dimensional user interaction interface is triggered by a user operation. For the convenience of distinction and description, the user operation that triggers the adjustment for the two-dimensional user interaction interface is referred to as a target user operation.

[0045] The target user operation may include but is not limited to: zoom operation, move operation, brightness adjustment operation and sliding operation. Zoom operation includes the operation of zooming in or out of the two-dimensional user interaction interface, that is, the zoom operation can adjust the interface size of the two-dimensional user interaction interface. Sliding operation is a user operation of sliding up and down or left and right on the interface content of the two-dimensional user interaction interface, that is, the sliding operation can adjust the position of the interface content of the two-dimensional user interaction interface. Moving operation refers to the operation of moving the entire two-dimensional user interaction interface, for example, zooming in or out, moving the two-dimensional user interaction interface to the left or right, etc. Moving operation can adjust the posture of the two-dimensional user interaction interface. For example, the posture includes position and orientation.

[0046] Depending on the target user interaction operation, the method of initiating the target user operation is different, and the method of the XR device responding to the target user operation is also different. An exemplary description is given below.

[0047] Example A1, when the target user operation is implemented as a zoom operation, for example, the way to initiate the zoom operation can be the user's one-hand pinch or open operation, "pinch" corresponds to a zoom-out operation, and "open" corresponds to a zoom-in operation. The zoom ratio is related to the pinching distance. For another example, the zoom operation can be implemented as the relative movement of both hands in the pinching state, and the zoom multiple is controlled according to the change in the distance of the line connecting the pinch points of the two hands. Based on this, the XR device can respond to the zoom operation on the two-dimensional user interaction interface and adjust the interface size of the two-dimensional user interaction interface.

[0048] Example A2, in the case where the target user operation is implemented as a sliding operation, for example, the sliding operation is implemented as a one-hand click slide or a one-hand pinch slide, and the sliding distance and sliding direction of the one-hand correspond to the sliding distance and sliding direction of the content on the two-dimensional user interaction interface. For example, the sliding distance of the one-hand pinch operation is 1cm, and the sliding direction is upward. Accordingly, the content on the two-dimensional user interaction interface slides upward by 1cm. For another example, the two-dimensional user interaction interface includes a sliding control, and the user slides the content on the two-dimensional user interaction interface by sliding the sliding control. Based on this, the XR device responds to the sliding operation on the two-dimensional user interaction interface and adjusts the interface content of the two-dimensional user interaction interface.

[0049] Example A3, when the target user operation is implemented as a moving operation, for example, the moving operation is implemented as a single-hand palm moving operation, that is, a moving operation in a single-hand palm open state, and the moving distance and moving direction of the single-hand palm moving operation correspond to the moving distance and moving direction of the two-dimensional user interaction interface, respectively. Based on this, the XR device responds to the moving operation on the two-dimensional user interaction interface and adjusts the interface posture of the two-dimensional user interaction interface. For example, the position control of the XR application is called, the position and posture are sensed according to the position control, and the position and posture of the two-dimensional user interaction interface are adjusted according to the position and posture.

[0050] It should be noted that the target user operation initiated by the user may be one or more of a zoom operation, a sliding operation, and a move operation. Accordingly, the above examples A1, A2, and A3 may be used alone or in combination. For example, the user first initiates a zoom operation and then a sliding operation; for another example, the user first initiates a move operation and then a sliding operation; for another example, the user first initiates a move operation, multiple sliding operations, and multiple zoom operations in sequence.

[0051] Optionally, in addition to being triggered by the target user's operation, the adjustment of the two-dimensional user interaction interface can also be triggered by an application event of the XR application that the user "does not perceive", or by a system event of the XR device. For example, in some scenarios, it may be necessary to dynamically update the content, size or style of the interaction interface. In these scenarios, when the set time point arrives or a set event occurs, the adjustment of the two-dimensional user interaction interface is automatically initiated by the application event or system event.

[0052] For example, when the XR device detects that the light in the environment where the XR device is located is dim, the system event automatically initiates an interface brightness adjustment operation for the two-dimensional user interaction interface, adjusts the interface brightness of the two-dimensional user interaction interface, and follows the changes in the interface brightness of the two-dimensional user interaction interface. The interface brightness of the target three-dimensional model is adjusted in conjunction to make the adjusted target three-dimensional model adapt to the brightness of the adjusted three-dimensional display area.

[0053] For another example, for the sake of eye health, application events of the XR device can be set in advance. The application event includes a timer. When the timer ends, the adjustment of the position or size of the two-dimensional user interaction interface is automatically triggered, such as adjusting farther or closer, moving left or right, zooming in or out of the page, etc. At this time, the position or size of the two-dimensional user interaction interface can be adjusted in response to the application event; the position or size of the target three-dimensional model can be adjusted in conjunction with the change in the position or size of the two-dimensional user interaction interface, so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

[0054] In an optional embodiment, during the development of the XR application, a view for perceiving interactive operations may be added to the two-dimensional user interaction interface of the XR application. Among them, a view may be added for perceiving zoom operations, sliding operations, and moving operations, etc. A view for detecting each interactive operation may also be added, and different interactive operations correspond to different views. Furthermore, in order to reduce the impact on existing views, the view may be implemented as a transparent view, or a view of other transparency, which is not limited.

[0055] The position of the view corresponding to the interactive operation in the display window corresponding to the XR application is not limited. The following takes the setting of corresponding views for zoom operation and sliding operation as an example to illustrate the hierarchy of views contained in the XR application. For the convenience of distinction and description, the view that monitors the zoom operation is called the zoom view, and the view that monitors the sliding operation is called the sliding view. For example, the XR application corresponds to a main window, also known as a display window. The main window is the main area in the XR application for displaying content and interacting with the user. The main window is associated with a main view, which can provide the main functions and interface elements of the XR application. The size of the main view can be the same as the size of the main window. The upper layer of the main view is the zoom view, and the upper layer of the zoom view is the interface view. The interface view can include at least one first-level sub-view, and each first-level sub-view can include one or more second-level sub-views.

[0056] Among them, for the convenience of distinction and description, the first-level interface sub-view that displays three-dimensional content is called the first-level sub-view of the three-dimensional interface, and the first-level interface sub-view that displays two-dimensional content is called the first-level sub-view of the two-dimensional interface; the first-level sub-view of the two-dimensional interface includes, in order from bottom to top: the first sliding sub-view and the second-level sub-view of the two-dimensional interface; the first-level sub-view of the three-dimensional interface includes, in order from bottom to top: the second sliding sub-view and the second-level sub-view of the three-dimensional interface. It should be noted that the size of each view is not limited. For example, the size information of the zoom view, the main view and the main window is the same. The first sliding sub-view and the second sliding sub-view are respectively the same as the size information of their first-level interface sub-views. The second-level sub-view of the three-dimensional interface or the second-level sub-view of the two-dimensional interface are respectively the same as the size information of their corresponding first-level interface sub-views. Figure 3 As shown, the hierarchical relationship of each view of the XR application corresponding display window is exemplified, but is not limited to this.

[0057] The following is an exemplary description of an implementation method for linkage adjustment of a two-dimensional user interface and a target three-dimensional model with respect to the interactive operations monitored by the above views.

[0058] Example B1, scaling operation.

[0059] A specific implementation method of responding to a zoom operation on a two-dimensional user interaction interface and adjusting the interface size of the two-dimensional user interaction interface includes: using a zoom view bound to a display window to which the two-dimensional user interaction interface belongs to perceive the zoom operation on the two-dimensional user interaction interface; in the case of perceiving the zoom operation, determining the zoom ratio and zoom direction of the two-dimensional user interaction interface according to the zoom parameters of the zoom operation, and adjusting the interface size of the two-dimensional user interaction interface according to the zoom ratio and zoom direction, wherein, since the two-dimensional user interaction interface includes a three-dimensional display area, the three-dimensional display area is zoomed following the zoom operation. The zoom parameters are different according to different zoom operations. For example, the zoom operation is a two-handed zoom operation, the zoom parameter can be the zoom trajectory information corresponding to the two hands, the zoom ratio can be the multiple of the zoom trajectory information relative to the initial distance between the two hands, and the zoom direction can be the change information of the center point of the line connecting the two hands. For another example, the zoom operation is a click control, and the control corresponds to a default parameter, for example, the zoom ratio is 50% or 75%, etc., and the zoom direction is that the center point of the two-dimensional user interaction interface remains unchanged.

[0060] When the zoom view detects a zoom operation, the target three-dimensional model is adjusted in linkage, taking into account the size information of the target three-dimensional model and whether the display position is aligned. A specific implementation method of following the changes in the visualization attributes of the two-dimensional user interaction interface to adjust the visualization attributes of the target three-dimensional model in linkage so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area includes: adjusting the size of the target three-dimensional model according to the size of the zoomed three-dimensional display area, or adjusting the size of the target three-dimensional model according to the zoom ratio of the zoomed three-dimensional display area; aligning the display position of the adjusted target three-dimensional model with the display position of the zoomed three-dimensional display area so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

[0061] Optionally, considering the model display capability provided by the three-dimensional rendering engine under the XR device, "meter" is usually used as the size unit so that the target three-dimensional model can be integrated with the real world. The user interface drawing capability provided by the user interface (UI) library of the XR device is usually in "pixel points" as the size unit. Therefore, if you want the target three-dimensional model to be well integrated with the two-dimensional user interaction interface, you can correctly maintain the conversion relationship between "meter" and "pixel points" in the XR application. When the XR application is enlarged as a whole, the page size of the XR application becomes larger, and the conversion relationship between "meter" and "pixel points" is changed; when the XR application is reduced as a whole, the page size of the XR application becomes smaller, and the conversion relationship between "meter" and "pixel points" is also changed. At this time, the conversion relationship between "meter" and "pixel points" after the scaling operation can be determined based on the conversion relationship between "meter" and "pixel points" before the scaling operation, combined with the scaling ratio, so as to timely update the size of the target three-dimensional model based on the conversion relationship between "meter" and "pixel points" after the scaling operation. It is explained here that using "meter" as the size unit to measure the size of the target three-dimensional model is only an example and is not limited to this.

[0062] For example, before the zoom operation, the conversion relationship between "meter" and "pixel" is 1 / 1360 (unit: meter / pixel), and the size information of the two-dimensional user interaction interface is 1280×720 pixels. The two-dimensional user interaction interface is converted to the length information in the unit of "meter", 1280×1 / 1360=0.94 meters, 720×1 / 1360=0.53 meters, that is, 0.94×0.53. After the zoom operation, the two-dimensional user interaction interface changes from 0.94×0.53 to 1.88×1.06, that is, the two-dimensional user interaction interface is enlarged by 2 times, and the zoom ratio is 2. If the zoom ratio is 2, the conversion relationship between "meter" and "pixel" changes from 1 / 1306 to 2 / 1360, and the size information of the two-dimensional user interaction interface is updated to: 1280×2 / 1360=1.88 meters, 720×2 / 1360=1.06 meters, that is, 1.88×1.06. If the size of the target 3D model is 0.5×0.2 before the scaling operation, the size of the target 3D model after the scaling operation is: 0.5×2=1 meter, 0.2×2=0.4 meter.

[0063] Example B2, sliding operation.

[0064] An implementation method for adjusting the interface content of a two-dimensional user interaction interface in response to a sliding operation on the two-dimensional user interaction interface includes: using a first sliding subview in an interface view to which the two-dimensional user interaction interface belongs to perceive the sliding operation on the two-dimensional user interaction interface; when the sliding operation is perceived, determining the target interface content of the corresponding display window in the two-dimensional user interaction interface according to the sliding parameters of the sliding operation and the position of the display window to which the two-dimensional user interaction interface belongs, and displaying the target interface content in the display window, wherein the three-dimensional display area slides following the sliding operation. For example, the position of the task display window may remain unchanged, and the two-dimensional user interaction interface may be moved from an initial position according to the sliding parameters of the sliding operation to obtain a final position of the two-dimensional user interaction interface, and based on the position information of the two-dimensional user interaction interface and the display window at the final position, the display content of the display window may be determined.

[0065] An implementation method for adjusting the visualization attributes of a target three-dimensional model in conjunction with changes in the visualization attributes of a two-dimensional user interaction interface so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area, including: using a second sliding sub-view to receive a notification message sent by a first sliding sub-view; the second sliding sub-view is a sub-view in a sub-view that carries a three-dimensional display area in an interface view to which the two-dimensional user interaction interface belongs, and a detailed introduction can be found in the aforementioned embodiment; adjusting the display position of the target three-dimensional model according to the sliding distance and sliding direction included in the notification message; determining the display content in the display window according to the display position of the target three-dimensional model so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

[0066] Among them, after the sliding operation, if it is determined that the target three-dimensional model is entirely located in the display window according to the display position of the target three-dimensional model, the target three-dimensional model is completely displayed in the display window according to the display position of the target three-dimensional model; if it is determined that part of the target three-dimensional model is located in the display window according to the display position of the target three-dimensional model, preset cover information is displayed in a partial area of ​​the display window according to the display position of the target three-dimensional model, and the cover information may be a two-dimensional image obtained by rendering the target three-dimensional model; if it is determined that the target three-dimensional model is not located in the display window according to the display position of the target three-dimensional model, the target three-dimensional model is no longer displayed in the display window, and the display content of the display window is determined according to the display position of the two-dimensional user interaction interface.

[0067] In an optional embodiment, the upper layer of the three-dimensional interface subview also includes a display subview. The three-dimensional interface subview is a first-level subview, and the display subview is a second-level subview. The display subview may have the same size information as the three-dimensional interface subview. The display subview includes at least one interactive control and description information corresponding to the three-dimensional content. Based on this, at least one interactive control and description information corresponding to the three-dimensional content can be displayed on the target three-dimensional model through the display subview. The interactive control can be a control for fast forward, rewind or double speed for the three-dimensional content, or it can be an interactive control for jumping to other interfaces. For example, if the three-dimensional content is implemented as a commodity, the interactive control can point to the detailed interface of the commodity. The description information of the three-dimensional content may include but is not limited to: basic information, state description, feature description or function description of the three-dimensional content. Optionally, considering that the interactive controls and description information displayed on the display subview are not blocked by the target three-dimensional model, it is desired to fully display the content on the display subview, and the height information of the display subview can be determined according to the thickness information of the target three-dimensional model, and the display subview is added above the target three-dimensional model.

[0068] Accordingly, when the user triggers any interactive control, the XR device can respond to the triggering operation for any interactive control and perform an interactive operation corresponding to the interactive control on the three-dimensional content. For example, the three-dimensional content is implemented as a three-dimensional video of a product, and the interactive control is implemented as a jump to the details interface of the product. The user clicks the interactive control, and the XR device responds to the click operation and jumps from the current interface to the details interface of the product. For another example, the three-dimensional content is implemented as a three-dimensional video of a product, and the interactive control is implemented as a video progress bar. The user clicks a certain position of the progress bar, and the XR device responds to the click operation and jumps to the display screen of the target three-dimensional model corresponding to the position of the progress bar.

[0069] Among them, by displaying sub-views, the hierarchical relationship between three-dimensional content and other interface elements (such as interactive controls or descriptive information) can be maintained, and these implementation details are transparent to XR application developers. Developers can simply regard three-dimensional content videos as part of the two-dimensional user interaction interface without the need to manually maintain the relationship between three-dimensional content and the two-dimensional user interaction interface, thereby improving developers' development efficiency for XR applications and reducing development costs.

[0070] In an optional embodiment, depending on the implementation of the XR device, the implementation method of displaying the two-dimensional user interaction interface in response to the interface display operation for the XR application is also different.

[0071] For example, the XR device is implemented as an MR device, the XR application is implemented as an MR application, and the MR screen includes a target position corresponding to the three-dimensional display area, that is, the three-dimensional content is displayed at the target position of the MR screen. An implementation method of displaying a two-dimensional user interaction interface in response to an interface display operation for an XR application includes: displaying a two-dimensional user interaction interface in an augmented reality screen in response to an interface display operation for an MR application. Specifically, the position of the two-dimensional user interaction interface can be adjusted according to the target position so that the three-dimensional display area corresponds to the target position. For example, the target is a rectangular area, and the three-dimensional display area is also a rectangular area. The correspondence between the three-dimensional display area and the target position indicates that the rectangular area of ​​the three-dimensional display area coincides with the three-dimensional area of ​​the target position. For another example, the target position is a position point, and the three-dimensional display area corresponds to a center point. The correspondence between the three-dimensional display area and the target position indicates that the center point of the three-dimensional display area coincides with the target position.

[0072] The position of the two-dimensional user interaction interface can be automatically adjusted according to the target position so that the three-dimensional display area corresponds to the target position. In addition, the position of the two-dimensional user interaction interface can be manually adjusted, and the MR device can adjust the position information of the two-dimensional user interaction interface in response to the adjustment operation of the display position of the two-dimensional user interaction interface until the three-dimensional display area corresponds to the target position.

[0073] For another example, the XR device is implemented as a VR device, the XR application is implemented as a VR application, the two-dimensional user interaction interface is implemented as a virtual screen, the position of the virtual screen is not limited, and in response to an interface display operation for the virtual reality application, the two-dimensional user interaction interface is displayed in the virtual reality screen, and the two-dimensional user interaction interface includes a three-dimensional display area.

[0074] In an optional embodiment, the XR application does not include a 3D content player, and the 3D content is displayed by the 3D content player. The internal implementation structure of the 3D content player is not limited. Figure 4 As shown, the 3D content player includes: an interaction layer 41, a kernel layer 42 and an operating system layer 43. The interaction layer is used to display a 2D user interaction interface in response to an interface display operation for an XR application, and the 2D user interaction interface includes a 3D display area; the operating system layer is used to create an initial 3D model associated with the 3D display area and adapted to its shape; the kernel layer is used to map the texture information of the 3D content associated with the 2D user interaction interface to the initial 3D model to obtain a target 3D model, and provide the target 3D model to the interaction layer; the interaction layer is used to display the target 3D model within the spatial range corresponding to the 3D display area, so as to realize the display of 3D content on the 2D user interaction interface.

[0075] The interactive layer is implemented as the interactive layer (XREnhancedPlayer-UI) of the three-dimensional content player of the XR application. The interactive layer can respond to the interface display operation for the XR application and display the two-dimensional user interaction interface. The two-dimensional user interaction interface includes: a three-dimensional display area. The interactive layer can provide a user interface for the three-dimensional content player, encapsulate the view corresponding to the target three-dimensional model, and can be connected to the interactive layer, so that when developing applications, the three-dimensional content can be quickly and conveniently embedded in the two-dimensional user interaction interface. Furthermore, the view of the perceived interactive operation mentioned above can be implemented by the interactive layer, and according to the perceived interactive operation, the size, position and content of the two-dimensional user interaction interface and the target three-dimensional model can be linked and adjusted through the kernel layer 42 and the operating system from 43.

[0076] exist Figure 4 In the 3D content player, the operating system layer is the operating system (XRAppOS). When developing a 2D user interaction interface, the shape of the 3D display area can be determined. The operating system layer can create an initial 3D model that matches the shape of the 3D display area based on the shape of the 3D display area. Optionally, the initial 3D model and the shape of the 3D display area can be the same shape, the same size, and the size is consistent. It can also be similar in shape, and the shape size of the initial 3D model is not larger than the shape scale of the 3D display area, etc. There is no specific limitation on this. The operating system layer can also call the 3D rendering engine to render the target 3D model, so that the interactive layer can display the rendered target 3D model, and finally achieve the purpose of displaying 3D content. Among them, the 3D rendering engine is a library function in the system library provided by the operating system of the XR device, such as but not limited to RealityKit, which is responsible for providing XR applications with rendering capabilities for displaying 3D content, and providing structures of multiple initial 3D models such as spheres, hemispheres, and rectangular planes. Optionally, the initial 3D model can be a mesh structure, but is not limited to this.

[0077] The kernel layer (XREnhancedPlayer-Core) is the core of the 3D content player. It is mainly responsible for decoding the 3D content and extracting the texture information. It obtains the texture information of the 3D content and maps the texture information to the initial 3D model to obtain the target 3D model. The target 3D model is provided to the operating system layer so that the operating system can render the target 3D model by calling the 3D rendering engine.

[0078] Further, in the process of dynamically adjusting the visualization attributes of the target three-dimensional model following the adjustment of the visualization attributes of the two-dimensional user interaction interface, the interaction layer 41 perceives the target user operation through the view of the perception interaction operation provided by it. When the target user operation is perceived, on the one hand, the kernel layer 42 is notified, and the kernel layer 42 determines the new texture information and the size of the initial three-dimensional model according to the target user operation, maps the new texture information to the new initial three-dimensional model to obtain a new target three-dimensional model, and provides it to the operating system layer 43; it is explained here that if the target user operation is a sliding operation, the new texture information is different from the original texture information, the size of the initial three-dimensional model remains unchanged, and accordingly, the size of the new target three-dimensional model remains unchanged, but the three-dimensional content carried changes; if the target user operation is the above-mentioned zoom operation or move operation, the new texture information is the same as the original texture information, the size or position of the initial unit model changes, and accordingly, the size or position of the new target three-dimensional model changes, and the three-dimensional content carried does not change. For the operating system layer 43, the new target three-dimensional model provided by the kernel layer 42 can be received, and the new target three-dimensional model is rendered, and the rendered new target three-dimensional model is provided to the interaction layer 41. The interaction layer 41 is also used to call the two-dimensional rendering engine in the system library provided by the operating system to re-render the two-dimensional user interaction interface according to the target user operation, and call the library function in the system library responsible for aligning the three-dimensional display area with the target three-dimensional model (or simply referred to as the position management library function) to align the new target three-dimensional model with the adjusted three-dimensional display area, so as to achieve the linkage adjustment of the two-dimensional user interaction interface and the target three-dimensional model in terms of size, position and content. In this example, the content update, size and position change of the target three-dimensional model and the two-dimensional user interaction interface are implemented by the application layer (i.e., the three-dimensional content player), and the position alignment between the target three-dimensional model and the three-dimensional display area is implemented in the system layer of the XR device (i.e., the library function in the system library) as an example for explanation.

[0079] It should be noted that the above description uses a 3D content player as part of an XR application for illustrative purposes, but the technical implementation is not limited to this. The 3D content player can be implemented at the system layer, or as a middle platform framework between the system layer and the application layer, and there is no limitation on this.

[0080] In this embodiment, a 3D content player can realize the integration of 3D content (such as 3D video and 3D images) and 2D user interaction interface based on the traditional 3D rendering engine and the 2D rendering engine, without introducing a 3D rendering engine that includes both the 2D user interaction interface and the 3D content playback, thereby making full use of the powerful user interface (UI) development capabilities provided by the native platform.

[0081] In this embodiment, sliding operations and zooming operations are implemented on the three-dimensional content player within the XR application, which belongs to the implementation of the application layer. The overall movement of the XR application window (such as adjusting closer or farther away) is the movement of the entire XR application together. The three-dimensional content is part of the XR application. This embodiment focuses on aligning the three-dimensional content with the three-dimensional display area of ​​the two-dimensional user interaction interface, and ensuring that the position of the three-dimensional display area corresponds to the position of the two-dimensional user interaction interface. In this way, when the XR application is moved as a whole, the three-dimensional content can be moved together to the corresponding position. In the case of the above-mentioned overall mobile XR application, it is implemented by the operating system (OS) of the XR device through the user interface framework provided by the system dynamic library, that is, the overall mobile XR application is implemented by the library function provided by the system.

[0082] When an XR application is started, the XR application notifies the operating system of the XR device which dynamic libraries it uses. The operating system loads these dynamic libraries into memory. Whenever the XR application calls the functional interface provided by the dynamic library, the logic in the dynamic library completes these interactive operations by calling the underlying interface provided by the system.

[0083] It should be noted that, in the above content, the initiation of a move operation on a two-dimensional user interaction interface is implemented at the system layer, and the sliding operation and zooming operation on the two-dimensional user interaction interface are implemented at the application layer, which is only an exemplary description, but not limited to this. For example, the initiation of a move operation, a sliding operation, and a zooming operation on a two-dimensional user interaction interface can be implemented at the system layer at the same time, or the initiation of a move operation, a sliding operation, and a zooming operation on a two-dimensional user interaction interface can be implemented at the application layer at the same time.

[0084] Among them, a map is usually applied to a 3D initial model (e.g., 3D Mesh) to display 3D content (e.g., 3D video) for the following reasons:

[0085] First, in XR devices that can display three-dimensional content, stereoscopic perception is generally provided by allowing the left and right eyes to see different images. The main simulation is the parallax of the human eye, allowing the left and right eyes to see different picture content, so that the brain can feel the depth of the object and obtain a sense of stereoscopic perception. Then, since the two-dimensional user interaction interface is purely flat, the content on the two-dimensional user interaction interface is rendered in the XR device, and the left and right eyes see the same. For three-dimensional models, in order to make the three-dimensional model have a sense of stereoscopic perception, the XR device will provide different pictures to the left and right eyes when rendering. Therefore, by providing different textures (video pictures) for the left and right eyes on the initial three-dimensional model, the user can see three-dimensional content with a sense of stereoscopic perception when watching the target three-dimensional model.

[0086] In the embodiment of the present application, the size, position and other parameters of the three-dimensional display area match the two-dimensional user interaction interface, and can be adjusted in linkage with the two-dimensional user interaction interface. The target three-dimensional model can adapt to the overall movement of the XR display window and be displayed in the correct position. The three-dimensional display area can respond to the layout changes of the two-dimensional user interaction interface. For example, when the interface content of the two-dimensional user interaction interface slides with the user's sliding operation, the target three-dimensional model in the three-dimensional display area also slides to the correct layout position. It supports superimposing a two-dimensional display subview on top of the three-dimensional display area, and maintaining the hierarchical relationship between the target three-dimensional model and the display subview, so that the control or description information presented by the two-dimensional display subview is correctly displayed and not blocked. In addition, there is no need to perceive the thickness information of the three-dimensional content. It is only necessary to specify whether the three-dimensional content is two-dimensional or three-dimensional, and regard the three-dimensional content (or target three-dimensional model) as a component of the two-dimensional user interaction interface. There is no need for developers to manually maintain the relationship between the three-dimensional content and the two-dimensional user interaction interface, nor is there a need to develop commonly used two-dimensional user interface components in the three-dimensional rendering engine. Only through a small amount of development work, three-dimensional content that is well integrated with the two-dimensional user interaction interface in visual perception can be presented to users in XR applications.

[0087] Regarding the embodiments of the present application Figure 1a The detailed implementation and beneficial effects of each step in the method have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0088] The present application also provides a page processing method, which is applied to an XR device, where an XR application is running, such as Figure 1b As shown, the method includes:

[0089] 101b. In response to a first operation initiated by a user on the XR application, display a two-dimensional user interaction interface in the real three-dimensional space, where the two-dimensional user interaction interface includes a three-dimensional display area;

[0090] 102b. Displaying the 3D content picture in the real space area corresponding to the 3D display area, and aligning the projection position of the 3D content picture on the 2D user interaction interface with the 3D display area;

[0091] 103b. In response to a second operation initiated by the user on the two-dimensional user interaction interface, the visualization attributes of the two-dimensional user interaction interface and the three-dimensional content screen are adjusted in a linked manner, and the projection position is kept aligned with the three-dimensional display area.

[0092] In this embodiment, the implementation method of "linked adjustment" can be referred to the above embodiment, which will not be repeated here. Among them, the implementation method of back-end technology implementation in steps 101b to 103b can be referred to the above embodiment, which will not be repeated here.

[0093] In this embodiment, the first operation can be any operation that can display a two-dimensional user interaction interface. The first operation can be triggered by a user, an application event, or a system event, and there is no limitation on this. For an introduction to the first operation, please refer to the aforementioned introduction to the interface display operation, which will not be repeated here. The second operation can be an adjustment operation for the two-dimensional user interaction interface. The second operation can be triggered by a user, an application event, or a system event. For a detailed introduction, please refer to the aforementioned introduction to "Adjustment for the Two-Dimensional User Interaction Interface". Among them, the second operation may act on the two-dimensional user interaction interface, or it may not act on the two-dimensional user interaction interface, for example, it may act on the three-dimensional video in the three-dimensional display area.

[0094] In this embodiment, both the 2D user interaction interface and the 3D content screen can support interaction, that is, the 2D user interaction interface and the 3D content screen can be adjusted. For example, by clicking on the controls on the 2D user interaction interface, you can jump to other pages, such as the home page, the product list page, or the product details page, or by clicking on the controls on the 2D user interaction interface, you can collect or add the product to the shopping cart. For another example, you can adjust the playback progress bar in the 3D content screen, such as pausing playback, dragging the progress bar to fast forward or rewind, and playing at double speed.

[0095] In an optional embodiment, in response to a second operation initiated by a user on a two-dimensional user interaction interface, the visualization attributes of the two-dimensional user interaction interface and the three-dimensional content screen are adjusted in a linked manner, and the implementation method of keeping the projection position aligned with the three-dimensional display area is not limited and is exemplarily described below.

[0096] Example X1, in response to a zoom operation initiated by the user on the two-dimensional user interaction interface, the size of the two-dimensional user interaction interface and the three-dimensional content screen are adjusted in conjunction, and the size of the three-dimensional display area and the size of the real space area carrying the three-dimensional content screen change synchronously to keep the projection position aligned with the three-dimensional display area.

[0097] Example X2, in response to a sliding operation initiated by the user on the two-dimensional user interaction interface, the contents of the two-dimensional user interaction interface and the three-dimensional content screen are adjusted in conjunction, and the size of the three-dimensional display area and the size of the real space area carrying the three-dimensional content screen remain unchanged to keep the projection position aligned with the three-dimensional display area.

[0098] Example X3, in response to the movement operation of the two-dimensional user interaction interface, the posture of the two-dimensional user interaction interface and the three-dimensional content screen are adjusted in conjunction, and the posture of the three-dimensional display area and the posture of the real space area carrying the three-dimensional content screen change synchronously to keep the projection position aligned with the three-dimensional display area.

[0099] One or more of the above examples X1 to X3 may exist at the same time. There is no limit on the number of times the zoom operation, sliding operation or moving operation is performed, and each operation may be performed once or multiple times.

[0100] In an optional embodiment, the two-dimensional user interaction interface also includes at least one two-dimensional display area, and each two-dimensional display area can display product information; the three-dimensional content screen has an association relationship with a certain product information on the two-dimensional interface. For example, the three-dimensional display area can display a relatively independent three-dimensional content screen, or a three-dimensional content screen associated with the product information displayed in the two-dimensional display area, or a three-dimensional model of the product information itself, or a three-dimensional application scene or related introduction video related to the product information. The three-dimensional video corresponding to the product information is played in the three-dimensional content screen, or the three-dimensional model corresponding to the product information is displayed in the three-dimensional display area, and the product information can be displayed in three dimensions. In response to a trigger operation for the product information in any two-dimensional display area, the three-dimensional content screen is replaced, and the replaced three-dimensional content screen is associated with the product information in any two-dimensional display area. For example, the two-dimensional user interaction interface also includes multiple cards, each card corresponds to a two-dimensional display area, and when the user clicks on a product card, the three-dimensional content screen automatically switches to the three-dimensional video corresponding to the product card.

[0101] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101a to 103a can be device A; for another example, the execution subject of steps 101a and 102a can be device A, and the execution subject of step 103a can be device B; and so on.

[0102] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 101a, 102a, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0103] Figure 5 The exemplary embodiment of the present application also provides a schematic diagram of the structure of a page processing device, which corresponds to an extended reality XR device, and an XR application is running on the XR device, such as Figure 5As shown, the device includes: a display module and an interaction module.

[0104] A display module, configured to display a two-dimensional user interaction interface in response to an interface display operation for an XR application, wherein the two-dimensional user interaction interface includes a three-dimensional display area, and the three-dimensional display area is associated with an initial three-dimensional model adapted to its shape;

[0105] A mapping module, used for mapping texture information of three-dimensional content associated with the two-dimensional user interaction interface onto the initial three-dimensional model to obtain a target three-dimensional model;

[0106] The display module is also used to display the target three-dimensional model within the spatial range corresponding to the three-dimensional display area, so as to display the three-dimensional content on the two-dimensional user interaction interface.

[0107] In an optional embodiment, the device also includes: an adjustment module and a linkage module; the adjustment module is used to respond to adjustments to the two-dimensional user interaction interface and adjust the visualization attributes of the two-dimensional user interaction interface, the visualization attributes including at least one of interface size, interface posture, interface brightness and interface content; the linkage module is used to follow the changes in the visualization attributes of the two-dimensional user interaction interface and perform linkage adjustments to the visualization attributes of the target three-dimensional model so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

[0108] Optionally, the adjustment module is specifically used to: respond to a zoom operation on the two-dimensional user interaction interface to adjust the interface size of the two-dimensional user interaction interface; and / or respond to a sliding operation on the two-dimensional user interaction interface to adjust the interface content of the two-dimensional user interaction interface; and / or respond to a move operation on the two-dimensional user interaction interface to adjust the interface posture of the two-dimensional user interaction interface.

[0109] Further optionally, the adjustment module is specifically used to: utilize the zoom view bound to the display window to which the two-dimensional user interaction interface belongs to perceive the zoom operation on the two-dimensional user interaction interface; when the zoom operation is perceived, determine the zoom ratio and zoom direction of the two-dimensional user interaction interface according to the zoom parameters of the zoom operation, and adjust the interface size of the two-dimensional user interaction interface according to the zoom ratio and zoom direction, wherein the three-dimensional display area is scaled following the zoom operation.

[0110] Further optionally, the linkage module is specifically used to: adjust the size of the target three-dimensional model according to the size or scaling ratio of the scaled three-dimensional display area; align the display position of the adjusted target three-dimensional model with the display position of the scaled three-dimensional display area so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

[0111] Further optionally, the adjustment module is specifically used to: utilize the first sliding sub-view in the interface view to which the two-dimensional user interaction interface belongs to perceive the sliding operation on the two-dimensional user interaction interface; when the sliding operation is perceived, determine the target interface content of the corresponding display window in the two-dimensional user interaction interface according to the sliding parameters of the sliding operation and the position of the display window to which the two-dimensional user interaction interface belongs, and display the target interface content in the display window, wherein the three-dimensional display area slides following the sliding operation.

[0112] Further optionally, the linkage module is specifically used to: use the second sliding sub-view to receive the notification message sent by the first sliding sub-view; the second sliding sub-view is a sub-view in the sub-view that carries the three-dimensional display area in the interface view to which the two-dimensional user interaction interface belongs; according to the sliding distance and sliding direction included in the notification message, adjust the display position of the target three-dimensional model; according to the display position of the target three-dimensional model, determine the display content in the display window, so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

[0113] In an optional embodiment, the display module is also used to: display at least one interactive control and description information corresponding to the three-dimensional content on the target three-dimensional model, and the at least one interactive control and description information are carried on the display sub-view; the display sub-view is a second-level sub-view in the first-level sub-view belonging to the three-dimensional display area; respond to a trigger operation on any interactive control, and perform an interactive operation corresponding to any interactive control on the three-dimensional content.

[0114] In an optional embodiment, the extended reality device is an augmented reality device, and the display module is specifically used to: display a two-dimensional user interaction interface in the augmented reality screen in response to an interface display operation for an augmented reality application; and the three-dimensional display area corresponds to a target position in the augmented reality screen.

[0115] The exemplary embodiment of the present application further provides a page processing device, which corresponds to an extended reality XR device, on which an XR application is running, and the device includes:

[0116] A display module, configured to respond to a first operation initiated by a user on the XR application and display a two-dimensional user interaction interface in a real three-dimensional space, where the two-dimensional user interaction interface includes a three-dimensional display area;

[0117] The display module is further used to display the three-dimensional content picture in the real space area corresponding to the three-dimensional display area, and the projection position of the three-dimensional content picture on the two-dimensional user interaction interface is aligned with the three-dimensional display area;

[0118] An adjustment module, configured to respond to a second operation initiated by a user on the two-dimensional user interaction interface and to adjust visualization properties of the two-dimensional user interaction interface and the three-dimensional content screen in a linked manner;

[0119] The holding module is used to keep the projection position aligned with the three-dimensional display area.

[0120] In an optional embodiment, the adjustment module is specifically used to: respond to a zoom operation initiated by a user on the two-dimensional user interaction interface, and jointly adjust the size of the two-dimensional user interaction interface and the three-dimensional content screen; the maintenance module is specifically used to: synchronously change the size of the three-dimensional display area and the size of the real space area carrying the three-dimensional content screen to keep the projection position aligned with the three-dimensional display area; and / or, the adjustment module is specifically used to: respond to a sliding operation initiated by the user on the two-dimensional user interaction interface, and jointly adjust the content of the two-dimensional user interaction interface and the three-dimensional content screen; the maintenance module is specifically used to: keep the size of the three-dimensional display area and the size of the real space area carrying the three-dimensional content screen unchanged to keep the projection position aligned with the three-dimensional display area; and / or, the adjustment module is specifically used to: respond to a move operation on the two-dimensional user interaction interface, and jointly adjust the posture of the two-dimensional user interaction interface and the three-dimensional content screen; the maintenance module is specifically used to: synchronously change the posture of the three-dimensional display area and the posture of the real space area carrying the three-dimensional content screen to keep the projection position aligned with the three-dimensional display area.

[0121] In an optional embodiment, the device also includes: a replacement module; the two-dimensional user interaction interface also includes at least one two-dimensional display area, and product information is displayed in the two-dimensional display area; the replacement module is used to respond to a trigger operation on the product information in any two-dimensional display area, replace the three-dimensional content screen, and the replaced three-dimensional content screen is associated with the product information in any two-dimensional display area.

[0122] The detailed implementation and beneficial effects of each step in the above-mentioned device provided in the embodiments of the present application have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0123] Figure 6 A schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application, wherein the electronic device corresponds to an extended reality XR device, and an XR application is running on the XR device, such as Figure 6 As shown, the device includes: a memory 64 and a processor 65.

[0124] The memory 64 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device.

[0125] The processor 65 is coupled to the memory 64 and is used to execute the computer program in the memory 64, so as to: display a two-dimensional user interaction interface in response to an interface display operation for an XR application, where the two-dimensional user interaction interface includes a three-dimensional display area, and the three-dimensional display area is associated with an initial three-dimensional model adapted to its shape; map texture information of the three-dimensional content associated with the two-dimensional user interaction interface to the initial three-dimensional model to obtain a target three-dimensional model; and display the target three-dimensional model within a spatial range corresponding to the three-dimensional display area to display the three-dimensional content on the two-dimensional user interaction interface.

[0126] In an optional embodiment, the processor 65 is further used to: respond to adjustments to the two-dimensional user interaction interface, adjust the visualization attributes of the two-dimensional user interaction interface, the visualization attributes including at least one of interface size, interface posture, interface brightness and interface content; follow the changes in the visualization attributes of the two-dimensional user interaction interface, and make linkage adjustments to the visualization attributes of the target three-dimensional model so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

[0127] Optionally, when the processor 65 responds to adjustments to the two-dimensional user interaction interface and adjusts the visualization attributes of the two-dimensional user interaction interface, it is specifically used to: respond to zoom operations on the two-dimensional user interaction interface and adjust the interface size of the two-dimensional user interaction interface; and / or respond to sliding operations on the two-dimensional user interaction interface and adjust the interface content of the two-dimensional user interaction interface; and / or respond to move operations on the two-dimensional user interaction interface and adjust the interface posture of the two-dimensional user interaction interface.

[0128] Further optionally, when the processor 65 responds to a zoom operation on the two-dimensional user interaction interface and adjusts the interface size of the two-dimensional user interaction interface, it is specifically used to: perceive the zoom operation on the two-dimensional user interaction interface using the zoom view bound to the display window to which the two-dimensional user interaction interface belongs; when a zoom operation is perceived, determine the zoom ratio and zoom direction of the two-dimensional user interaction interface according to the zoom parameters of the zoom operation, and adjust the interface size of the two-dimensional user interaction interface according to the zoom ratio and the zoom direction, wherein the three-dimensional display area is scaled following the zoom operation.

[0129] Further optionally, when the processor 65 follows the changes in the visualization attributes of the two-dimensional user interaction interface and makes linkage adjustments to the visualization attributes of the target three-dimensional model so that the adjusted target three-dimensional model fits the adjusted three-dimensional display area, it is specifically used to: adjust the size of the target three-dimensional model according to the size or scaling ratio of the scaled three-dimensional display area; align the display position of the adjusted target three-dimensional model with the display position of the scaled three-dimensional display area so that the adjusted target three-dimensional model fits the adjusted three-dimensional display area.

[0130] Further optionally, when the processor 65 responds to a sliding operation on the two-dimensional user interaction interface and adjusts the interface content of the two-dimensional user interaction interface, it is specifically used to: utilize the first sliding sub-view in the interface view to which the two-dimensional user interaction interface belongs to perceive the sliding operation on the two-dimensional user interaction interface; when a sliding operation is perceived, determine the target interface content of the corresponding display window in the two-dimensional user interaction interface according to the sliding parameters of the sliding operation and the position of the display window to which the two-dimensional user interaction interface belongs, and display the target interface content in the display window, wherein the three-dimensional display area slides following the sliding operation.

[0131] Further optionally, when the processor 65 follows the changes in the visualization attributes of the two-dimensional user interaction interface and makes linkage adjustments to the visualization attributes of the target three-dimensional model so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area, it is specifically used to: use the second sliding sub-view to receive the notification message sent by the first sliding sub-view; the second sliding sub-view is a sub-view in the sub-view that carries the three-dimensional display area in the interface view to which the two-dimensional user interaction interface belongs; adjust the display position of the target three-dimensional model according to the sliding distance and sliding direction included in the notification message; determine the display content in the display window according to the display position of the target three-dimensional model, so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

[0132] In an optional embodiment, the processor 65 is further used to: display at least one interactive control and description information corresponding to the three-dimensional content on the target three-dimensional model, wherein the at least one interactive control and description information are carried on a display sub-view; the display sub-view is a second-level sub-view in the first-level sub-view belonging to the three-dimensional display area; and respond to a trigger operation on any interactive control, perform an interactive operation corresponding to any interactive control on the three-dimensional content.

[0133] In an optional embodiment, the extended reality device is an augmented reality device, and when the processor 65 displays the two-dimensional user interaction interface in response to the interface display operation for the XR application, it is specifically used to: display the two-dimensional user interaction interface in the augmented reality screen in response to the interface display operation for the augmented reality application; the three-dimensional display area corresponds to the target position in the augmented reality screen.

[0134] In an optional embodiment, the XR application includes a three-dimensional content player, which includes: an interaction layer, a kernel layer and an operating system layer; the interaction layer is used to display a two-dimensional user interaction interface in response to an interface display operation for the XR application, and the two-dimensional user interaction interface includes a three-dimensional display area; the operating system layer is used to create an initial three-dimensional model associated with the three-dimensional display area and adapted to its shape; the kernel layer is used to map the texture information of the three-dimensional content associated with the two-dimensional user interaction interface to the initial three-dimensional model to obtain a target three-dimensional model; the interaction layer is used to display the target three-dimensional model within the spatial range corresponding to the three-dimensional display area to realize the display of three-dimensional content on the two-dimensional user interaction interface.

[0135] Regarding the embodiments of the present application Figure 6 The detailed implementation and beneficial effects of each step in the illustrated device have been described in detail in the aforementioned embodiments and will not be elaborated on here.

[0136] Further, if Figure 6 As shown, the electronic device also includes: a communication component 66, a display 67, a power component 68, an audio component 69 and other components. Figure 6 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 6 In addition, Figure 6 The components in the dashed box are optional components, not mandatory components, and the specific components depend on the product form of the electronic device. The electronic device of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or a server device such as a conventional server, a cloud server, or a server array. If the electronic device of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it can include Figure 6 If the electronic device of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Figure 6 Components within the dashed box.

[0137] The embodiment of the present application also provides an electronic device, which is applied to an extended reality XR device, and an XR application runs on the XR device; the implementation structure of the electronic device is similar to Figure 6 The implementation structure of the electronic device shown is the same or similar, and can refer to Figure 6 The electronic device provided in this embodiment is implemented as follows: Figure 6The difference between the electronic devices in the illustrated embodiments mainly lies in that the functions implemented by the processor executing the computer program stored in the memory are different. For the electronic device provided in this embodiment, its processor executes the computer program stored in the memory, which can be used to: respond to the first operation initiated by the user for the XR application, display the two-dimensional user interaction interface in the real three-dimensional space, the two-dimensional user interaction interface includes a three-dimensional display area; display the three-dimensional content screen in the real space area corresponding to the three-dimensional display area, and the projection position of the three-dimensional content screen on the two-dimensional user interaction interface is aligned with the three-dimensional display area; respond to the second operation initiated by the user for the two-dimensional user interaction interface, jointly adjust the visualization attributes of the two-dimensional user interaction interface and the three-dimensional content screen, and keep the projection position aligned with the three-dimensional display area.

[0138] In an optional embodiment, when the processor responds to a second operation initiated by the user on the two-dimensional user interaction interface, and jointly adjusts the visualization attributes of the two-dimensional user interaction interface and the three-dimensional content screen, and keeps the projection position aligned with the three-dimensional display area, it is specifically used to: respond to a zoom operation initiated by the user on the two-dimensional user interaction interface, and jointly adjust the size of the two-dimensional user interaction interface and the three-dimensional content screen, and the size of the three-dimensional display area and the size of the real space area carrying the three-dimensional content screen change synchronously to keep the projection position aligned with the three-dimensional display area; and / or respond to a sliding operation initiated by the user on the two-dimensional user interaction interface, and jointly adjust the content of the two-dimensional user interaction interface and the three-dimensional content screen, and the size of the three-dimensional display area and the size of the real space area carrying the three-dimensional content screen remain unchanged to keep the projection position aligned with the three-dimensional display area; and / or respond to a move operation on the two-dimensional user interaction interface, and jointly adjust the posture of the two-dimensional user interaction interface and the three-dimensional content screen, and the posture of the three-dimensional display area and the posture of the real space area carrying the three-dimensional content screen change synchronously to keep the projection position aligned with the three-dimensional display area.

[0139] In an optional embodiment, the two-dimensional user interaction interface also includes at least one two-dimensional display area, in which product information is displayed; the processor is also used to: respond to a trigger operation on the product information in any two-dimensional display area, replace the three-dimensional content screen, and the replaced three-dimensional content screen is associated with the product information in any two-dimensional display area.

[0140] Accordingly, the embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed, the processor can implement the above Figure 1a The steps in the method embodiment shown may be executed by an electronic device.

[0141] Accordingly, the embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, causes the processor to implement the above Figure 1a The steps in the method embodiment shown may be executed by an electronic device.

[0142] The above-mentioned memory can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0143] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0144] The display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0145] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0146] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0147] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program codes.

[0148] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device for implementing the functions specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0149] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0150] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0151] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0152] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0153] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0154] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0155] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A page processing method, It is characterized in that Applied to an extended reality (XR) device, where an XR application is running on the XR device, the method includes: In response to an interface display operation for an XR application, display a two-dimensional user interaction interface, wherein the two-dimensional user interaction interface includes a three-dimensional display area, and the three-dimensional display area is associated with an initial three-dimensional model adapted to its shape; Mapping the texture information of the three-dimensional content associated with the two-dimensional user interaction interface onto the initial three-dimensional model to obtain a target three-dimensional model; wherein the target three-dimensional model carrying different three-dimensional contents is obtained by mapping the texture information of different three-dimensional contents to the initial three-dimensional model; Displaying the target three-dimensional model within the real space corresponding to the three-dimensional display area, so as to realize displaying the three-dimensional content on the two-dimensional user interaction interface; wherein, along with the interactive operation on the two-dimensional user interaction interface, the two-dimensional user interaction interface and the target three-dimensional model are adjusted in linkage, so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area; Wherein, when the interactive operation is implemented as a sliding operation, the steps of linkage adjustment are as follows: Using the first sliding subview in the interface view to which the two-dimensional user interaction interface belongs to sense a sliding operation on the two-dimensional user interaction interface, sending a notification message when the sliding operation is sensed, and the three-dimensional display area slides along with the sliding operation; A notification message sent by the first sliding sub-view is received by using a second sliding sub-view; the second sliding sub-view is a sub-view in the sub-view that carries the three-dimensional display area in the interface view to which the two-dimensional user interaction interface belongs; according to the sliding distance and sliding direction included in the notification message, the target three-dimensional model is adjusted to make the adjusted target three-dimensional model adapt to the adjusted three-dimensional display area.

2. The method according to claim 1, It is characterized in that Also includes: In response to the adjustment of the two-dimensional user interaction interface, adjusting a visualization attribute of the two-dimensional user interaction interface, the visualization attribute comprising at least one of interface size, interface posture, interface brightness, and interface content; Following the change of the visualization attribute of the two-dimensional user interaction interface, the visualization attribute of the target three-dimensional model is linked to be adjusted, so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

3. The method according to claim 2, It is characterized in that In response to the adjustment to the two-dimensional user interaction interface, adjusting the visualization attribute of the two-dimensional user interaction interface includes: In response to a zooming operation on the two-dimensional user interaction interface, adjusting an interface size of the two-dimensional user interaction interface; and / or In response to a sliding operation on the two-dimensional user interaction interface, adjusting interface content of the two-dimensional user interaction interface; and / or In response to a movement operation on the two-dimensional user interaction interface, an interface posture of the two-dimensional user interaction interface is adjusted.

4. The method according to claim 3, It is characterized in that In response to a zoom operation on the two-dimensional user interaction interface, adjusting the interface size of the two-dimensional user interaction interface includes: Using the zoom view bound to the display window to which the two-dimensional user interaction interface belongs to perceive the zoom operation on the two-dimensional user interaction interface; When the zoom operation is sensed, the zoom ratio and zoom direction of the two-dimensional user interaction interface are determined according to the zoom parameters of the zoom operation, and the interface size of the two-dimensional user interaction interface is adjusted according to the zoom ratio and zoom direction, wherein the three-dimensional display area is scaled following the zoom operation.

5. The method according to claim 4, It is characterized in that Following the change of the visualization attribute of the two-dimensional user interaction interface, the visualization attribute of the target three-dimensional model is adjusted in linkage so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area, including: Adjusting the size of the target three-dimensional model according to the size of the scaled three-dimensional display area or the scaling ratio; The display position of the adjusted target three-dimensional model is aligned with the display position of the scaled three-dimensional display area, so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

6. The method according to claim 3, It is characterized in that In response to a sliding operation on the two-dimensional user interaction interface, adjusting the interface content of the two-dimensional user interaction interface includes: When the sliding operation is sensed, the target interface content corresponding to the display window in the two-dimensional user interaction interface is determined according to the sliding parameters of the sliding operation and the position of the display window to which the two-dimensional user interaction interface belongs, and the target interface content is displayed in the display window.

7. The method according to claim 6, It is characterized in that According to the sliding distance and the sliding direction included in the notification message, the target three-dimensional model is adjusted so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area, including: Adjusting the display position of the target three-dimensional model according to the sliding distance and the sliding direction included in the notification message; The display content in the display window is determined according to the display position of the target three-dimensional model, so that the adjusted target three-dimensional model is adapted to the adjusted three-dimensional display area.

8. The method according to claim 3, It is characterized in that Also includes: Displaying at least one interactive control and description information corresponding to the three-dimensional content on the target three-dimensional model, wherein the at least one interactive control and description information are carried on a display sub-view; The display sub-view is a second-level sub-view in the first-level sub-view to which the three-dimensional display area belongs; In response to a triggering operation on any interactive control, an interactive operation corresponding to the any interactive control is performed on the three-dimensional content.

9. The method according to claim 1, It is characterized in that The extended reality XR device is an augmented reality device, which displays a two-dimensional user interaction interface in response to an interface display operation for an XR application, including: In response to an interface display operation for an augmented reality application, displaying the two-dimensional user interaction interface in an augmented reality screen; The three-dimensional display area corresponds to the target position in the augmented reality picture.

10. The method according to any one of claims 1 to 9, It is characterized in that The XR application includes a three-dimensional content player, and the three-dimensional content player includes: an interaction layer, a kernel layer, and an operating system layer; An interaction layer, for displaying a two-dimensional user interaction interface in response to an interface display operation for an XR application, wherein the two-dimensional user interaction interface includes a three-dimensional display area; The operating system layer is used to create an initial three-dimensional model associated with the three-dimensional display area and adapted to its shape; A kernel layer, used for mapping texture information of the three-dimensional content associated with the two-dimensional user interaction interface onto the initial three-dimensional model to obtain a target three-dimensional model; The interactive layer is used to display the target three-dimensional model within the spatial range corresponding to the three-dimensional display area, so as to realize the display of the three-dimensional content on the two-dimensional user interaction interface.

11. A page processing method, It is characterized in that Applied to an extended reality (XR) device, where an XR application is running on the XR device, the method includes: In response to a first operation initiated by a user for an XR application, a two-dimensional user interaction interface is displayed in a real three-dimensional space, wherein the two-dimensional user interaction interface includes a three-dimensional display area, and the three-dimensional display area is associated with an initial three-dimensional model adapted to its shape, and a target three-dimensional model carrying different three-dimensional contents is obtained by mapping texture information of different three-dimensional contents to the initial three-dimensional model; Displaying a 3D content picture in a real space area corresponding to the 3D display area, wherein a projection position of the 3D content picture on the 2D user interaction interface is aligned with the 3D display area, and the 3D content picture is displayed through the target 3D model; In response to a second operation initiated by a user on the two-dimensional user interaction interface, the visualization attributes of the two-dimensional user interaction interface and the three-dimensional content screen are adjusted in a linked manner, and the projection position is kept aligned with the three-dimensional display area; Wherein, when the second operation is implemented as a sliding operation, the steps of linkage adjustment are as follows: Using the first sliding subview in the interface view to which the two-dimensional user interaction interface belongs to sense a sliding operation on the two-dimensional user interaction interface, sending a notification message when the sliding operation is sensed, and the three-dimensional display area slides along with the sliding operation; A notification message sent by the first sliding sub-view is received by using a second sliding sub-view; the second sliding sub-view is a sub-view in the sub-view that carries the three-dimensional display area in the interface view to which the two-dimensional user interaction interface belongs; and the three-dimensional content screen is adjusted according to the sliding distance and sliding direction included in the notification message so that the adjusted three-dimensional content screen is adapted to the adjusted three-dimensional display area.

12. The method according to claim 11, It is characterized in that In response to a second operation initiated by a user on the two-dimensional user interaction interface, the visualization attributes of the two-dimensional user interaction interface and the three-dimensional content screen are adjusted in a linked manner, and the projection position is kept aligned with the three-dimensional display area, including: In response to a zoom operation initiated by a user on the two-dimensional user interaction interface, the size of the two-dimensional user interaction interface and the three-dimensional content screen are adjusted in a linked manner, and the size of the three-dimensional display area and the size of the real space area carrying the three-dimensional content screen are changed synchronously, so as to keep the projection position aligned with the three-dimensional display area; and / or In response to a sliding operation initiated by a user on the two-dimensional user interaction interface, the two-dimensional user interaction interface and the content of the three-dimensional content screen are adjusted in a linked manner, and the size of the three-dimensional display area and the size of the real space area carrying the three-dimensional content screen remain unchanged, so as to keep the projection position aligned with the three-dimensional display area; and / or In response to a movement operation on the two-dimensional user interaction interface, the postures of the two-dimensional user interaction interface and the three-dimensional content screen are adjusted in conjunction, and the posture of the three-dimensional display area and the posture of the real space area carrying the three-dimensional content screen change synchronously to keep the projection position aligned with the three-dimensional display area.

13. The method according to claim 11 or 12, It is characterized in that The two-dimensional user interaction interface also includes at least one two-dimensional display area, and the two-dimensional display area displays product information; The method further includes: in response to a triggering operation on the commodity information in any two-dimensional display area, replacing the three-dimensional content picture, and associating the replaced three-dimensional content picture with the commodity information in any two-dimensional display area.

14. An electronic device, It is characterized in that include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method of any one of claims 1-10 and claims 11-13.

15. A computer-readable storage medium storing a computer program, It is characterized in that When the computer program is executed by a processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 10 and claims 11 to 13.

16. A computer program product comprising a computer program / instructions, It is characterized in that When the computer program / instructions are executed by a processor, the processor is caused to implement the steps in the method of any one of claims 1-10 and claims 11-13.