Image rendering methods, apparatus and storage media

Abstract

This disclosure relates to an image rendering method, apparatus, and storage medium. The image rendering method includes: acquiring image parameter information, and rendering a display interface for displaying an image to be rendered based on the image parameter information, wherein the display interface reserves a reserved area for displaying the image to be rendered by an extended reality device; sending interface information characterizing the display interface to the extended reality device, so that the extended reality device renders the image to be rendered in the reserved area. This disclosure saves data transmission bandwidth between the extended reality device and the terminal, reduces power consumption of both the extended reality device and the terminal device, reduces preview latency, and improves the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of extended reality technology, and in particular to image rendering methods, apparatus and storage media. Background Technology

[0002] Extended Reality (XR) refers to the use of computers to combine the real and virtual worlds, creating a virtual environment for human-computer interaction. XR can be understood as a collective term for various technologies such as Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR). By integrating the visual interaction technologies of these three technologies, it provides users with a seamless "immersive" experience between the virtual and real worlds.

[0003] In related technologies, extended reality devices capture images, interact with terminals, and ultimately display the images. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an image rendering method, apparatus and storage medium.

[0005] According to a first aspect of the present disclosure, an image rendering method is provided, the method comprising:

[0006] Image parameter information is acquired, and a display interface for displaying the image to be rendered is rendered based on the image parameter information. The display interface has a reserved area for displaying the image to be rendered by the extended reality device. Interface information representing the display interface is sent to the extended reality device so that the extended reality device renders the image to be rendered in the reserved area.

[0007] In one embodiment, the interface information includes the window position and window size information of the image to be rendered; sending the interface information representing the display interface to the extended reality device includes: encoding the window position and window size information of the image to be rendered and then sending it to the extended reality device.

[0008] In one embodiment, rendering a display interface for displaying the image based on the image parameter information includes: determining the scene corresponding to the image parameter information; determining a display interface matching the scene based on the correspondence between the scene and the display interface; and rendering the display interface matching the scene.

[0009] According to a second aspect of the present disclosure, an image rendering method is provided, the method comprising:

[0010] Obtain interface information representing the display interface, wherein a reserved area is reserved for displaying an image to be rendered by an extended reality device, and is rendered by a device other than the extended reality device; render the image to be rendered in the reserved area; display the rendered image.

[0011] In one embodiment, the interface information includes the window position and window size information of the image to be rendered; obtaining the interface information representing the display interface includes: decoding the interface information to obtain the window position and window size information of the image to be rendered; rendering the image to be rendered in the reserved area includes: obtaining the reserved area according to the window position and window size information of the image to be rendered; and rendering the image to be rendered in the reserved area.

[0012] According to a third aspect of the present disclosure, an image rendering apparatus is provided, the apparatus comprising:

[0013] The input unit is used to acquire image parameter information;

[0014] The processing unit is used to render a display interface for displaying the image to be rendered based on the image parameter information, wherein the display interface has a reserved area for displaying the image to be rendered on the extended reality device.

[0015] The sending unit is used to send interface information representing the display interface to the extended reality device, so that the extended reality device renders the image to be rendered in the reserved area.

[0016] In one embodiment, the interface information includes the window position and window size information of the image to be rendered; the sending unit sends the interface information representing the display interface to the extended reality device in the following manner: the window position and window size information of the image to be rendered are encoded and then sent to the extended reality device.

[0017] In one embodiment, the processing unit renders a display interface for displaying the image based on the image parameter information in the following manner: determining the scene corresponding to the image parameter information; determining a display interface matching the scene based on the correspondence between the scene and the display interface; and rendering the display interface matching the scene.

[0018] According to a fourth aspect of the present disclosure, an image rendering apparatus is provided, the apparatus comprising:

[0019] The receiving unit is used to acquire interface information representing the display interface, wherein the display interface has a reserved area for displaying an image to be rendered by the extended reality device, and is rendered by a device other than the extended reality device.

[0020] The processing unit is used to render the image to be rendered in the reserved area;

[0021] The display unit is used to display the rendered image.

[0022] In one embodiment, the interface information includes the window position and window size information of the image to be rendered; the receiving unit obtains the interface information representing the display interface in the following manner: decoding the interface information to obtain the window position and window size information of the image to be rendered; the processing unit renders the image to be rendered in the reserved area in the following manner: obtaining the reserved area according to the window position and window size information of the image to be rendered; rendering the image to be rendered in the reserved area.

[0023] According to a fifth aspect of the present disclosure, an image rendering apparatus is provided, comprising:

[0024] processor;

[0025] Memory used to store processor-executable instructions;

[0026] The processor is configured to execute the image rendering method described in the first aspect or any embodiment of the first aspect.

[0027] According to a sixth aspect of the present disclosure, an image rendering apparatus is provided, comprising:

[0028] processor;

[0029] Memory used to store processor-executable instructions;

[0030] The processor is configured to execute the image rendering method described in the second aspect or any embodiment of the second aspect.

[0031] According to a seventh aspect of this disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor of a terminal, enable the terminal to execute the image rendering method described in the first aspect or any embodiment of the first aspect.

[0032] According to an eighth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor of an extended reality device, enable the extended reality device to perform the image rendering method described in the second aspect or any embodiment of the second aspect.

[0033] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: A display interface for displaying an image to be rendered is rendered based on image parameter information, wherein a reserved area is reserved for displaying the image to be rendered by the extended reality device. Interface information characterizing the display interface is sent to the extended reality device, so that the extended reality device renders the image to be rendered in the reserved area. Through this disclosure, compared to a scheme that renders both the display interface and the image to be displayed, data transmission bandwidth between the extended reality device and the terminal can be saved, power consumption of both the extended reality device and the terminal device can be reduced, and latency in previewing the image can be reduced, thus improving the user experience.

[0034] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0036] Figure 1 This is a flowchart illustrating an image rendering method according to an exemplary embodiment.

[0037] Figure 2 This is a flowchart illustrating a method for sending interface information representing a display interface to an extended reality device according to an exemplary embodiment.

[0038] Figure 3 This is a flowchart illustrating a method for rendering a display interface for displaying an image based on image parameter information, according to an exemplary embodiment.

[0039] Figure 4 This is a flowchart illustrating an image rendering method according to an exemplary embodiment.

[0040] Figure 5 This is a flowchart illustrating a method for rendering an image to be rendered in a reserved area according to an exemplary embodiment.

[0041] Figure 6 This is a block diagram illustrating an image rendering apparatus for an XR glasses device according to an exemplary embodiment.

[0042] Figure 7 This is a block diagram illustrating an image rendering apparatus according to an exemplary embodiment.

[0043] Figure 8 This is a block diagram illustrating an image rendering apparatus according to an exemplary embodiment.

[0044] Figure 9This is a block diagram illustrating an apparatus for image rendering according to an exemplary embodiment. Detailed Implementation

[0045] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0046] The method provided in this disclosure can be applied to image rendering scenarios based on extended reality devices for image display.

[0047] One common image rendering method for extended reality (AR) devices is to perform rendering entirely on the AR device itself. The AR device bears a heavy workload of computation and processing. For typical headband or headset models, this method results in bulky AR devices with a poor user experience. Another method involves the AR device performing only lightweight computational tasks, while the terminal handles image rendering. In this approach, the terminal encodes and compresses the rendered image before sending the encoded and compressed data to the AR device. The AR device then decodes and displays the image. This method allows for a more lightweight product but consumes significant data transmission bandwidth between devices and introduces latency. The process of the human eye seeing a real-world image is highly sensitive to latency; large delays can cause discomfort, even dizziness or nausea, resulting in a poor user experience.

[0048] In view of this, this disclosure provides an image rendering method in which the terminal renders the display interface and reserves a window area (hereinafter referred to as the reserved area) for the image to be rendered and displayed by the extended reality device. The extended reality device renders the image to be displayed within the reserved area, realizing split rendering of image rendering. This can save bandwidth, reduce latency, reduce power consumption, and improve user experience.

[0049] Figure 1 This is a flowchart illustrating an image rendering method according to an exemplary embodiment, the image rendering method being applied to a terminal. For example, the terminal may be a terminal that interacts with an extended reality device. Figure 1 As shown, the image rendering method includes the following steps:

[0050] In step S11, image parameter information is obtained.

[0051] In this embodiment of the disclosure, the camera in the extended reality device captures images and sends the captured image parameter information to the terminal. The terminal obtains the image parameter information sent by the extended reality device.

[0052] In step S12, a display interface for displaying the image to be rendered is rendered based on the image parameter information. The display interface has a reserved area for displaying the image to be rendered on the extended reality device.

[0053] In this embodiment of the present disclosure, when the terminal renders based on image parameter information, it reserves the area to be rendered for displaying the extended reality device image and does not render the reserved area, which can improve the terminal rendering speed, thereby reducing image rendering latency and processing power consumption.

[0054] In this context, the image to be rendered by the extended reality device can be understood as a preview image. In the image rendering method provided in this embodiment, the terminal uses a GPU (graphics processing unit) to render the UI (user interface) of the extended reality device's camera based on some image parameter information of the image to be rendered (which may be information about the image captured by the extended reality device, or input information). There is no preview image. The reserved area can be understood as the area in the UI interface used to display the preview image.

[0055] In this embodiment of the disclosure, the terminal renders a display interface other than the reserved area used to display the image to be rendered.

[0056] In step S13, interface information representing the display interface is sent to the extended reality device so that the extended reality device can render the image to be rendered in the reserved area.

[0057] In this embodiment, there is no need to compress and transmit the image from the extended reality device to the terminal; only the area of ​​the camera image to be displayed needs to be reserved, which reduces the terminal's rendering power consumption. Furthermore, the terminal sends interface information representing the display interface to the extended reality device, which saves bandwidth compared to sending the entire rendered image to the extended reality device.

[0058] The following describes the image rendering methods involved in the above embodiments.

[0059] In one embodiment of this disclosure, the interface information sent by the terminal to the extended reality device needs to include information about the extended reality device rendering in a reserved area. For example, the reserved area is a window. The interface information includes window information, such as the window size information of the image to be rendered (also known as window size parameter information), window position information (also known as the coordinate information of the window in the display interface), etc.

[0060] In this embodiment, the terminal sends the window position and size information of the image to be rendered to the extended reality device, eliminating the need to send the rendered image. This saves the terminal from the processing steps of generating the preview image. For example, the image captured by the extended reality device does not need to go through a series of operations such as compression, transmission, reception, decompression, rendering by the terminal, and then compression, transmission, and decompression before being displayed on the glasses. This series of complex processes can save power consumption and reduce latency.

[0061] In this embodiment of the disclosure, when the terminal sends interface information to the extended reality device, in order to further reduce transmission latency and transmission bandwidth, the interface information to be sent can be encoded and compressed before being sent to the extended reality device.

[0062] Figure 2 This is a flowchart illustrating, according to an exemplary embodiment, a method for sending interface information representing a display interface to an extended reality device, such as... Figure 2 As shown, it includes the following steps:

[0063] In step S21, the window position and window size information of the image to be rendered are encoded.

[0064] In step S22, the encoded window position and window size information are sent to the extended reality device.

[0065] In this embodiment, the terminal encodes interface information such as window position and size information of the image to be rendered, and sends it to the extended reality device (ARD) via a communication connection between the terminal and the ARD. For example, the encoded interface information can be sent to the ARD via USB (Universal Serial Bus) or Wi-Fi (mobile hotspot). The ARD decodes the encoded information and then renders and displays it based on the window position and size information. Specifically, the terminal sends UI interface information including reserved area, window position, and window size information to the ARD. Compared to the terminal sending the rendered image to the ARD directly, this method saves power and improves latency. Experiments show that this method can save 30%-40% of power compared to the terminal sending the rendered image directly to the ARD.

[0066] In this embodiment, the terminal renders different display interfaces based on different scenarios, and different display interfaces correspond to different interface information. The correspondence between scenarios and display interfaces is predefined. When rendering the UI interface, the terminal can determine the display interface of the current scenario to be rendered based on the predefined correspondence between scenarios and UI interfaces.

[0067] Figure 3This is a flowchart illustrating a display interface for displaying images based on image parameter information, according to an exemplary embodiment. Figure 3 As shown, it includes the following steps:

[0068] In step S31, the scene corresponding to the image parameter information is determined.

[0069] In this embodiment of the disclosure, when the terminal renders the display interface for displaying the image based on image parameter information, it can perform auxiliary scene analysis on the image according to the information collected by the camera of the extended reality device. Different scenes correspond to different reserved area information. For example, the reserved area in a guiding scene is relatively smaller than the reserved area in a non-guiding scene.

[0070] In step S32, the display interface matching the scene is determined based on the correspondence between the scene and the display interface.

[0071] In this embodiment of the disclosure, the terminal pre-stores predefined information corresponding to various scenarios. The predefined information corresponding to different scenarios is different, and the display interface will also be different.

[0072] In step S33, the display interface of the matching scene is rendered.

[0073] In this embodiment, the display interface rendering method based on the correspondence between the scene and the display interface allows for the reuse of the display interface when the scene remains unchanged, eliminating the need for rendering the display interface each time. For example, for updating and sending interface information, the extended reality device can first capture an image of a certain scene, render the display interface on the terminal, and send the rendered display interface to the extended reality device. When the extended reality device captures an image of the same scene a second time, if there is no update to the interface information, the terminal does not need to render again or send new interface information to the extended reality device. The extended reality device renders the second captured image within the reserved area based on the interface information received the first time, such as a reserved area. If the scene of the third captured image by the extended reality device changes, the interface information also changes. For example, if the user interacts with an icon, the terminal will re-render the display interface based on the received changed image parameter information and send the re-rendered interface information to the extended reality device, which then renders the image based on the new interface information.

[0074] The extended reality image rendering method for terminals provided in this disclosure can save data transmission bandwidth between extended reality devices and terminals, reduce power consumption for image rendering between extended reality devices and terminals, and reduce latency for displaying preview images.

[0075] Based on the same concept, embodiments of this disclosure also provide a method for image rendering.

[0076] Figure 4 This is a flowchart illustrating an image rendering method according to an exemplary embodiment. This image rendering method is applied to an extended reality device. Figure 4 As shown, it includes the following steps:

[0077] In step S41, interface information representing the display interface is obtained.

[0078] In this embodiment of the disclosure, a reserved area is reserved in the display interface for displaying the image to be rendered by the extended reality device, and the image is rendered by a device other than the extended reality device.

[0079] In one embodiment of this disclosure, the interface information sent by the terminal to the extended reality device needs to include information about the extended reality device rendering in the reserved area.

[0080] In step S42, the image to be rendered is rendered in the reserved area.

[0081] In this embodiment of the present disclosure, the extended reality device can render and display the preview image received from the vision-related camera sensor in the corresponding reserved area based on the received interface information.

[0082] In step S43, the rendered image is displayed.

[0083] The image rendering method provided in this disclosure allows the extended reality device to render the image to be rendered in a reserved area without rendering other display interfaces besides the image to be rendered, thereby reducing the power consumption and latency of the extended reality device in rendering the image.

[0084] In one embodiment of this disclosure, the reserved area is a window. Interface information includes window information, such as the window size and position information of the image to be rendered.

[0085] The window size and position information received by the extended reality device can be encoded and compressed by the terminal. The extended reality device decodes the received encoded and compressed information to obtain the position and size of the reserved area for rendering the image to be rendered. Based on this position and size, the device renders the image to be rendered in the reserved area and adds the image to be rendered to the reserved area to obtain the rendered image.

[0086] Figure 5 This is a flowchart illustrating a method for rendering an image to be rendered in a reserved area according to an exemplary embodiment, such as... Figure 5As shown, it includes the following steps:

[0087] In step S51, the interface information is decoded to obtain information including the window position and window size of the image to be rendered.

[0088] In this embodiment of the disclosure, the interface information may include the window position and window size information of the image to be rendered, and may also include the UI interface.

[0089] In step S52, the reserved area is obtained according to the window position and window size information of the image to be rendered.

[0090] In this embodiment of the disclosure, the reserved area may include the content of the UI interface, such as the icons of the UI interface.

[0091] In step S53, the image to be rendered is rendered in the reserved area.

[0092] In this embodiment of the disclosure, the display interface that can be obtained after the image to be rendered in the reserved area may include a preview image or UI interface information.

[0093] The image rendering method provided in this disclosure involves a terminal reserving an area for the image to be displayed during image rendering. The terminal sends the window coordinates, size parameters, and compressed data of the reserved area to an extended reality device. Upon receiving the compressed data, the extended reality device decompresses it, renders the image based on the window information transmitted from the terminal, composites it into the reserved area, and then transmits the rendered image to the display unit for display. This method saves bandwidth, reduces latency, lowers the power consumption of the extended reality device, and improves the user experience.

[0094] In this embodiment of the disclosure, the extended reality device displays the rendered image, which may be a display buffer with a preview image. For example, it may include a UI interface and an image placed in a reserved area, and the display buffer with the preview image is sent to the display module for display.

[0095] In this embodiment of the disclosure, extended reality means that the image information captured by the vision-related camera in the device remains unchanged. For example, the size of the captured image remains unchanged, and the terminal renders the content to be displayed according to the requirements of different scenarios after receiving the image parameter information.

[0096] The image rendering method described above is illustrated below with reference to examples.

[0097] The extended reality device involved in the embodiments of this disclosure may be, for example, an XR device such as XR glasses. The terminal may be a smart terminal such as a smartphone.

[0098] The following explanation uses XR glasses as the augmented reality device and a smartphone as the terminal.

[0099] In summary, XR glasses may include multiple cameras, including a vision-related camera for vision-related operations, such as an RGB camera for image acquisition.

[0100] In this embodiment of the disclosure, different cameras correspond to different functions. Information related to vision collected by cameras other than the vision camera is also sent to the terminal. For example, information collected by cameras other than the vision camera to assist in scene analysis is also sent to the terminal.

[0101] In this embodiment of the disclosure, the XR glasses may include a WIFI / USB module for interacting with a smartphone and transmitting information with the smartphone.

[0102] In this embodiment of the disclosure, the XR glasses may include a decoder for decoding information received from the WIFI / USB module and transmitting the decoded information to the GPU.

[0103] In this embodiment, the XR glasses may include a GPU for receiving decoding information transmitted by a decoder and a preview image transmitted by a camera. Based on the decoding information, the GPU renders the preview image in a corresponding reserved area and sends the rendered interface with the preview image to a display unit. The display unit is located in the XR glasses and is used for displaying the image.

[0104] In this embodiment of the disclosure, the XR glasses may include a display unit for receiving the rendered interface and displaying the interface.

[0105] In this embodiment of the disclosure, the smartphone may include an input unit for inputting image information and transmitting the image information to the GPU.

[0106] In this embodiment of the disclosure, the smartphone may include a GPU, which renders the image based on the received image information to obtain a rendered UI image and window position coordinates and size information, and sends the rendered UI image and window position coordinates and size information to an encoder.

[0107] In this embodiment of the disclosure, the smartphone may include an encoder for encoding and transmitting the rendered UI image, window position coordinates and size information to the WIFI / USB module.

[0108] In this embodiment of the disclosure, the smartphone may include a Wi-Fi / USB module for interacting with XR glasses and transmitting information with them. This Wi-Fi / USB module is different from a standard Wi-Fi / USB module; it is located within the smartphone.

[0109] Figure 6 This is a block diagram illustrating an image rendering apparatus for an XR glasses device according to an exemplary embodiment. (Refer to...) Figure 6 The device includes XR glasses and a smartphone. The XR glasses include a WIFI / USB module, a decoder, a GPU, a camera sensor, and a display unit. The smartphone includes an input unit, a GPU, an encoder, and a WIFI / USB module.

[0110] 1. The smartphone's input unit inputs image parameter information to the GPU. The GPU renders the UI interface based on the image parameter information. That is, the UI interface and the window position coordinates and size information of the reserved area are obtained through the smartphone's GPU processing.

[0111] 2. After encoding the UI interface and the window position coordinates and size information of the reserved area obtained by the GPU, send them to the XR glasses via WIFI or USB module.

[0112] 3. After receiving the encoded information, the WIFI or USB module of the XR glasses will send the encoded information to the decoder for decoding.

[0113] 4. The decoder sends the decoded UI interface, as well as the window position coordinates and size information of the reserved area, to the GPU. The preview image captured by the XR glasses' camera is also sent to the GPU.

[0114] 5. The GPU renders a display buffer with a preview image based on the window's position coordinates and size information, and sends it to the display module for display.

[0115] 6. The display unit displays the rendered interface.

[0116] This device is a split-type device, and the two parts of the split-type device need to interact. The device that captures and displays images can be XR glasses, and the other part of the split-type device can be a smartphone. Unlike head-mounted devices, split-type devices are lighter and provide a better user experience.

[0117] Based on the same concept, embodiments of this disclosure also provide an image rendering apparatus.

[0118] It is understood that the image rendering apparatus provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0119] Figure 7 This is a block diagram illustrating an image rendering apparatus according to an exemplary embodiment. (Refer to...) Figure 7 The device 100 can be a terminal. The device 100 includes an input unit 101, a processing unit 102, and a sending unit 103.

[0120] Input unit 101 is used to acquire image parameter information.

[0121] The processing unit 102 is used to render a display interface for displaying an image to be rendered based on image parameter information. The display interface has a reserved area for displaying the image to be rendered on the extended reality device.

[0122] The sending unit 103 is used to send interface information representing the display interface to the extended reality device so that the extended reality device can render the image to be rendered in the reserved area.

[0123] In one embodiment, the interface information includes the window position and window size information of the image to be rendered.

[0124] The sending unit 103 sends the interface information representing the display interface to the extended reality device in the following manner: it encodes the window position and window size information of the image to be rendered and then sends it to the extended reality device.

[0125] In one embodiment, the processing unit 102 renders a display interface for displaying an image based on image parameter information as follows: Determine the scene corresponding to the image parameter information; determine a display interface matching the scene based on the correspondence between the scene and the display interface; and render the display interface matching the scene.

[0126] Figure 8 This is a block diagram illustrating an image rendering apparatus according to an exemplary embodiment. (Refer to...) Figure 8 The device 100 can be an extended reality device. The device 200 includes a receiving unit 201, a processing unit 202, and a display unit 203.

[0127] The receiving unit 201 is used to acquire interface information representing the display interface, in which a reserved area is reserved for displaying an image to be rendered by the extended reality device, and is rendered by another device different from the extended reality device.

[0128] The processing unit 202 is used to render the image to be rendered in the reserved area.

[0129] Display unit 203 is used to display the rendered image.

[0130] In one embodiment, the interface information includes the window position and window size information of the image to be rendered.

[0131] The receiving unit 201 acquires interface information representing the display interface by decoding the interface information to obtain information including the window position and window size of the image to be rendered. The processing unit 202 renders the image to be rendered in the reserved area by obtaining the reserved area according to the window position and window size information of the image to be rendered, and then rendering the image to be rendered in the reserved area.

[0132] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0133] Figure 9 This is a block diagram illustrating an apparatus 300 for image rendering according to an exemplary embodiment. For example, apparatus 300 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0134] Reference Figure 9 The device 300 may include one or more of the following components: processing component 302, memory 304, power component 306, multimedia component 308, audio component 310, input / output (I / O) interface 312, sensor component 314, and communication component 316.

[0135] Processing component 302 typically controls the overall operation of device 300, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 302 may include one or more processors 320 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 302 may include one or more modules to facilitate interaction between processing component 302 and other components. For example, processing component 302 may include a multimedia module to facilitate interaction between multimedia component 308 and processing component 302.

[0136] Memory 304 is configured to store various types of data to support the operation of device 300. Examples of such data include instructions for any application or method operating on device 300, contact data, phonebook data, messages, pictures, videos, etc. Memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0137] The power supply component 306 provides power to the various components of the device 300. The power supply component 306 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 300.

[0138] Multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 308 includes a front-facing camera and / or a rear-facing camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0139] Audio component 310 is configured to output and / or input audio signals. For example, audio component 310 includes a microphone (MIC) configured to receive external audio signals when device 300 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 304 or transmitted via communication component 316. In some embodiments, audio component 310 also includes a speaker for outputting audio signals.

[0140] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0141] Sensor assembly 314 includes one or more sensors for providing status assessments of various aspects of device 300. For example, sensor assembly 314 may detect the on / off state of device 300, the relative positioning of components such as the display and keypad of device 300, changes in the position of device 300 or a component of device 300, the presence or absence of user contact with device 300, the orientation or acceleration / deceleration of device 300, and temperature changes of device 300. Sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 314 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0142] Communication component 316 is configured to facilitate wired or wireless communication between device 300 and other devices. Device 300 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 316 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0143] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0144] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, which can be executed by a processor 320 of the device 300 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0145] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0146] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0147] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0148] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0149] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0150] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image rendering method, characterized in that, The method includes: Image parameter information is obtained, and a display interface for displaying the image to be rendered is rendered based on the image parameter information. The display interface has a reserved area for displaying the image to be rendered on the extended reality device. The interface information representing the display interface is sent to the extended reality device so that the extended reality device renders the image to be rendered in the reserved area.

2. The method according to claim 1, characterized in that, The interface information includes the window position and window size information of the image to be rendered; Sending interface information representing the display interface to the extended reality device includes: The window position and window size information of the image to be rendered are encoded and sent to the extended reality device.

3. The method according to claim 2, wherein rendering the display interface for displaying the image based on the image parameter information includes: Determine the scene corresponding to the image parameter information; Based on the correspondence between scenes and display interfaces, determine the display interface that matches the scene; Render a display interface that matches the scene.

4. An image rendering method, characterized in that, The method includes: Obtain interface information representing the display interface, wherein the display interface has a reserved area for displaying the image to be rendered by the extended reality device, and is rendered by another device different from the extended reality device; The image to be rendered is rendered in the reserved area; Display the rendered image.

5. The method according to claim 4, wherein the interface information includes the window position and window size information of the image to be rendered; The acquisition of interface information representing the display interface includes: Decode the interface information to obtain information including the window position and window size of the image to be rendered; The step of rendering the image to be rendered in the reserved area includes: The reserved area is obtained according to the window position and window size information of the image to be rendered; The image to be rendered is rendered in the reserved area.

6. An image rendering apparatus, characterized in that, include: The input unit is used to acquire image parameter information; The processing unit is used to render a display interface for displaying the image to be rendered based on the image parameter information, wherein the display interface has a reserved area for displaying the image to be rendered on the extended reality device. The sending unit is used to send interface information representing the display interface to the extended reality device, so that the extended reality device renders the image to be rendered in the reserved area.

7. The image rendering apparatus according to claim 6, characterized in that, The interface information includes the window position and window size information of the image to be rendered; The sending unit sends the interface information representing the display interface to the extended reality device in the following manner: The window position and window size information of the image to be rendered are encoded and sent to the extended reality device.

8. The apparatus according to claim 6, characterized in that, The processing unit renders a display interface for displaying the image based on the image parameter information in the following manner: Determine the scene corresponding to the image parameter information; Based on the correspondence between scenes and display interfaces, determine the display interface that matches the scene; Render a display interface that matches the scene.

9. An image rendering apparatus, characterized in that, include: The receiving unit is used to acquire interface information representing the display interface, wherein the display interface has a reserved area for displaying an image to be rendered by the extended reality device, and is rendered by a device other than the extended reality device. The processing unit is used to render the image to be rendered in the reserved area; The display unit is used to display the rendered image.

10. The apparatus according to claim 9, characterized in that, The interface information includes the window position and window size information of the image to be rendered; The receiving unit acquires the interface information representing the display interface in the following manner: Decode the interface information to obtain information including the window position and window size of the image to be rendered; The processing unit renders the image to be rendered in the reserved area in the following manner: The reserved area is obtained according to the window position and window size information of the image to be rendered; The image to be rendered is rendered in the reserved area.

11. An image rendering apparatus, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to: execute the image rendering method according to any one of claims 1 to 3, or execute the image rendering method according to any one of claims 4 to 5.

12. A storage medium, characterized in that, The storage medium stores instructions that, when executed by the processor of the terminal, enable the terminal to perform the image rendering method according to any one of claims 1 to 3; or, when executed by the processor of the extended display device, enable the extended reality device to perform the image rendering method according to any one of claims 4 to 5.

Images