Image rendering method and device and storage medium

Abstract

The invention relates to an image rendering method and device and a storage medium. The image rendering method comprises the steps that image parameter information is acquired, a display interface used for displaying a to-be-rendered image is rendered based on the image parameter information, and a reserved area used for displaying the to-be-rendered image of the augmented reality device is reserved in the display interface; and interface information representing the display interface is sent to an augmented reality device, so that the augmented reality device renders the to-be-rendered image in the reserved area. According to the method and the device, the data transmission bandwidth between the augmented reality device and the terminal is saved, the power consumption of the augmented reality device and the terminal device is reduced, the time delay of previewing the image is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of extended reality technology, and in particular to an image rendering method, device and storage medium. Background Art

[0002] Extended Reality (XR) refers to the use of computers to combine the real and the virtual to create a virtual environment for human-computer interaction. Among them, XR can be understood as a general term for multiple technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR). By integrating the visual interaction technologies of the three, it brings the experiencer an "immersive feeling" of seamless transition between the virtual world and the real world.

[0003] In the related technology, the extended reality device collects images, interacts with the terminal, and finally displays the images. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an image rendering method, an apparatus and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image rendering method, the method comprising:

[0006] Obtain image parameter information, and render a display interface for displaying an image to be rendered based on the image parameter information, wherein a reserved area for displaying the image to be rendered by an extended reality device is reserved in the display interface; and send interface information representing the display interface to the extended reality device, so that the extended reality device renders the image to be rendered in the reserved area.

[0007] In one embodiment, the interface information includes the window position and window size information of the image to be rendered; sending the interface information representing the display interface to the extended reality device includes: encoding the window position and window size information of the image to be rendered and sending them to the extended reality device.

[0008] In one embodiment, rendering a display interface for displaying the image based on the image parameter information includes: determining a scene corresponding to the image parameter information; determining a display interface matching the scene based on a correspondence between the scene and the display interface; and rendering a display interface matching the scene.

[0009] According to a second aspect of an embodiment of the present disclosure, there is provided an image rendering method, the method comprising:

[0010] Obtain interface information representing a display interface, wherein a reserved area is reserved in the display interface for displaying an image to be rendered by an extended reality device and is rendered by a device other than the extended reality device; render the image to be rendered in the reserved area; and display the rendered image.

[0011] In one embodiment, the interface information includes the window position and window size information of the image to be rendered; the obtaining of interface information representing the display interface includes: decoding the interface information to obtain the window position and window size information of the image to be rendered; rendering the image to be rendered in the reserved area includes: obtaining the reserved area according to the window position and window size information of the image to be rendered; rendering the image to be rendered in the reserved area.

[0012] According to a third aspect of an embodiment of the present disclosure, there is provided an image rendering device, the device comprising:

[0013] An input unit, used for obtaining image parameter information;

[0014] A processing unit, configured to render a display interface for displaying the image to be rendered based on the image parameter information, wherein a reserved area for displaying the image to be rendered by the extended reality device is reserved in the display interface;

[0015] A sending unit is used to send interface information representing the display interface to an extended reality device, so that the extended reality device renders the image to be rendered in the reserved area.

[0016] In one embodiment, the interface information includes the window position and window size information of the image to be rendered; the sending unit sends the interface information representing the display interface to the extended reality device in the following manner: the window position and window size information of the image to be rendered are encoded and sent to the extended reality device.

[0017] In one embodiment, the processing unit renders a display interface for displaying the image based on the image parameter information in the following manner: determining a scene corresponding to the image parameter information; based on a correspondence between the scene and the display interface, determining a display interface matching the scene; and rendering a display interface matching the scene.

[0018] According to a fourth aspect of an embodiment of the present disclosure, there is provided an image rendering device, the device comprising:

[0019] A receiving unit, configured to obtain interface information representing a display interface, wherein the display interface has a reserved area for displaying an image to be rendered by an extended reality device and is rendered by a device other than the extended reality device;

[0020] A processing unit, configured to render the image to be rendered in the reserved area;

[0021] The display unit is used to display the rendered image.

[0022] In one embodiment, the interface information includes the window position and window size information of the image to be rendered; the receiving unit obtains the interface information representing the display interface in the following manner: decodes the interface information to obtain the window position and window size information of the image to be rendered; the processing unit renders the image to be rendered in the reserved area in the following manner: obtains the reserved area according to the window position and window size information of the image to be rendered; and renders the image to be rendered in the reserved area.

[0023] According to a fifth aspect of an embodiment of the present disclosure, there is provided an image rendering device, including:

[0024] processor;

[0025] a memory for storing processor-executable instructions;

[0026] The processor is configured to: execute the image rendering method described in the first aspect or any one of the implementations of the first aspect.

[0027] According to a sixth aspect of an embodiment of the present disclosure, there is provided an image rendering device, including:

[0028] processor;

[0029] a memory for storing processor-executable instructions;

[0030] Wherein, the processor is configured to: execute the image rendering method described in the second aspect or any one of the implementations of the second aspect.

[0031] According to the seventh aspect of the embodiments of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the image rendering method described in the first aspect or any one of the embodiments of the first aspect.

[0032] According to an eighth aspect of the embodiments of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of an extended reality device, the extended reality device can execute the image rendering method described in the second aspect or any one of the embodiments of the second aspect.

[0033] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: rendering a display interface for displaying an image to be rendered based on image parameter information, wherein a reserved area for displaying the image to be rendered by an extended reality device is reserved in the display interface. Interface information representing the display interface is sent to the extended reality device so that the extended reality device renders the image to be rendered in the reserved area. Through the present disclosure, compared with the solution of rendering the display interface and the image to be displayed, it is possible to save the data transmission bandwidth between the extended reality device and the terminal, reduce the power consumption of the extended reality device and the terminal device, reduce the delay of the preview image, and improve the user experience.

[0034] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0036] Figure 1 The figure is a flowchart of an image rendering method according to an exemplary embodiment.

[0037] Figure 2 The present invention is a flowchart of a method for sending interface information representing a display interface to an extended reality device according to an exemplary embodiment.

[0038] Figure 3 The present invention is a flowchart showing a method for rendering a display interface for displaying an image based on image parameter information according to an exemplary embodiment.

[0039] Figure 4 The figure is a flowchart of an image rendering method according to an exemplary embodiment.

[0040] Figure 5 The present invention is a flowchart of a method for rendering an image to be rendered in a reserved area according to an exemplary embodiment.

[0041] Figure 6 It is a block diagram of an image rendering apparatus for an XR glasses device according to an exemplary embodiment.

[0042] Figure 7 The invention is a block diagram of an image rendering device according to an exemplary embodiment.

[0043] Figure 8 The invention is a block diagram of an image rendering device according to an exemplary embodiment.

[0044] Fig. 9The invention is a block diagram of a device for image rendering according to an exemplary embodiment. DETAILED DESCRIPTION

[0045] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0046] The method provided by the embodiment of the present disclosure can be applied to an image rendering scenario in which images are displayed based on an extended reality device.

[0047] Among the commonly used image rendering methods of extended reality devices, one method is to render completely on the extended reality device side. The extended reality device side undertakes a large number of computing and processing tasks. For the generally used headband or head-mounted mode, the extended reality device using this image rendering method is relatively bulky and the user experience is not good. Another method is to only perform some lightweight computing tasks on the extended reality device side, and the terminal performs image rendering. Among them, the terminal encodes and compresses the rendered image, and sends the encoded and compressed data to the extended reality device side. The extended reality device decodes and displays the image. This method can achieve a relatively lightweight product, but it occupies the data transmission bandwidth between devices and will cause delays. The process of human eyes seeing real-life images has very high requirements for delay. If the delay is large, the human eye will feel uncomfortable, and even feel dizzy and vomiting, and the user experience is not good.

[0048] In view of this, the present disclosure provides a method for image rendering, in which a terminal renders a display interface and reserves a window area (hereinafter referred to as a reserved area) for an image that an extended reality device needs to render and display later. The extended reality device renders the image to be rendered in the reserved area to realize split rendering of image rendering, which can save bandwidth and reduce latency, reduce power consumption, and improve user experience.

[0049] Figure 1 FIG. 1 is a flowchart of an image rendering method according to an exemplary embodiment, and the image rendering method is applied to a terminal. For example, the terminal may be a terminal that interacts with an extended reality device. Figure 1 As shown, the image rendering method includes the following steps:

[0050] In step S11, image parameter information is acquired.

[0051] In the embodiment of the present disclosure, the camera in the extended reality device collects images and sends the collected image parameter information to the terminal. The terminal obtains the image parameter information sent by the extended reality device.

[0052] In step S12, a display interface for displaying the image to be rendered is rendered based on the image parameter information, and a reserved area for displaying the image to be rendered by the extended reality device is reserved in the display interface.

[0053] In the disclosed embodiment, when the terminal performs rendering based on the image parameter information, an area for displaying the image to be rendered by the extended reality device is reserved, and the reserved area is not rendered. This can increase the terminal rendering speed and thereby reduce the image rendering delay and processing power consumption.

[0054] Among them, the image to be rendered by the extended reality device can be understood as a preview image. In the image rendering method provided by the embodiment of the present disclosure, the terminal uses a GPU (graphics processing unit) to render the UI (user interface) interface of the camera of the extended reality device based on some image parameter information of the image to be rendered (which can be information about the image collected by the extended reality device, which can be input information), without a preview image. Among them, the reserved area can be understood as the area in the UI interface for displaying preview images.

[0055] In the embodiment of the present disclosure, the terminal renders other display interfaces for displaying the image to be rendered except for the reserved area.

[0056] In step S13, the interface information representing the display interface is sent to the extended reality device, so that the extended reality device renders the image to be rendered in the reserved area.

[0057] In the disclosed embodiment, there is no need to compress the image of the extended reality device and transmit it to the terminal. It is only necessary to reserve the area where the camera image is to be displayed, which can reduce the power consumption of terminal rendering. The terminal sends the interface information representing the display interface to the extended reality device, which can save the bandwidth of the terminal to transmit information to the extended reality device compared to rendering all images and sending them to the extended reality device.

[0058] The following is an explanation of the image rendering method involved in the above-mentioned embodiment.

[0059] In one implementation, in the disclosed embodiment, the interface information sent by the terminal to the extended reality device needs to include information about the extended reality device rendering in the reserved area. For example, the reserved area is a window. The interface information includes window information, such as window size information of the image to be rendered (also referred to as window size parameter information), window position information (also referred to as coordinate information of the window in the display interface), and other information.

[0060] In the disclosed embodiment, the terminal sends the window position and window size information of the image to be rendered to the extended reality device, and there is no need to send the rendered image, thus saving the terminal's processing flow for generating the preview image. For example, the image collected by the extended reality device does not need to go through a series of operations, such as compression, transmission, reception, decompression, rendering by the terminal, and then compression, transmission, decompression to the glasses for display. This series of complex processes can save power consumption and reduce latency.

[0061] In the disclosed embodiment, when the terminal sends interface information to the extended reality device, in order to further reduce the transmission delay and the transmission bandwidth, the sent interface information can be encoded and compressed and then sent to the extended reality device.

[0062] Figure 2 is a flow chart showing a method of sending interface information representing a display interface to an extended reality device according to an exemplary embodiment. Figure 2 As shown, the following steps are included:

[0063] In step S21, the window position and window size information of the image to be rendered are encoded.

[0064] In step S22, the encoded window position and window size information is sent to the extended reality device.

[0065] In the disclosed embodiment, the terminal encodes the interface information such as the window position information and the window size information of the image to be rendered, and sends it to the extended reality device through the communication connection between the terminal and the extended reality device. For example, the encoded interface information can be sent to the extended reality device via USB (Universal Serial Bus) or WIFI (mobile hotspot). After decoding, the extended reality device renders and displays it based on the window position information and the window size information. Among them, the terminal sends the UI interface information, window position information and window size information including the reserved area to the extended reality device, which can save power consumption relative to the terminal sending the rendered image to the extended reality device after the rendering is completed, and improve the delay problem. Through experiments, it is found that this method can save 30%-40% of power consumption relative to the terminal sending the rendered image to the extended reality device after the rendering is completed.

[0066] Among them, the terminal renders different display interfaces according to different scenes, and different display interfaces correspond to different interface information. In the embodiment of the present disclosure, the correspondence between the predefined scenes and the display interfaces is predefined. When rendering the UI interface, the terminal can determine the display interface of the scene that needs to be rendered currently based on the correspondence between the predefined scenes and the UI interface.

[0067] Figure 3is a flowchart showing a method of rendering a display interface for displaying an image based on image parameter information according to an exemplary embodiment. Figure 3 As shown, the following steps are included:

[0068] In step S31, the scene corresponding to the image parameter information is determined.

[0069] In the disclosed embodiment, when the terminal renders a display interface for displaying an image based on the image parameter information, the image can be subjected to auxiliary scene analysis according to the information collected by the camera of the extended reality device, and different scenes correspond to different reserved area information. For example, the reserved area in the guidance scene is relatively smaller than the reserved area in the non-guidance scene.

[0070] In step S32, based on the correspondence between the scene and the display interface, a display interface matching the scene is determined.

[0071] In the disclosed embodiment, the terminal pre-stores predefined information corresponding to a plurality of scenarios. Different scenarios correspond to different predefined information, and the display interface will also be different.

[0072] In step S33, a display interface matching the scene is rendered.

[0073] In the disclosed embodiment, the display interface rendering method based on the correspondence between the scene and the display interface can reuse the display interface when the scene remains unchanged, without rendering the display interface every time. For example, for the update and transmission of interface information, the extended reality device may collect an image for a certain scene for the first time, render the display interface at the terminal, and send the rendered display interface to the extended reality device. When the extended reality device collects the image of the same scene for the second time, if the interface does not have the update of the interface information, the terminal does not need to render it again, and does not need to send new interface information to the extended reality device. The extended reality device renders the image collected for the second time in the reserved area based on the interface information received for the first time, such as the reserved area. If the scene in which the extended reality device collects the image for the third time changes, the interface information also changes. For example: when the user operates the icon, the terminal will re-render the display interface based on the received changed image parameter information, send the re-rendered interface information to the extended reality device, and the extended reality device renders the image based on the new interface information.

[0074] The image rendering method for extended reality applied to a terminal provided in the embodiment of the present disclosure can save the data transmission bandwidth between the extended reality device and the terminal, reduce the power consumption of image rendering between the extended reality device and the terminal, and reduce the delay in displaying a preview image.

[0075] Based on the same concept, the embodiment of the present disclosure also provides an image rendering method.

[0076] Figure 4 FIG. 1 is a flowchart of an image rendering method according to an exemplary embodiment. The image rendering method is applied to an extended reality device. Figure 4 As shown, the following steps are included:

[0077] In step S41, interface information representing a display interface is obtained.

[0078] In the disclosed embodiment, a reserved area is reserved in the display interface for displaying an image to be rendered by an extended reality device, and the image is rendered by a device other than the extended reality device.

[0079] In one implementation, in the disclosed embodiment, the interface information sent by the terminal to the extended reality device needs to include information rendered by the extended reality device in the reserved area.

[0080] In step S42, the image to be rendered is rendered in the reserved area.

[0081] In the disclosed embodiment, the extended reality device may render and display the preview image received from the vision-related camera sensor in a corresponding reserved area based on the received interface information.

[0082] In step S43, the rendered image is displayed.

[0083] The image rendering method provided by the embodiment of the present disclosure enables the extended reality device to render the image to be rendered in a reserved area without rendering other display interfaces except the image to be rendered, thereby reducing the power consumption of the extended reality device in rendering the image and reducing the delay.

[0084] In one implementation, in the disclosed embodiment, the reserved area is a window. The interface information includes window information, such as window size information and window position information of the image to be rendered.

[0085] The window size information and window position information received by the extended reality device may be information encoded and compressed by the terminal. The extended reality device decodes the received encoded and compressed information to obtain the position and size of the reserved area for rendering the image to be rendered, and then based on the position and size, renders the image to be rendered in the reserved area, adds the image to be rendered to the reserved area, and obtains a rendered image.

[0086] Figure 5 is a flowchart of a method for rendering an image to be rendered in a reserved area according to an exemplary embodiment. Figure 5As shown, the following steps are included:

[0087] In step S51, the interface information is decoded to obtain the window position and window size information including the image to be rendered.

[0088] In the embodiment of the present disclosure, the interface information may include the window position and window size information of the image to be rendered, and may also include a UI interface.

[0089] In step S52, a reserved area is obtained according to the window position and window size information of the image to be rendered.

[0090] In the embodiment of the present disclosure, the reserved area may include content of the UI interface, for example, icons of the UI interface.

[0091] In step S53, the image to be rendered is rendered in the reserved area.

[0092] In the embodiment of the present disclosure, the display interface that can be obtained after the image to be rendered is rendered in the reserved area may include a preview image and may also include UI interface information.

[0093] In the method for image rendering provided by the embodiment of the present disclosure, when rendering an image, the terminal reserves an area for the image to be displayed, and sends the window coordinate information, size parameters and compressed data of the reserved area to the extended reality device. After receiving the compressed data, the extended reality device decompresses it, renders the image based on the window information transmitted by the terminal, and then synthesizes it into the reserved area, transmits the rendered image to the display unit, and displays the image, which can save bandwidth and reduce latency, while reducing the power consumption of the extended reality device and improving the user experience.

[0094] In the disclosed embodiment, the extended reality device displays the rendered image, which may be a display buffer with a preview image, for example: it may include a UI interface and an image placed in a reserved area, and the display buffer with the preview image is sent to a display module for display.

[0095] In the embodiment of the present disclosure, the extended reality is that the image information collected by the vision-related camera in the device is unchanged, for example: the size of the collected image remains unchanged, and after the terminal receives the image parameter information, it renders the display content according to the requirements of different scenes.

[0096] The present disclosure embodiment will now illustrate the above-mentioned image rendering method with reference to examples.

[0097] The extended reality device involved in the embodiments of the present disclosure may be, for example, an XR device such as XR glasses. The terminal may be an intelligent terminal such as a smart phone.

[0098] The following description takes the extended reality device as XR glasses and the terminal as a smartphone as an example.

[0099] In summary, the embodiments of the present disclosure may include multiple cameras in the XR glasses, including vision-related cameras for vision-related operations, such as RGB cameras for image acquisition.

[0100] In the disclosed embodiment, different cameras correspond to different functions, and vision-related information collected by cameras other than the visual camera will also be sent to the terminal. For example, information collected by cameras other than the visual camera for assisting scene analysis will also be sent to the terminal.

[0101] In the disclosed embodiment, the XR glasses may include a WIFI / USB module for interacting with a smartphone and transmitting information with the smartphone.

[0102] In the embodiment of the present disclosure, the XR glasses may include a decoder for decoding information received from the WIFI / USB module and passing the decoded information to the GPU.

[0103] In the disclosed embodiment, the XR glasses may include a GPU for receiving the decoded information transmitted by the decoder and the preview image transmitted by the camera, and based on the decoded information, rendering the preview image in the corresponding reserved area, and sending the rendered interface with the preview image to the display unit. The display unit is located in the XR glasses and is used for displaying images.

[0104] In the embodiment of the present disclosure, the XR glasses may include a display unit for receiving a rendered interface and displaying the interface.

[0105] In the embodiment of the present disclosure, the smart phone may include an input unit for inputting image information and transmitting the image information to the GPU.

[0106] In the disclosed embodiment, the smart phone may include a GPU, which renders the image based on the received image information to obtain the rendered UI image and the window position coordinates and size information, and sends the rendered UI image and the window position coordinates and size information to the encoder.

[0107] In the embodiment of the present disclosure, the smart phone may include an encoder for encoding the rendered UI image and the window position coordinates and size information and transmitting them to the WIFI / USB module.

[0108] In the disclosed embodiment, the smartphone may include a WIFI / USB module for interacting with the XR glasses and transmitting information with the XR glasses. This WIFI / USB module is different from the WIFI / USB module, which is inside the smartphone.

[0109] Figure 6 1 is a block diagram of an image rendering device for an XR glasses device according to an exemplary embodiment. Figure 6 The device includes XR glasses and a smart phone. The XR glasses include a WIFI / USB module, a decoder, a GPU, a camera sensor and a display unit, and the smart phone includes an input unit, a GPU, an encoder and a WIFI / USB module.

[0110] 1. The input unit of the smartphone inputs the image parameter information to the GPU, and the GPU renders the UI interface based on the image parameter information, that is, the UI interface and the window position coordinates and size information of the reserved area are obtained through processing by the GPU of the smartphone.

[0111] 2. The UI interface and the window position coordinates and size information of the reserved area obtained by GPU processing are encoded and sent to the XR glasses via WIFI or USB module.

[0112] 3. After the WIFI or USB module of the XR glasses receives the coded information, it will send the coded information to the decoder for decoding.

[0113] 4. The decoder sends the decoded UI interface and the window position coordinates and size information of the reserved area to the GPU. The preview image captured by the camera of the XR glasses is also sent to the GPU.

[0114] 5. The GPU renders a display buffer with a preview image based on the window position coordinates and size information, and sends it to the display module for display.

[0115] 6. The display unit displays the rendered interface.

[0116] This device uses a split device, and the two devices of the split device need to interact. The device for image acquisition and display can be XR glasses, and the other device of the split device can be a smartphone. Unlike head-mounted devices, the split device is lighter and provides a better user experience.

[0117] Based on the same concept, an embodiment of the present disclosure also provides an image rendering device.

[0118] It is understandable that, in order to realize the above functions, the image rendering device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0119] Figure 7 is a block diagram of an image rendering device according to an exemplary embodiment. Figure 7 The device 100 may be a terminal. The device 100 includes an input unit 101, a processing unit 102 and a sending unit 103.

[0120] The input unit 101 is used to obtain image parameter information.

[0121] The processing unit 102 is used to render a display interface for displaying the image to be rendered based on the image parameter information, and a reserved area for displaying the image to be rendered by the extended reality device is reserved in the display interface.

[0122] The sending unit 103 is used to send the interface information representing the display interface to the extended reality device, so that the extended reality device renders the image to be rendered in the reserved area.

[0123] In one implementation, the interface information includes the window position and window size information of the image to be rendered.

[0124] The sending unit 103 sends the interface information representing the display interface to the extended reality device in the following manner: the window position and window size information of the image to be rendered are encoded and sent to the extended reality device.

[0125] In one implementation, the processing unit 102 renders a display interface for displaying an image based on the image parameter information in the following manner: determining a scene corresponding to the image parameter information, determining a display interface matching the scene based on a correspondence between the scene and the display interface, and rendering the display interface matching the scene.

[0126] Figure 8 is a block diagram of an image rendering device according to an exemplary embodiment. Figure 8 The device 100 may be an extended reality device. The device 200 includes a receiving unit 201 , a processing unit 202 and a display unit 203 .

[0127] The receiving unit 201 is used to obtain interface information representing a display interface, in which a reserved area for displaying an image to be rendered by an extended reality device is reserved and is rendered by a device other than the extended reality device.

[0128] The processing unit 202 is used to render the image to be rendered in the reserved area.

[0129] The display unit 203 is used to display the rendered image.

[0130] In one implementation, the interface information includes the window position and window size information of the image to be rendered.

[0131] The receiving unit 201 obtains the interface information representing the display interface in the following manner: decodes the interface information to obtain the window position and window size information including the image to be rendered. The processing unit 202 renders the image to be rendered in the reserved area in the following manner: obtains the reserved area according to the window position and window size information of the image to be rendered. Renders the image to be rendered in the reserved area.

[0132] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0133] Fig. 9 1 is a block diagram of a device 300 for image rendering according to an exemplary embodiment. For example, the device 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0134] Reference Fig. 9 , the device 300 may include one or more of the following components: a processing component 302 , a memory 304 , a power component 306 , a multimedia component 308 , an audio component 310 , an input / output (I / O) interface 312 , a sensor component 314 , and a communication component 316 .

[0135] The processing component 302 generally controls the overall operation of the device 300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 302 may include one or more modules to facilitate the interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate the interaction between the multimedia component 308 and the processing component 302.

[0136] The memory 304 is configured to store various types of data to support operations on the device 300. Examples of such data include instructions for any application or method operating on the device 300, contact data, phone book data, messages, pictures, videos, etc. The memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0137] The power component 306 provides power to the various components of the device 300. The power component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 300.

[0138] The multimedia component 308 includes a screen that provides an output interface between the device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 includes a front camera and / or a rear camera. When the device 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0139] The audio component 310 is configured to output and / or input audio signals. For example, the audio component 310 includes a microphone (MIC), and when the device 300 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 304 or sent via the communication component 316. In some embodiments, the audio component 310 also includes a speaker for outputting audio signals.

[0140] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0141] The sensor assembly 314 includes one or more sensors for providing various aspects of the status assessment of the device 300. For example, the sensor assembly 314 can detect the open / closed state of the device 300, the relative positioning of components, such as the display and keypad of the device 300, the sensor assembly 314 can also detect the position change of the device 300 or a component of the device 300, the presence or absence of user contact with the device 300, the orientation or acceleration / deceleration of the device 300, and the temperature change of the device 300. The sensor assembly 314 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 314 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0142] The communication component 316 is configured to facilitate wired or wireless communication between the device 300 and other devices. The device 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0143] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0144] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, and the instructions can be executed by the processor 320 of the device 300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0145] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0146] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0147] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0148] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0149] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0150] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. An image rendering method, characterized in that: The method comprises: Acquire image parameter information, and render a display interface for displaying the image to be rendered based on the image parameter information, wherein a reserved area for displaying the image to be rendered by the extended reality device is reserved in the display interface; Interface information representing the display interface is sent to an extended reality device, so that the extended reality device renders the image to be rendered in the reserved area.

2. The method according to claim 1, characterized in that: The interface information includes the window position and window size information of the image to be rendered; The sending the interface information representing the display interface to the extended reality device includes: The window position and window size information of the image to be rendered are encoded and sent to the extended reality device.

3. The method according to claim 2, wherein the rendering of a display interface for displaying the image based on the image parameter information comprises: Determining the scene corresponding to the image parameter information; Based on the correspondence between the scene and the display interface, determining a display interface matching the scene; Rendering a display interface matching the scene.

4. An image rendering method, characterized in that: The method comprises: Acquire interface information representing a display interface, wherein the display interface has a reserved area for displaying an image to be rendered by an extended reality device and is rendered by a device other than the extended reality device; Rendering the image to be rendered in the reserved area; Displays the rendered image.

5. The method according to claim 4, wherein the interface information includes the window position and window size information of the image to be rendered; The obtaining of interface information representing the display interface includes: Decoding the interface information to obtain window position and window size information including the image to be rendered; Rendering the image to be rendered in the reserved area includes: Obtaining a reserved area according to the window position and window size information of the image to be rendered; The image to be rendered is rendered in the reserved area.

6. An image rendering device, characterized in that: include: An input unit, used for obtaining image parameter information; A processing unit, configured to render a display interface for displaying the image to be rendered based on the image parameter information, wherein a reserved area for displaying the image to be rendered by the extended reality device is reserved in the display interface; A sending unit is used to send interface information representing the display interface to an extended reality device, so that the extended reality device renders the image to be rendered in the reserved area.

7. The image rendering device according to claim 6, characterized in that: The interface information includes the window position and window size information of the image to be rendered; The sending unit sends the interface information representing the display interface to the extended reality device in the following manner: The window position and window size information of the image to be rendered are encoded and sent to the extended reality device.

8. The device according to claim 6, characterized in that The processing unit renders a display interface for displaying the image based on the image parameter information in the following manner: Determining the scene corresponding to the image parameter information; Based on the correspondence between the scene and the display interface, determining a display interface matching the scene; Rendering a display interface matching the scene.

9. An image rendering device, characterized in that: include: A receiving unit, configured to obtain interface information representing a display interface, wherein the display interface has a reserved area for displaying an image to be rendered by an extended reality device and is rendered by a device other than the extended reality device; A processing unit, configured to render the image to be rendered in the reserved area; The display unit is used to display the rendered image.

10. The device according to claim 9, characterized in that The interface information includes the window position and window size information of the image to be rendered; The receiving unit obtains the interface information representing the display interface in the following manner: Decoding the interface information to obtain window position and window size information including the image to be rendered; The processing unit renders the image to be rendered in the reserved area in the following manner: Obtaining a reserved area according to the window position and window size information of the image to be rendered; The image to be rendered is rendered in the reserved area.

11. An image rendering device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to: execute the image rendering method described in any one of claims 1 to 3, or execute the image rendering method described in any one of claims 4 to 5.

12. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by the processor of the terminal, the terminal is enabled to execute the image rendering method described in any one of claims 1 to 3; or when the instructions in the storage medium are executed by the processor of the extended display device, the extended reality device is enabled to execute the image rendering method described in any one of claims 4 to 5.

Images