Image updating method and device, electronic equipment and storage medium
By determining whether to render a background image frame based on pose data in an extended reality device, the problem of high hardware resource consumption is solved, latency and frame drops are reduced, and the user experience is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING PICO TECH
- Filing Date
- 2025-08-15
- Publication Date
- 2026-06-23
AI Technical Summary
Existing extended reality devices consume a lot of hardware resources during image updates, which can easily lead to performance issues such as latency and frame drops.
By acquiring the pose data of the extended reality device, it is determined whether a new background image frame needs to be rendered. If not, the last rendered background image frame up to the current moment is used for display, reducing the update frequency of the background image frame.
It reduces the consumption of hardware resources, reduces performance issues such as latency and frame drops, and improves the user experience.
Smart Images
Figure CN120997329B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image updating technology, and in particular to an image updating method, apparatus, electronic device, and storage medium. Background Technology
[0002] Extended Reality (XR) refers to the use of computers to combine the real and virtual worlds, creating an interactive virtual environment. It is also a general term for various technologies such as AR, VR, and MR. Extended Reality brings users a sense of immersion by seamlessly transitioning between the virtual and real worlds.
[0003] In related technologies, when playing video through a player in an extended reality environment using an extended reality device, a multi-layer composite display method is typically used for image updates to ensure that the video played by the player has a higher resolution than the background image frames used to create the extended reality environment. Specifically, during the image update process, background image frames and player video content frames are rendered on different, independent layers, with the resolution of the video content image frames set to be higher than that of the background image frames. Using this method for image updates in extended reality devices consumes significant hardware resources and may also cause performance issues such as latency and dropped frames. Summary of the Invention
[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides an image updating method, apparatus, electronic device, and storage medium.
[0005] In a first aspect, this disclosure provides an image updating method, the method being applicable to extended reality devices, the method comprising:
[0006] In response to the target window being in a displayed state in the extended reality environment, the pose data of the extended reality device is acquired;
[0007] Based on the pose data of the extended reality device, it is determined whether to render a new background image frame within the current image update cycle; the background image frame is an image frame used to create the extended reality environment.
[0008] If it is determined that no new background image frame will be rendered within the current image update cycle, the target background image frame is determined based on the last rendered background image frame up to the current moment.
[0009] The target background image frame is displayed on the screen of the extended reality device.
[0010] Secondly, this disclosure also provides an image updating apparatus suitable for extended reality devices, comprising:
[0011] The acquisition module is used to acquire the pose data of the extended reality device in response to the target window being in a display state in the extended reality environment;
[0012] The first determining module is used to determine, based on the pose data of the extended reality device, whether to render a new background image frame within the current image update cycle; the background image frame is an image frame used to create the extended reality environment;
[0013] The second determining module is used to determine the target background image frame based on the last rendered background image frame up to the current time if it is determined that no new background image frame will be rendered in the current image update cycle.
[0014] A display module is used to display the target background image frame on the screen of the extended reality device.
[0015] Thirdly, this disclosure also provides an electronic device, the electronic device comprising:
[0016] One or more processors;
[0017] Storage device for storing one or more programs;
[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the image update method as described above.
[0019] Fourthly, this disclosure also provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the image update method described above.
[0020] The technical solution provided in this disclosure has the following advantages compared with the prior art:
[0021] The technical solution provided in this disclosure involves setting a response to the target window being in a displayed state in the extended reality environment, acquiring the pose data of the extended reality device; based on the pose data of the extended reality device, determining whether to render a new background image frame in the current image update cycle; the background image frame is an image frame used to create the extended reality environment; if it is determined that no new background image frame will be rendered in the current image update cycle, a target background image frame is determined based on the last rendered background image frame up to the current moment; and the target background image frame is displayed on the screen of the extended reality device. Essentially, the background image frame is not re-rendered in every image update cycle. Instead, based on the pose data of the extended reality device, the impact on the user's viewing experience if an old background image frame is used for display is estimated. If the impact on the user's viewing experience is low, the old background image frame is used. This limits unnecessary updates to the background image frame, reduces the update frequency of the background image frame, thereby reducing the consumption of hardware resources for rendering the background image frame and reducing the probability of performance problems such as latency and frame drops in the extended reality device. Attached Figure Description
[0022] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0023] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0024] Figure 1 A flowchart of an image update method provided in this disclosure embodiment;
[0025] Figure 2 This is a schematic diagram of the structure of an image updating device according to an embodiment of the present disclosure;
[0026] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. Detailed Implementation
[0027] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.
[0028] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.
[0029] Figure 1 The flowchart illustrates an image update method provided in an embodiment of this disclosure. This method can be executed by an image update device, which can be implemented in software and / or hardware and can be configured in an extended reality device.
[0030] The extended reality devices described in this application may include, but are not limited to, the following types:
[0031] The computer-based augmented reality device uses a PC to perform calculations and output data related to augmented reality functions, while the external computer-based augmented reality device uses the data output from the PC to achieve the augmented reality effect.
[0032] Mobile extended reality devices support setting up mobile terminals (such as smartphones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connections with the mobile terminal, the mobile terminal performs calculations related to extended reality functions and outputs data to the mobile extended reality device, such as watching extended reality videos through an app on the mobile terminal.
[0033] All-in-one extended reality devices have processors for performing virtual functions, thus enabling independent extended reality input and output functions. They do not require connection to a PC or mobile terminal, offering a high degree of freedom of use.
[0034] like Figure 1 As shown, the method may specifically include:
[0035] S110. In response to the target window being in a displayed state in the extended reality environment, obtain the pose data of the extended reality device.
[0036] Extended reality environments can be virtual scenes created by extended reality devices using background image frames. These virtual scenes can be simulations of the real world, semi-simulated / semi-fictional, or purely fictional. Examples of extended reality environments include dark environments, movie theater environments, summer beach camping environments, or cozy winter rooms. Different extended reality environments can include different environmental elements. For example, a summer beach camping environment includes the ocean, beach, rocks, tents, and campfire. Users can move around within these extended reality environments.
[0037] The target window can be, for example, a user interface component or display area used to display specific content. For instance, the target window could be a video playback window used to play video. The target window could also be an interactive control panel for an application, or a progress indicator panel displayed when switching to an extended reality environment to show the resource loading status.
[0038] In an extended reality environment, the target window is in a displayed state, which means that the user can view the target window in the extended reality environment.
[0039] Pose data for extended reality devices can be, for example, the position and / or orientation data of the extended reality device in space. In some scenarios, pose data for extended reality devices can be determined from data acquired from at least one of the following: an IMU (Inertial Measurement Unit), a sensor supporting three-degree-of-freedom (3-DoF) tracking, and a sensor supporting six-degree-of-freedom (6-DoF) tracking.
[0040] S120. Based on the pose data of the extended reality device, determine whether to render a new background image frame within the current image update cycle; the background image frame is an image frame used to create the extended reality environment.
[0041] In practice, the images displayed on the screen of an extended reality device are updated at a preset frequency. Each image update process means completing the entire process from image rendering to displaying the rendered image on the screen. The image update method provided in this application performs one image update per execution.
[0042] The current image update cycle is the image update cycle corresponding to the current execution of the image update method provided in this application.
[0043] It should be noted that in actual use, during the image update process, the technical solution provided in this application can be used for some image update cycles, while the technical solution provided in this application can not be used for others.
[0044] Background image frames can be, for example, image frames used to create an extended reality environment, and these background image frames include environmental elements. For example, a background image frame includes environmental elements such as walls, seats, steps, and other audience members. Because the background image frame includes these environmental elements, users feel as if they are in a movie theater environment when viewing it.
[0045] The essence of this step is that instead of re-rendering a new background image every time the image displayed on the extended reality device's screen is updated, a new background image is re-rendered in some scenarios and not in others.
[0046] The reason for setting up a method based on the pose data of the extended reality device to determine whether to render a new background image frame within the current image update cycle is to use the pose data of the extended reality device to estimate whether not rendering a new background image frame within the current image update cycle will cause strong visual discomfort to the user, such as causing the user to perceive image stuttering.
[0047] Studies have shown that when a user's head movements are small, using existing background image frames instead of new ones in the current image update cycle does not cause strong visual discomfort. However, when head movements are large, using existing background image frames instead of new ones in the current image update cycle will cause strong visual discomfort for the user.
[0048] Based on this, optionally, the implementation method of this step may include: determining the amount of change in the pose data of the extended reality device within a preset time period based on the pose data of the extended reality device; and determining whether to render a new background image frame within the current image update cycle based on the relationship between the amount of change in the pose data of the extended reality device and a set threshold.
[0049] The preset time length can be, for example, a pre-specified time length, a fixed time length set for calculating the changes in pose data of the extended reality device. This application does not limit the specific value of the preset time length. In practice, it can be set according to actual needs.
[0050] Optionally, the change in the pose data of the extended reality device within a preset time length includes the change in the displacement vector magnitude and / or angle of the extended reality device within the preset time length.
[0051] The threshold value can be set as a parameter to measure whether the user's head movement is too large. This application does not restrict the specific value of the threshold. In practice, it can be set according to actual needs.
[0052] Furthermore, the relationship between the change in pose data of the extended reality device and a set threshold can be set to determine whether to render a new background image frame in the current image update cycle, including: if the change in pose data of the extended reality device is less than the set threshold, it is determined that no new background image frame will be rendered in the current image update cycle; if the change in pose data of the extended reality device is greater than or equal to the set threshold, it is determined that a new background image frame will be rendered in the current image update cycle.
[0053] If the change in pose data of the extended reality device is less than a set threshold, it means that the user's head movement is small. Using the background image frame rendered in the previous image update cycle will not cause strong visual discomfort to the user. Therefore, it is determined that no new background image frame will be rendered in the current image update cycle to reduce the hardware resource consumption caused by rendering background image frames.
[0054] If the change in pose data of the extended reality device is greater than or equal to a set threshold, it means that the user's head movement is large. If a new background image frame is not rendered within the current image update cycle, it will cause strong visual discomfort to the user. Therefore, it is determined to render a new background image frame within the current image update cycle.
[0055] S130. If it is determined that no new background image frame will be rendered within the current image update cycle, the target background image frame is determined based on the last rendered background image frame up to the current time.
[0056] Optionally, if it is determined that a new background image frame will be rendered within the current image update cycle, the new background image frame rendered within the current image update cycle will be used as the target background image frame.
[0057] For example, suppose that in the Nth image update cycle, the image update method provided in this application is not used, that is, the step of "determining whether to render a new background image frame in the current image update cycle based on the pose data of the extended reality device" is not performed, and a new background image frame is rendered by default in each image update cycle. Suppose that a new background image frame, named background image frame a, is rendered in the Nth image update cycle, and background image frame a is used as the target background image frame in the Nth image update cycle.
[0058] In the N+1th image update cycle, the image update method provided in this application is used to determine that no new background image frame will be rendered in the N+1th image update cycle. Since background image frame a is the last background image frame rendered up to the current time, the target background image frame for the N+1th image update cycle is determined based on background image frame a.
[0059] In the N+2th image update cycle, the image update method provided in this application is used to determine that no new background image frame will be rendered in the N+2th image update cycle. Since background image frame a is the last background image frame rendered up to the current time, the target background image frame for the N+2th image update cycle is determined based on background image frame a.
[0060] In the N+3rd image update cycle, the image update method provided in this application is used to determine that a new background image frame will be rendered in the N+3rd image update cycle. Assuming that the rendered new background image frame is background image frame b, background image frame b will be used as the target background image frame in the N+3rd image update cycle.
[0061] Optionally, there are various methods for "determining the target background image frame based on the last rendered background image frame up to the current time," and this application does not limit this method. For example, determining the target background image frame based on the last rendered background image frame up to the current time includes: using the last rendered background image frame up to the current time as the target background image frame. This setting can significantly reduce the hardware resource consumption of image updates.
[0062] Alternatively, "determining the target background image frame based on the last rendered background image frame up to the current moment" can include: adjusting the last rendered background image frame up to the current moment based on the pose data of the extended reality device to obtain the target background image frame, so that the target background image frame is adapted to the pose data of the current extended reality device. Essentially, this setup involves asynchronously time-warping the last rendered background image up to the current moment based on the pose data of the extended reality device to generate a new background image frame; this newly generated background image frame is then used as the target background image frame. This allows for synchronized changes in the background image frame with head movement, giving the user the visual perception that the image changes with head movement.
[0063] S140. Display the target background image frame on the screen of the extended reality device.
[0064] There are multiple ways to implement this step, and this application does not limit them. For example, the implementation method of this step includes: rendering a new first image frame corresponding to the target window within the current image update cycle; rendering the background image frame and the first image frame on independent layers; using the first image frame as the content of the target window; compositing the new first image frame with the target background image frame to obtain a second image frame; and displaying the second image frame on the screen of the extended reality device.
[0065] In image rendering, layers are the basic units that carry image content. Each layer can be drawn and updated independently, possessing its own content, attributes (such as transparency), and rendering logic. To achieve efficient image processing, the background image frame and the first image frame are rendered on two independent layers, ensuring that they do not interfere with each other during the drawing process and that the rendering frequency can be flexibly controlled according to their respective update needs. To support this independent rendering mechanism, optionally, three buffers are pre-configured: a first buffer, a second buffer, and a third buffer. The first buffer stores the first image frame, i.e., the content corresponding to the target window. The first image frame is re-rendered in each image update cycle. The second buffer stores the background image frame; using the technical solution provided in this application, the background image frame is not re-rendered in every image update cycle, but only when needed. The third buffer serves as a compositing buffer, used to overlay and blend the image frames from the first and second buffers to generate the final image frame (i.e., the second image frame) for display on the screen. By assigning different image content to different independent layers and using different buffers, isolation between layers can be achieved, and the rendering frequency of background image frames can be more controllable, which helps to reduce the hardware consumption caused by rendering background image frames.
[0066] To reiterate, in the technical solution provided in this application, the first image frame corresponding to the target window is updated by default during each image update process.
[0067] The above technical solution obtains the pose data of the extended reality device in response to the target window being displayed in the extended reality environment; based on the pose data, it determines whether to render a new background image frame within the current image update cycle; the background image frame is the image frame used to create the extended reality environment; if it is determined that no new background image frame should be rendered within the current image update cycle, the target background image frame is determined based on the last rendered background image frame up to the current moment; and the target background image frame is displayed on the screen of the extended reality device. Essentially, the background image frame is not re-rendered in every image update cycle. Instead, based on the pose data of the extended reality device, the impact on the user's viewing experience if an old background image frame is used for display is estimated. If the impact on the user's viewing experience is low, the old background image frame is used. This limits unnecessary updates to the background image frame, reduces the update frequency, and thus reduces the consumption of hardware resources for rendering the background image frame, reducing the probability of performance issues such as latency and frame drops in the extended reality device.
[0068] Based on the above technical solution, S110 may optionally include: in response to the target window being displayed in the extended reality environment, determining whether the user is looking at the target window; if the user is looking at the target window, acquiring the pose data of the extended reality device.
[0069] In practice, there are various methods to determine whether a user is gazing at a target window, and this application does not limit this approach. For example, gaze tracking technology can be used to determine whether a user is gazing at a target window. Specifically, gaze tracking technology can utilize built-in sensors in extended reality devices, such as head-tracking sensors and eye-tracking sensors, to capture and calculate the user's head and / or eye motion data; and convert the motion data captured by the head-tracking sensors and / or eye-tracking sensors into gaze location information. Specifically, "converting the motion data captured by the head-tracking sensors and / or eye-tracking sensors into gaze location information" can include: using a Pupil-Corneal Reflection (Pupil-CR) eye-tracking algorithm to convert the motion data captured by the head-tracking sensors and / or eye-tracking sensors into gaze location information. Alternatively, the motion data captured by the head-tracking sensors and / or eye-tracking sensors can be input into a deep learning model capable of gaze prediction to obtain the gaze location information.
[0070] In practice, when a user is focused on the target window, their sensitivity to images other than the target window is low. In this case, reducing the rendering frequency of background image frames has little impact on the user. However, if the user is focused on an area other than the target window, their sensitivity to background image frames increases significantly. In this case, reducing the rendering frequency of background image frames will result in noticeable screen stuttering, affecting the user's viewing experience. By setting a response to the target window being displayed in the extended reality environment, the system determines whether the user is focused on the target window. If the user is focused on the target window, the system acquires the pose data of the extended reality device. Essentially, this means that the image update method proposed in this application is executed only when the user is focused on the target window, thus avoiding the user's perception of screen stuttering due to using the technical solution provided in this application at inappropriate times.
[0071] Furthermore, if the target window is a video playback window, the resolution of the first image frame is greater than the resolution of the background image frame. This setting allows the video played in the video playback window to have a higher visual effect.
[0072] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.
[0073] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.
[0074] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0075] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.
[0076] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0077] Figure 2 This is a schematic diagram of an image updating device according to an embodiment of this disclosure. The image updating device provided in this embodiment can be configured in a client or a server. See also Figure 2 The image updating device specifically includes:
[0078] The acquisition module 310 is used to acquire the pose data of the extended reality device in response to the target window being in a display state in the extended reality environment;
[0079] The first determining module 320 is used to determine, based on the pose data of the extended reality device, whether to render a new background image frame within the current image update cycle; the background image frame is an image frame used to create the extended reality environment;
[0080] The second determining module 330 is used to determine the target background image frame based on the last rendered background image frame up to the current time if it is determined that no new background image frame will be rendered in the current image update cycle.
[0081] Display module 340 is used to display the target background image frame on the screen of the extended reality device.
[0082] Furthermore, the first determining module 320 is used for:
[0083] Based on the pose data of the augmented reality device, determine the amount of change in the pose data of the augmented reality device within a preset time period;
[0084] Based on the relationship between the change in pose data of the extended reality device and a set threshold, it is determined whether to render a new background image frame within the current image update cycle.
[0085] Furthermore, the first determining module 320 is used for:
[0086] If the change in the pose data of the extended reality device is less than a set threshold, it is determined that no new background image frame will be rendered in the current image update cycle.
[0087] If the change in the pose data of the extended reality device is greater than or equal to a set threshold, it is determined to render a new background image frame within the current image update cycle.
[0088] Furthermore, the second determining module 330 is also used for:
[0089] If it is determined that a new background image frame will be rendered within the current image update cycle, the new background image frame rendered within the current image update cycle will be used as the target background image frame.
[0090] Furthermore, the second determining module 330 is also used for:
[0091] Based on the pose data of the extended reality device, the background image frame rendered up to the current time is adjusted to obtain a target background image frame, so that the target background image frame is adapted to the pose data of the current extended reality device.
[0092] Furthermore, the display module 340 is used for:
[0093] Within the current image update cycle, a new first image frame corresponding to the target window is rendered; the background image frame and the first image frame are rendered on separate layers; the first image frame is used as the content of the target window.
[0094] The new first image frame is combined with the target background image frame to obtain the second image frame;
[0095] The second image frame is displayed on the screen of the extended reality device.
[0096] Furthermore, module 310 is used for:
[0097] In response to the target window being displayed in an extended reality environment, determine whether the user is looking at the target window;
[0098] If the user is looking at the target window, the pose data of the extended reality device is obtained.
[0099] Furthermore, the target window is a video playback window, and the resolution of the first image frame is greater than the resolution of the background image frame.
[0100] The image updating apparatus provided in this disclosure can execute the steps performed by the client or server in the image updating method provided in this disclosure, and has the execution steps and beneficial effects, which will not be described in detail here.
[0101] The following is a detailed reference. Figure 3 The diagram illustrates a structural schematic suitable for implementing the electronic device 1000 in the embodiments of this disclosure. The electronic device 1000 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, etc., as well as fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0102] like Figure 3 As shown, the electronic device 1000 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1008 into a random access memory (RAM) 1003 to implement the image update method as described in the embodiments of this disclosure. The RAM 1003 also stores various programs and information required for the operation of the electronic device 1000. The processing device 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.
[0103] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic device 1000 to exchange information with other devices wirelessly or via wired communication. Although Figure 3 An electronic device 1000 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0104] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the image update method as described above. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1009, or installed from storage device 1008, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of embodiments of this disclosure.
[0105] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include information signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated information signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0106] In some implementations, clients and servers may communicate using any known or future network protocol, such as HTTP (Hypertext Transfer Protocol), and may interconnect with any form or medium of digital information communication (e.g., a communication network). Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any known or future network.
[0107] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
[0108] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to:
[0109] In response to the target window being in a displayed state in the extended reality environment, the pose data of the extended reality device is acquired;
[0110] Based on the pose data of the extended reality device, it is determined whether to render a new background image frame within the current image update cycle; the background image frame is an image frame used to create the extended reality environment.
[0111] If it is determined that no new background image frame will be rendered within the current image update cycle, the target background image frame is determined based on the last rendered background image frame up to the current moment.
[0112] The target background image frame is displayed on the screen of the extended reality device.
[0113] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.
[0114] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0116] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.
[0117] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0118] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0119] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including:
[0120] One or more processors;
[0121] Memory, used to store one or more programs;
[0122] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the image update methods provided in this disclosure.
[0123] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements an image update method as described in any of the present disclosure.
[0124] This disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, implement the image update method described above.
[0125] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0126] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An image update method, characterized in that, The method is applicable to extended reality devices, and the method includes: In response to the target window being in a displayed state in the extended reality environment, the pose data of the extended reality device is acquired; Based on the pose data of the extended reality device, it is determined whether to render a new background image frame within the current image update cycle; the background image frame is an image frame used to create the extended reality environment. If it is determined that no new background image frame will be rendered within the current image update cycle, the target background image frame is determined based on the last rendered background image frame up to the current moment. The target background image frame is displayed on the screen of the extended reality device; and, within the current image update cycle, a new first image frame corresponding to the target window is rendered, the first image frame being used as the content of the target window.
2. The method according to claim 1, characterized in that, The step of determining whether to render a new background image frame within the current image update cycle based on the pose data of the extended reality device includes: Based on the pose data of the extended reality device, determine the amount of change in the pose data of the extended reality device within a preset time period; Based on the relationship between the change in pose data of the extended reality device and a set threshold, it is determined whether to render a new background image frame within the current image update cycle.
3. The method according to claim 2, characterized in that, The determination of whether to render a new background image frame within the current image update cycle based on the relationship between the change in pose data of the extended reality device and a set threshold includes: If the change in the pose data of the extended reality device is less than a set threshold, it is determined that no new background image frame will be rendered in the current image update cycle. If the change in the pose data of the extended reality device is greater than or equal to a set threshold, it is determined to render a new background image frame within the current image update cycle.
4. The method according to claim 3, characterized in that, The method further includes: If it is determined that a new background image frame will be rendered within the current image update cycle, the new background image frame rendered within the current image update cycle will be used as the target background image frame.
5. The method according to claim 1, characterized in that, Based on the last rendered background image frame up to the current moment, the target background image frame is determined, including: Based on the pose data of the extended reality device, the background image frame rendered up to the current time is adjusted to obtain a target background image frame, so that the target background image frame is adapted to the pose data of the current extended reality device.
6. The method according to claim 1, characterized in that, Displaying the target background image frame on the screen of the extended reality device includes: The background image frame and the first image frame are rendered on separate layers; The new first image frame is combined with the target background image frame to obtain the second image frame; The second image frame is displayed on the screen of the extended reality device.
7. The method according to claim 1, characterized in that, The step of acquiring the pose data of the extended reality device in response to the target window being in a displayed state in the extended reality environment includes: In response to the target window being displayed in an extended reality environment, determine whether the user is looking at the target window; If the user is looking at the target window, the pose data of the extended reality device is obtained.
8. The method according to claim 6, characterized in that, The target window is a video playback window, and the resolution of the first image frame is greater than the resolution of the background image frame.
9. An image updating device, characterized in that, The device is suitable for extended reality devices, including: The acquisition module is used to acquire the pose data of the extended reality device in response to the target window being in a display state in the extended reality environment; The first determining module is used to determine, based on the pose data of the extended reality device, whether to render a new background image frame within the current image update cycle; the background image frame is an image frame used to create the extended reality environment; The second determining module is used to determine the target background image frame based on the last rendered background image frame up to the current time if it is determined that no new background image frame will be rendered in the current image update cycle. The display module is configured to display the target background image frame on the screen of the extended reality device; and, within the current image update cycle, render a new first image frame corresponding to the target window, the first image frame being used as the content of the target window.
10. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-8.