Display method and electronic device

Abstract

The present application relates to the technical field of terminals. Provided are a display method and an electronic device. After an electronic device displays a first picture including a real scene image and a virtual scene image, the electronic device can acquire pose information of a user and display a second picture on the basis of the pose information of the user, wherein the second picture is obtained on the basis of adjusting the first picture. The image quality of the virtual scene image in the first picture is superior to the image quality of the virtual scene image in the second picture; alternatively, the image quality of the real scene image in the first picture is superior to the image quality of the real scene image in the second picture. By means of lowering the image quality of the real scene image or the virtual scene image in the first picture, the electronic device can reduce computational resources allocated to the real scene image or the virtual scene image included in the first picture, thereby saving on the computational resources of the electronic device.

Description

Display method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on September 6, 2023, with application number 202311148413.X and application name “Display Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a display method and electronic device. Background Art

[0003] Devices with mixed reality (MR) capabilities are referred to as MR devices. MR devices can overlay images of the real world with virtual content to display an MR image. Because users have limited field of view and attention, when using MR devices, they may focus on one part of the MR image while ignoring another. For example, when watching a movie using an MR device, users may focus more on the movie image and ignore the surrounding image.

[0004] When rendering MR images, MR devices typically use the same computing resources to render all regions of the same MR image frame. However, since users may ignore parts of the MR image, using the same computing resources to render both the ignored and the focused regions of the MR image results in a waste of computing resources.

[0005] Summary of the Invention

[0006] Embodiments of the present application provide a display method and an electronic device that can dynamically adjust the computing resources allocated to rendering each region on an MR image, thereby saving computing resources used for rendering the MR image.

[0007] To achieve the above objectives, the present invention adopts the following technical solutions:

[0008] In a first aspect, an embodiment of the present application provides a display method, which is applied to an electronic device, and the electronic device can display a first screen. The first screen includes a real scene image and a virtual scene image. The real scene image includes an image of the environment surrounding the electronic device, and the virtual scene image includes an image corresponding to virtual content. The electronic device displays a second screen based on the user's posture information. The second screen is obtained based on the adjustment of the first screen. The image quality of the virtual scene image in the first screen is better than the image quality of the virtual scene image in the second screen, or the image quality of the real scene image in the first screen is better than the image quality of the real scene image in the second screen.

[0009] In the method of the embodiment of the present application, the electronic device can strengthen the visual effect of the real scene image in the first screen and weaken the visual effect of the virtual scene image in the first screen by adjusting the first screen. This satisfies the differentiated needs of users for the display content of the electronic device, which is beneficial to the user's experience when looking at the real scene image in the second screen. In addition, the electronic device can save the computing resources of the electronic device by reducing the computing resources allocated to the virtual scene image included in the first screen. Alternatively, the electronic device can strengthen the visual effect of the virtual scene image in the first screen and weaken the visual effect of the real scene image in the first screen by adjusting the first screen. This satisfies the differentiated needs of users for the display content of the electronic device, which is beneficial to the user's experience when looking at the virtual scene image in the second screen. In addition, the electronic device can save the computing resources of the electronic device by reducing the computing resources allocated to the real scene image included in the first screen.

[0010] In combination with the first aspect, in a possible design, the image quality of the virtual image is related to at least one of the following information: the area of ​​the virtual image, the distance between the virtual image and the edge of the picture where the virtual image is located, and the rendering parameters of the virtual image.

[0011] The image quality of the real scene image is related to at least one of the following information: the area of ​​the real scene image, the distance between the real scene image and the edge of the screen where the real scene image is located, the rendering parameters of the real scene image, and the shooting parameters of the camera used to obtain the real scene image.

[0012] The rendering parameters include at least one of a clarity parameter, a resolution parameter, a color saturation parameter, a brightness parameter, a color temperature parameter, and a contrast parameter. The shooting parameters include at least one of a shooting frame rate and a shutter speed.

[0013] In this solution, the electronic device can strengthen or weaken the visual effect of the virtual image in the first screen by adjusting at least one of the area of ​​the virtual image in the first screen, the distance between the virtual image and the edge of the screen where the virtual image is located, and the rendering parameters of the virtual image. The electronic device can also strengthen or weaken the visual effect of the real image in the first screen by adjusting at least one of the area of ​​the real image in the first screen, the distance between the real image and the edge of the screen where the real image is located, the rendering parameters of the real image, and the shooting parameters of the camera used to obtain the real image. By strengthening the visual effect of the real image or virtual image in the first screen, the electronic device can meet the differentiated needs of users for the display content of the electronic device.

[0014] In conjunction with the first aspect, in one possible design, at least one item of information corresponding to the virtual image in the first picture is adjusted to obtain the second picture. And / or at least one item of information corresponding to the real image in the first picture is adjusted to obtain the second picture.

[0015] In this solution, the electronic device can obtain and display a second screen by adjusting at least one item of information corresponding to the virtual image in the first screen and / or by adjusting at least one item of information corresponding to the real image in the first screen. The visual effect of the real image in the second screen is stronger than the visual effect of the virtual image in the second screen. Alternatively, the visual effect of the virtual image in the second screen is stronger than the visual effect of the real image in the second screen. This can meet users' differentiated needs for display content on electronic devices.

[0016] In conjunction with the first aspect, in one possible design, when the image quality of the virtual image in the first screen is better than the image quality of the virtual image in the second screen, the area of ​​the virtual image in the first screen is larger than the area of ​​the virtual image in the second screen. When the image quality of the real image in the first screen is better than the image quality of the real image in the second screen, the area of ​​the virtual image in the first screen is smaller than the area of ​​the virtual image in the second screen.

[0017] In this solution, if the area of ​​the virtual image in the first screen is larger than the area of ​​the virtual image in the second screen, and the area of ​​the real image blocked by the virtual image in the second screen is smaller than the area of ​​the real image blocked by the virtual image in the first screen, the user experience when viewing the real image included in the to-be-rendered screen can be improved. If the area of ​​the virtual image in the first screen is smaller than the area of ​​the virtual image in the second screen, and the area of ​​the real image blocked by the virtual image in the second screen is larger than the area of ​​the real image blocked by the virtual image in the first screen, the user experience when viewing the virtual image included in the to-be-rendered screen can be improved.

[0018] In combination with the first aspect, in a possible design, when the image quality of the virtual scene image in the first picture is better than the image quality of the virtual scene image in the second picture, the distance between the virtual scene image in the second picture and the edge of the second picture is smaller than the distance between the virtual scene image in the first picture and the edge of the first picture.

[0019] In this solution, the distance between the virtual scene image in the second screen and the center of the user's field of view is greater than the distance between the virtual scene image in the first screen and the center of the user's field of view, which can improve the user's experience when viewing the real scene image included in the screen to be rendered.

[0020] In combination with the first aspect, in one possible design, when the image quality of the real scene image in the first screen is better than the image quality of the real scene image in the second screen, the distance between the virtual scene image in the first screen and the edge of the first screen is smaller than the distance between the virtual scene image in the second screen and the edge of the second screen.

[0021] In this solution, the distance between the virtual image in the second screen and the center of the user's field of view is smaller than the distance between the virtual image in the first screen and the center of the user's field of view, which can improve the user's experience when viewing the virtual image included in the screen to be rendered.

[0022] In conjunction with the first aspect, in one possible design, when the image quality of the virtual scene image in the first screen is better than the image quality of the virtual scene image in the second screen, the image quality of the real scene image in the second screen is better than the image quality of the real scene image in the first screen. When the image quality of the real scene image in the first screen is better than the image quality of the real scene image in the second screen, the image quality of the virtual scene image in the second screen is better than the image quality of the virtual scene image in the first screen.

[0023] In this solution, the electronic device can improve the image quality of the real scene image in the first screen while reducing the image quality of the virtual scene image in the first screen. This can enhance the visual effect of the real scene image in the first screen and weaken the visual effect of the virtual scene image in the first screen. This meets the differentiated needs of users for the display content of electronic devices and improves the user experience when viewing the real scene image in the second screen.

[0024] While reducing the image quality of the real scene image in the first screen, the electronic device can also increase the image quality of the virtual scene image in the first screen. This can enhance the visual effect of the virtual scene image in the first screen and weaken the visual effect of the real scene image in the first screen. This meets the differentiated needs of users for the display content of electronic devices and improves the user experience when viewing the virtual scene image in the second screen.

[0025] In conjunction with the first aspect, in one possible design, the posture information includes information about the user's gaze point on the first screen. The gaze point information is used to characterize the location of the user's gaze point on the first screen. When the gaze point is located at the location of the real scene image in the first screen, the image quality of the virtual scene image in the first screen is better than the image quality of the virtual scene image in the second screen. Alternatively, when the gaze point is located at the location of the virtual scene image in the first screen, the image quality of the real scene image in the first screen is better than the image quality of the real scene image in the second screen.

[0026] In this solution, since the gaze point information is used to represent the position of the user's gaze point on the first screen, the electronic device can determine whether the content the user is focusing on in the first screen is a real image or a virtual image based on the user's gaze point information.

[0027] When the gaze point is located at the location of the real-scene image in the first screen, the electronic device can determine that the content the user is focusing on in the first screen is the real-scene image. Therefore, the electronic device can adjust the first screen by reducing the image quality of the virtual-scene image in the first screen. On the one hand, the electronic device can reduce the image quality of the virtual-scene image in the first screen to reduce the computing resources allocated to the virtual-scene image included in the first screen, thereby saving the computing resources of the electronic device. On the other hand, the electronic device can enhance the visual effect of the real-scene image in the first screen by reducing the image quality of the virtual-scene image in the first screen, meeting the user's demand for differentiated display content.

[0028] When the gaze point is located at the position of the virtual image in the first screen, the electronic device can determine that the content that the user is paying attention to in the first screen is a virtual image. Therefore, the electronic device can adjust the first screen by reducing the image quality of the real image in the first screen. On the one hand, the electronic device can reduce the image quality of the real image in the first screen to reduce the computing resources allocated to the real image included in the first screen, thereby saving the computing resources of the electronic device. On the other hand, the electronic device can enhance the visual effect of the virtual image in the first screen by reducing the image quality of the real image in the first screen, meeting the user's demand for differentiated display content.

[0029] In combination with the first aspect, in one possible design, the posture information includes the user's head rotation information. The user's head rotation information includes the user's head rotation angle in the horizontal direction and the user's head rotation angle in the vertical direction. When the user's head rotation angle in the horizontal direction is greater than a first threshold, and / or the user's head rotation angle in the vertical direction is greater than a second threshold, the image quality of the virtual scene image in the first screen is better than the image quality of the virtual scene image in the second screen. When the user's head rotation angle in the horizontal direction is less than or equal to the first threshold, and the user's head rotation angle in the vertical direction is less than or equal to the second threshold, the image quality of the real scene image in the first screen is better than the image quality of the real scene image in the second screen.

[0030] In this solution, the electronic device can determine whether the content the user is focusing on in the first screen is a real image or a virtual image based on the scenario in which the user is capturing a real image corresponding to an image of the real-world environment. For example, the user may be looking down at a physical object such as a mobile phone, tablet, or paper document. The user may also be looking up at the sky. The user may be walking with their head down. The user may be turning their head left and right to look at the surrounding environment.

[0031] After the electronic device displays the first screen, if it detects that the rotation angle of the user's head in the horizontal direction is greater than a first threshold, and / or that the rotation angle of the user's head in the vertical direction is greater than a second threshold, it can be determined that the content that the user is paying attention to in the first screen is a real-scene image. Therefore, the electronic device can adjust the first screen by reducing the image quality of the virtual scene image in the first screen. On the one hand, the electronic device can reduce the image quality of the virtual scene image in the first screen to reduce the computing resources allocated to the virtual scene image included in the first screen, thereby saving computing resources of the electronic device. On the other hand, the electronic device can enhance the visual effect of the real scene image in the first screen by reducing the image quality of the virtual scene image in the first screen, thereby meeting the user's demand for differentiated display content.

[0032] After the electronic device displays the first screen, if it detects that the rotation angle of the user's head in the horizontal direction is less than or equal to the first threshold, and the rotation angle of the user's head in the vertical direction is less than or equal to the second threshold, it can be determined that the content that the user is paying attention to in the first screen is a virtual image. Therefore, the electronic device can adjust the first screen by reducing the image quality of the real-scene image in the first screen. On the one hand, the electronic device can reduce the image quality of the real-scene image in the first screen to reduce the computing resources allocated to the real-scene image included in the first screen, thereby saving computing resources of the electronic device. On the other hand, the electronic device can reduce the image quality of the real-scene image in the first screen to enhance the visual effect of the virtual image in the first screen, thereby meeting the user's demand for differentiated display content.

[0033] In conjunction with the first aspect, in one possible design, the posture information includes user movement information. The user movement information is used to characterize the user's head movement trajectory. When the distance the user's head moves is greater than a preset distance, the image quality of the virtual scene image in the first screen is better than the image quality of the virtual scene image in the second screen. When the distance the user's head moves is less than or equal to the preset distance, the image quality of the real scene image in the first screen is better than the image quality of the real scene image in the second screen. The distance the user's head moves is obtained based on the user's movement information.

[0034] In this solution, because the user's movement information represents the user's head movement trajectory, the electronic device can determine the distance the user's head has moved based on the user's movement information. Based on the distance the user's head has moved, the electronic device can determine whether the content the user is focusing on in the first screen is a real image or a virtual image.

[0035] If the electronic device determines that the distance the user's head has moved is greater than a preset distance, it can determine that the content the user is focusing on in the first screen is a real-scene image. Therefore, the electronic device can adjust the first screen by reducing the image quality of the virtual-scene image in the first screen. On the one hand, by reducing the image quality of the virtual-scene image in the first screen, the electronic device can reduce the computing resources allocated to the virtual-scene image included in the first screen, thereby saving computing resources of the electronic device. On the other hand, by reducing the image quality of the virtual-scene image in the first screen, the electronic device can enhance the visual effect of the real-scene image in the first screen, meeting the user's demand for differentiated display content.

[0036] If the electronic device determines that the distance the user's head has moved is less than or equal to a preset distance, it can be determined that the content the user is focusing on in the first screen is a virtual image. Therefore, the electronic device can adjust the first screen by reducing the image quality of the real image in the first screen. On the one hand, the electronic device can reduce the image quality of the real image in the first screen to reduce the computing resources allocated to the real image included in the first screen, thereby saving computing resources of the electronic device. On the other hand, the electronic device can enhance the visual effect of the virtual image in the first screen by reducing the image quality of the real image in the first screen, thereby meeting the user's demand for differentiated display content.

[0037] In combination with the first aspect, in one possible design, the target application in the electronic device corresponds to at least one application screen, and the at least one application screen includes a third screen. Before displaying the first screen, the electronic device may display a setting interface. The setting interface is used to set a rendering strategy for at least one application screen. The rendering strategy is used to adjust the screen to be rendered. The electronic device may receive a first trigger operation input by the user in the setting interface. The first trigger operation is used to indicate that the rendering strategy for at least one application screen is set to the target rendering strategy. In response to the first trigger operation, the electronic device sets the rendering strategy for at least one application screen to the target rendering strategy.

[0038] In this solution, the electronic device can provide the user with a setting interface for setting a rendering strategy for at least one application screen, so as to meet the user's differentiated needs for displaying content of each application screen.

[0039] In conjunction with the first aspect, in one possible design, the electronic device may receive a second trigger operation input by a user. The second trigger operation is used to instruct the display of a third screen. In response to the second trigger operation, the electronic device adjusts the third screen based on the target rendering strategy. Thereafter, the electronic device may display the first screen resulting from the adjustment of the third screen.

[0040] In this solution, the electronic device can render each application screen based on the rendering strategy customized by the user for each application screen, so as to meet the user's differentiated needs for the display content of each application screen.

[0041] In conjunction with the first aspect, in one possible design, the rendering strategy includes a first rendering strategy and a second rendering strategy. The second image is obtained by adjusting the rendering strategy for the first image. The first rendering strategy includes reducing rendering parameters of a virtual image included in the image to be rendered. The second rendering strategy includes reducing rendering parameters of a real image included in the image to be rendered. The rendering parameters include at least one of a clarity parameter, a resolution parameter, a color saturation parameter, a brightness parameter, a color temperature parameter, and a contrast parameter.

[0042] In this solution, possible settings for rendering strategies are provided.

[0043] In conjunction with the first aspect, in one possible design, the first rendering strategy further includes increasing rendering parameters of a real scene image included in the image to be rendered. Furthermore, the first rendering strategy also includes reducing the size of a virtual scene image included in the image to be rendered. Furthermore, the first rendering strategy also includes adjusting the virtual scene image included in the image to be rendered toward an edge of the image to be rendered. Furthermore, the first rendering strategy also includes increasing at least one of a camera's frame rate or shutter speed. The real scene image included in the image to be rendered is acquired by a camera.

[0044] In this solution, a possible setting method for the first rendering strategy is provided. By adjusting the image to be rendered through the first rendering strategy, the electronic device can enhance the visual effect of the real scene image in the image to be rendered and weaken the visual effect of the virtual scene image in the image to be rendered. This meets the differentiated needs of users for the display content of electronic devices and is beneficial to the user's experience when looking at the real scene image in the image to be rendered. In addition, the electronic device can save computing resources of the electronic device by reducing the computing resources allocated to the virtual scene image included in the image to be rendered.

[0045] In conjunction with the first aspect, in one possible design, the second rendering strategy includes increasing rendering parameters of a virtual scene image included in the image to be rendered. Furthermore, the virtual scene image included in the image to be rendered may be enlarged. Furthermore, the virtual scene image included in the image to be rendered may be adjusted toward the center of the image to be rendered ...

[0046] In this solution, a possible setting method for the second rendering strategy is provided. By adjusting the image to be rendered through the second rendering strategy, the electronic device can enhance the visual effect of the virtual image in the image to be rendered and weaken the visual effect of the real image in the image to be rendered. This meets the differentiated needs of users for the display content of electronic devices and is beneficial to the user's experience when looking at the virtual image in the image to be rendered. In addition, the electronic device can save computing resources of the electronic device by reducing the computing resources allocated to the real image included in the image to be rendered.

[0047] In a second aspect, embodiments of the present application provide an electronic device. The electronic device includes a memory and one or more processors, the memory and the processors being coupled; the memory is configured to store computer program code, the computer program code comprising computer instructions, and when the processor executes the computer instructions, the electronic device executes the method described in the first aspect and any possible design thereof.

[0048] In a third aspect, embodiments of the present application provide a chip system. The chip system includes one or more interface circuits and one or more processors. The interface circuits and the processors are interconnected via wiring. The interface circuits are configured to receive signals from a memory of an electronic device and send signals to the processors. The signals include computer instructions stored in the memory. When the processors execute the computer instructions, the electronic device performs the method described in the first aspect and any possible design thereof.

[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium comprising computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method described in the first aspect and any possible design thereof.

[0050] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the method described in the first aspect and any possible design thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] FIG1 is a schematic diagram of a mixed reality image;

[0052] FIG2 is a schematic diagram of a movie viewing image displayed by an electronic device based on foveated rendering technology;

[0053] FIG3 is a schematic diagram of a mixed reality image displayed by an electronic device based on foveated rendering technology;

[0054] FIG4 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0055] FIG5 is a schematic diagram of a setting interface provided in an embodiment of the present application;

[0056] FIG6 is a schematic diagram of adjusting a picture to be rendered provided by an embodiment of the present application;

[0057] FIG7 is a schematic diagram of another method for adjusting a rendering image according to an embodiment of the present application;

[0058] FIG8 is a schematic diagram of adjusting a first image according to an embodiment of the present application;

[0059] FIG9 is a schematic diagram of another method for adjusting the first image according to an embodiment of the present application;

[0060] FIG10 is a schematic diagram of a module structure of an electronic device provided in an embodiment of the present application;

[0061] FIG11 is a schematic structural diagram of a display device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0063] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

[0064] MR devices can sense their surroundings through cameras to acquire corresponding environmental images. By overlaying the acquired environmental images with virtual images corresponding to virtual content, MR images can be generated and displayed. For example, referring to FIG1 , MR image 101 includes virtual images (e.g., images 102 and 103) and real images (e.g., images 104 and 105).

[0065] The virtual content is pre-stored in the MR device, or is a digital signal corresponding to a virtual image acquired by the MR device from an input source. Virtual images can include at least one of virtual human images, virtual plant and animal images, and virtual object images. For example, referring again to Figure 1 , MR image 101 includes a virtual human image 102 and a virtual display screen image 103. The MR device can play media within the display range of virtual display screen image 103, so that when a user wearing the MR device and looking at virtual display screen image 103, they can experience a movie-watching experience.

[0066] Because users have limited field of view and attention span, when using an MR device, they may focus on one part of the MR image while ignoring another. For example, when watching a movie using an MR device, a user may pay more attention to the movie image in the MR image and ignore other parts of the image. Alternatively, when walking while wearing an MR device, a user may pay more attention to the environmental image in the MR image that corresponds to the environment surrounding the MR device and ignore the virtual image corresponding to the virtual content.

[0067] However, when rendering MR images, MR devices typically use the same computing resources to render various regions within the same MR image frame. In this case, since users may ignore certain parts of the MR image, using the same computing resources to render both the ignored and the focused regions of the MR image would result in a waste of computing resources.

[0068] In one technique, the central region of the MR image that falls within the user's field of view (also referred to as the gaze zone in embodiments of this application) can be determined, and the MR image can be rendered based on the user's gaze zone and foveated rendering technology. Foveated rendering technology renders the MR image from the area corresponding to the user's gaze zone to the area corresponding to the area farther from the user's gaze zone in descending resolution, which can conserve computing resources of the MR device.

[0069] For example, referring to FIG2 , MR image 200 includes regions 201, 202, and 203. Region 201 represents the user's gaze area. The MR device can render region 201 at 100% resolution, region 202 at 70% resolution, and region 203 at 40% resolution, thereby conserving computing resources of the MR device.

[0070] However, rendering MR images based on the user's gaze area and gaze point rendering technology cannot meet the user's demand for differentiated display content, which is not conducive to user experience.

[0071] For example, when a user is watching a movie using an MR device, if the MR image is rendered based on the user's gaze area and gaze point rendering technology, the MR image may be as shown in Figure 2. MR image 200 includes a movie frame 204, and the clarity of each area in movie frame 204 varies, affecting the user's viewing experience using the MR device.

[0072] For another example, when a user is walking while wearing an MR device, if the MR image is rendered based on the user's gaze area and gaze point rendering technology, the MR image may be as shown in Figure 3, including an environmental screen 301 corresponding to the environment surrounding the MR device and a virtual screen 302 corresponding to virtual content. The clarity gradually decreases from the image area corresponding to the user's gaze area 303 to the image area corresponding to the gaze area 303 away from the user. However, when a user is walking while wearing an MR device, they will pay more attention to the environmental image corresponding to the environment surrounding the MR device in the MR image. Therefore, the MR device rendering of the MR image based on the user's gaze area and gaze point rendering technology cannot provide the user with a clear and complete environmental image, affecting the user's experience when walking while wearing the MR device.

[0073] Based on the above problems, an embodiment of the present application provides a display method, which can be applied to an electronic device. The electronic device can display a first screen including a real scene image and a virtual scene image. The real scene image is obtained based on an environmental image corresponding to the environment surrounding the electronic device, and the virtual scene image includes a virtual image corresponding to virtual content. The electronic device can obtain the user's posture information and display a second screen based on the user's posture information. The second screen is obtained based on the adjustment of the first screen. The image quality of the virtual scene image in the first screen is better than the image quality of the virtual scene image in the second screen, or the image quality of the real scene image in the first screen is better than the image quality of the real scene image in the second screen.

[0074] It should be noted that the user's posture information in the embodiment of the present application may include the user's gaze point information on the first screen, the user's head rotation information, and the user's movement information. The user's posture information may reflect the content that the user is paying attention to in the first screen.

[0075] If the content the user is focusing on in the first screen is a real-world image, the electronic device can adjust the first screen to enhance the visual effect of the real-world image in the first screen and weaken the visual effect of the virtual-world image in the first screen. This meets the user's differentiated needs for electronic device display content and improves the user's experience when viewing the real-world image in the second screen. Furthermore, the electronic device can conserve computing resources by reducing the computing resources allocated to the virtual-world image included in the first screen.

[0076] If the content the user is focusing on in the first screen is a virtual image, the electronic device can adjust the first screen to enhance the visual effect of the virtual image in the first screen and weaken the visual effect of the real image in the first screen. This meets the user's differentiated needs for electronic device display content and improves the user's experience when viewing the virtual image in the second screen. Furthermore, the electronic device can conserve computing resources by reducing the computing resources allocated to the real image included in the first screen.

[0077] For example, the electronic devices provided in the embodiments of the present application may be wearable devices with MR functions, such as MR glasses and MR helmets. The electronic devices provided in the embodiments of the present application may also be non-wearable devices with MR functions, such as mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, notebook computers, in-vehicle devices, ultra-mobile personal computers (UMPCs), netbooks, etc.

[0078] When the electronic device provided in the embodiments of the present application is a non-wearable device with MR functionality, the electronic device can be used in conjunction with augmented reality (AR) / MR glasses. For example, the electronic device can receive an MR scene video stream, and the AR / MR glasses can display the MR scene video.

[0079] In addition, the electronic device in the embodiment of the present application may also be a personal digital assistant (PDA), an AR device, etc. The embodiment of the present application does not impose any special restrictions on the specific form of the electronic device.

[0080] 4 is a schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and the like.

[0081] It should be understood that the structure illustrated in the embodiments of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0082] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0083] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0084] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0085] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0086] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0087] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0088] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0089] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0090] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0091] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0092] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0093] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with a network and other devices through wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0094] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0095] Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N display screens 194, where N is a positive integer greater than one.

[0096] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format such as RGB, YUV, etc.

[0097] In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1. For example, the cameras 193 may include a first camera for capturing images of the real world environment and a second camera for capturing images of the user's eyes. The electronic device may obtain an MR image based on the images of the environment captured by the first camera and the virtual images corresponding to the virtual content. The electronic device may determine the location of the user's gaze point in the MR image based on the images of the user's eyes captured by the second camera.

[0098] In some embodiments, camera 193 may include a second camera for capturing images of the user's eyes, but may not include a first camera for capturing images of the real-world environment. In this case, the electronic device may be externally connected to the first camera. The electronic device can capture images of the real-world environment based on the external first camera.

[0099] In other embodiments, camera 193 may include a first camera for capturing images of the real-world environment, but may not include a second camera for capturing images of the user's eyes. In this case, the electronic device may be externally connected to the second camera. The electronic device may obtain images of the user's eyes based on the external second camera. Based on the images of the user's eyes, the electronic device may determine the location of the user's gaze point in the MR image.

[0100] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0101] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0102] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, eye movement recognition, speech recognition, and text comprehension.

[0103] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, music, video and other files are stored in an external memory card. For another example, the rendering strategy of the application screen corresponding to each application in the electronic device is stored in the external memory card. When the electronic device renders the application screen corresponding to the target application, the corresponding rendering strategy can be retrieved from the external memory card, and the corresponding application screen can be rendered based on the retrieved rendering strategy.

[0104] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, a rendering strategy for the application screen corresponding to each application in the electronic device, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0105] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0106] The sensor module 180 may include a pressure sensor, a gyro sensor, an accelerometer sensor, an air pressure sensor, a magnetic sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0107] An accelerometer sensor can detect the magnitude of an electronic device's acceleration in all directions (generally three axes). It can also detect the magnitude and direction of gravity when the electronic device is stationary. It can also be used to identify the posture of the user wearing the electronic device, for example, when walking, jumping, squatting, standing up, and other actions.

[0108] For example, the technical solutions involved in the following embodiments can all be implemented in an electronic device having the above hardware architecture. The display method provided by the embodiments of the present application is described in detail below with reference to the accompanying drawings and application scenarios.

[0109] In some embodiments, the electronic device may acquire an image of the environment surrounding the electronic device. Based on the acquired image of the environment and virtual images corresponding to virtual content (e.g., virtual characters, virtual plants and animals, virtual objects), the electronic device may obtain a to-be-rendered image. The electronic device may render the to-be-rendered image to display an MR image corresponding to the to-be-rendered image.

[0110] The MR image displayed by the electronic device includes a real scene image and a virtual scene image. The real scene image includes an image of the environment around the electronic device, and the virtual scene image includes a virtual image corresponding to the virtual content.

[0111] In some embodiments, an electronic device can capture an image of the environment surrounding the electronic device based on a first camera. The electronic device may include the first camera. That is, the first camera is part of the electronic device. For example, the first camera may be embedded in the electronic device. The electronic device may also not include the first camera. That is, the first camera may exist independently of the electronic device. For example, the electronic device may be connected to the first camera via a USB port.

[0112] In some embodiments, the electronic device has a custom rendering function. The custom rendering function means that the electronic device supports the user to set the rendering strategy of the electronic device. Among them, the rendering strategy is used to adjust the screen to be rendered. Based on the custom rendering function of the electronic device, the user can set the rendering strategy for the application screen corresponding to each application in the electronic device. The application screen rendered by the electronic device based on the rendering strategy set by the user is more in line with the user's viewing habits, which is beneficial to the user experience.

[0113] In some embodiments, the electronic device may display a settings interface for setting a rendering strategy for at least one application screen corresponding to a target application. The target application may be any application included in the electronic device. The user may input a first trigger operation in the settings interface to indicate that the rendering strategy for at least one application screen corresponding to the target application is set to a target rendering strategy. In response to the first trigger operation, the electronic device may set the rendering strategy for at least one application screen corresponding to the target application to the target rendering strategy.

[0114] Afterwards, if the electronic device receives a second trigger operation input by the user for instructing to display a third screen, for example, the user selects a link for jumping to display the third screen, the electronic device can adjust the third screen based on the target rendering strategy in response to the second trigger operation. The electronic device renders the adjusted third screen and can display the first screen. The first screen is obtained based on the adjustment of the third screen, and the third screen is included in at least one application screen corresponding to the target application.

[0115] Exemplarily, referring to FIG5 , an application 500 may be installed in the electronic device. The application 500 is used to set a rendering strategy for the application screen corresponding to each application in the electronic device. When the electronic device receives a selection operation of the application 500 input by the user (for example, the user clicks on the icon corresponding to the application 500), the setting interface 501 may be displayed. The setting interface 501 includes at least one setting control (for example, control 502). Any setting control corresponds to an application in the electronic device. The setting control is used to set a rendering strategy for the application screen of the corresponding application.

[0116] The user can select the control 502 corresponding to the target application, and the electronic device can display a setting menu 503 in response to the user's selection operation on the control 502. The setting menu 503 includes multiple rendering strategy items (for example, item 504, item 505, item 506). Each rendering strategy item corresponds to a rendering strategy. The user can select item 505, and the electronic device can set the rendering strategy for the application screen corresponding to the target application (such as the gallery) to the rendering strategy corresponding to item 505 in response to the user's selection operation on item 505.

[0117] Afterwards, when the electronic device renders the application screen corresponding to the target application, for example, when the electronic device renders the third screen corresponding to the gallery, the electronic device can adjust the third screen based on the rendering strategy corresponding to item 505, and render the adjusted third screen to display the first screen.

[0118] It should be noted that since the user's selection operation on item 505 is used to instruct that the rendering strategy for the application screen corresponding to the target application be set to the rendering strategy corresponding to item 505, the first trigger operation in the embodiment of the present application is used to instruct that the rendering strategy for at least one application screen corresponding to the target application be set to the target rendering strategy. Therefore, the first trigger operation in the embodiment of the present application includes the user's selection operation on item 505 in the above example.

[0119] In some embodiments, the rendering strategy includes at least a first rendering strategy and a second rendering strategy. The first rendering strategy is used to adjust the virtual image included in the image to be rendered to reduce the computing resources allocated to the virtual image included in the image to be rendered. The second rendering strategy is used to adjust the real image included in the image to be rendered to reduce the computing resources allocated to the real image included in the image to be rendered. The description of the real image and the virtual image can be referred to the description in the previous embodiment and is not repeated here.

[0120] In some embodiments, the first rendering strategy includes reducing rendering parameters of the virtual scene image included in the to-be-rendered image, wherein the rendering parameters may include at least one of a clarity parameter, a resolution parameter, a color saturation parameter, a brightness parameter, a color temperature parameter, and a contrast parameter.

[0121] When the electronic device renders the image to be rendered, if only the rendering parameters of the virtual image included in the image to be rendered are reduced, the computing resources allocated by the electronic device for rendering the virtual image included in the image to be rendered will also be reduced. Therefore, rendering the image to be rendered based on the first rendering strategy can save computing resources.

[0122] In some embodiments, the first rendering strategy includes increasing rendering parameters for a real-scene image included in the to-be-rendered image. When the electronic device renders the to-be-rendered image, if only the rendering parameters for the real-scene image included in the to-be-rendered image are increased, the computing resources allocated by the electronic device to render the real-scene image included in the to-be-rendered image will also increase, and the image quality of the real-scene image in the rendered image will also be improved, thereby enhancing the user experience when viewing the real-scene image included in the to-be-rendered image.

[0123] In some embodiments, the first rendering strategy also includes reducing the area of ​​the virtual image included in the image to be rendered. For example, referring to Figure 6 , the area of ​​virtual image 601 included in image to be rendered 600 can be reduced to obtain an adjusted image to be rendered 602. By reducing the area of ​​the virtual image in the image to be rendered, the electronic device can reduce the area of ​​the real image obscured by the virtual image in the image to be rendered, thereby improving the user experience when viewing the real image included in the image to be rendered.

[0124] In some embodiments, the first rendering strategy further includes adjusting the virtual image included in the image to be rendered toward the edge of the image to be rendered. For example, referring again to FIG6 , virtual image 601 included in image 600 to be rendered can be adjusted from position A to position B. By adjusting the position of the virtual image included in the image to be rendered, the electronic device can move the virtual image away from the center of the user's field of view, thereby improving the user's viewing experience of the real image included in the image to be rendered.

[0125] In some embodiments, the first rendering strategy includes performing a first adjustment on a capture parameter of the first camera. The description of the first camera can be found in the description of the aforementioned embodiments and is not repeated here. The first adjustment includes increasing at least one of a capture frame rate and a shutter speed of the first camera.

[0126] When the first camera captures images of the environment surrounding the electronic device, if only the first camera's frame rate and / or shutter speed are increased, the time interval between two adjacent frames of the environment image captured by the first camera will be shortened. That is, within the same timeframe, the first camera can capture a greater number of environment images. Since the real-world image in the image to be rendered is derived from the environment captured by the first camera, the greater the number of environment images captured by the first camera, the more image resources available for generating the real-world image, and the better the image quality of the resulting real-world image, thereby enhancing the user's viewing experience.

[0127] In some embodiments, the first rendering strategy further includes performing optimization processing on the environment image captured by the first camera. This optimization processing is processing for improving the image quality of the environment image captured by the first camera. For example, this optimization processing may include noise reduction, ghosting removal, dynamic range enhancement, and hole filling. The specific content and implementation methods of this optimization processing are not limited in this embodiment of the present application.

[0128] The higher the quality of the environment image, the higher the quality of the real-scene image included in the to-be-rendered image. Therefore, optimizing the environment image can improve its quality, which in turn improves the quality of the real-scene image included in the to-be-rendered image, thereby enhancing the user experience when viewing the real-scene image.

[0129] In some embodiments, the second rendering strategy includes reducing the rendering parameters of the real-scene image included in the to-be-rendered image. When the electronic device renders the to-be-rendered image, if only the rendering parameters of the real-scene image included in the to-be-rendered image are reduced, the computing resources allocated by the electronic device to render the real-scene image included in the to-be-rendered image will also be reduced. Therefore, rendering the to-be-rendered image based on the second rendering strategy can save computing resources.

[0130] In some embodiments, the second rendering strategy includes increasing rendering parameters for a virtual image included in the image to be rendered. When the electronic device renders the image to be rendered, if only the rendering parameters for the virtual image included in the image to be rendered are increased, the computing resources allocated by the electronic device to render the virtual image included in the image to be rendered will also increase, and the image quality of the virtual image in the rendered image will also be improved, thereby enhancing the user experience when viewing the virtual image included in the image to be rendered.

[0131] In some embodiments, the second rendering strategy further includes amplifying a virtual image included in the image to be rendered. For example, referring to FIG7 , a virtual image 701 included in an image to be rendered 700 can be amplified to obtain an adjusted image to be rendered 702. By amplifying the virtual image in the image to be rendered, the electronic device can enhance the user's viewing experience of the virtual image included in the image to be rendered.

[0132] In some embodiments, the second rendering strategy further includes adjusting the virtual image included in the image to be rendered closer to the center of the image to be rendered. For example, referring again to FIG. 7 , the virtual image 701 included in the image to be rendered 700 can be adjusted from position A to position B. By adjusting the virtual image included in the image to be rendered closer to the center of the image to be rendered, the electronic device can enhance the user's viewing experience of the virtual image included in the image to be rendered.

[0133] In some embodiments, the second rendering strategy includes performing a second adjustment on the shooting parameters of the first camera. The description of the first camera can be found in the description of the aforementioned embodiments and is not repeated here. The second adjustment includes reducing at least one of the shooting frame rate and shutter speed of the first camera.

[0134] When the first camera acquires an image of the environment around the electronic device, if only the shooting frame rate and / or shutter rate of the first camera is reduced, the computing resources required by the first camera to capture the image of the environment around the electronic device will also be reduced, which can save the computing resources of the first camera.

[0135] In some embodiments, after the electronic device displays the target screen, it can obtain the user's posture information. Based on the obtained user's posture information, the electronic device can determine whether to switch the rendering strategy. If the electronic device determines to switch the rendering strategy, the electronic device can adjust the target screen based on the switched rendering strategy to display the adjusted target screen. The target screen can be any screen corresponding to any application. For example, the target screen can be the desktop screen corresponding to a desktop application. For another example, the target screen can be the first screen corresponding to the target application, etc., which is not limited in the embodiments of the present application.

[0136] For ease of description, the following takes the first screen corresponding to the first application as an example to explain in detail how the electronic device determines whether to switch the rendering strategy based on the user's posture information, and how the electronic device adjusts the first screen based on the switched rendering strategy.

[0137] In some embodiments, the user's posture information includes information about the user's gaze point on the first screen. The user's gaze point information on the first screen is used to represent the location of the user's gaze point on the first screen. The following describes in detail a method for an electronic device to automatically switch rendering strategies based on the user's gaze point information on the first screen.

[0138] In some embodiments, the electronic device may include a second camera, or the electronic device may be externally connected to a second camera. The second camera is used to obtain an eye image of the user. The electronic device may obtain eye feature point information included in the eye image of the user based on a second target model. The second target model is used to analyze the input eye image to output eye feature point information corresponding to the eye image. Based on the obtained eye feature point information and a target algorithm (e.g., an eye movement algorithm), the electronic device may determine the user's gaze point information on the first screen.

[0139] It should be noted that the embodiments of the present application do not limit the manner in which the electronic device obtains the user's gaze point information on the first screen. The electronic device can determine the user's gaze point information on the first screen based on the user's eye feature point information in the above embodiments. The electronic device can also determine the user's gaze point information on the first screen based on infrared detection technology, a detector for identifying the user's eye movement (e.g., a myoelectric current detector), or other methods.

[0140] In some embodiments, the electronic device can determine the position of the user's gaze point on the first screen based on the user's gaze point information. If the rendering strategy of the electronic device before obtaining the user's gaze point information is the first rendering strategy, then after the electronic device obtains the user's gaze point information, if the electronic device determines that the user's gaze point is located at the position of the real-life image included in the first screen, the electronic device determines not to switch the rendering strategy. That is, the electronic device does not adjust the first screen. Among them, the description of the first rendering strategy can be found in the description of the aforementioned embodiment, and will not be repeated here.

[0141] In some embodiments, if the electronic device's rendering strategy before obtaining the user's gaze point information is the second rendering strategy, then after obtaining the user's gaze point information, if the electronic device determines that the user's gaze point is located at the location of the virtual scene image included in the first screen, the electronic device determines not to switch the rendering strategy. In other words, the electronic device does not adjust the first screen.

[0142] In some embodiments, if the electronic device's rendering strategy before obtaining the user's gaze point information is not the first rendering strategy, then after obtaining the user's gaze point information, if the electronic device determines that the user's gaze point is located at the location of the real-scene image included in the first screen, it indicates that the user is paying more attention to the real-scene image included in the first screen. The electronic device can switch the rendering strategy to the first rendering strategy and adjust the first screen based on the first rendering strategy to display the second screen.

[0143] In some embodiments, since the first rendering strategy includes reducing the rendering parameters of the virtual image included in the to-be-rendered screen, that is, the first rendering strategy includes reducing the rendering parameters of the virtual image included in the first screen. Therefore, referring to FIG8 , the image quality of the virtual image 802 included in the first screen 801 is better than the image quality of the virtual image 811 included in the second screen 810. That is, the electronic device can highlight the visual effect of the real image included in the first screen by weakening the visual effect of the virtual image included in the first screen. The electronic device can render the adjusted first screen to display the second screen. Compared with the user looking at the first screen, the user has a better experience when looking at the real image included in the second screen.

[0144] In some embodiments, since the first rendering strategy includes increasing the rendering parameters of the real-scene image included in the to-be-rendered screen, that is, the first rendering strategy includes increasing the rendering parameters of the real-scene image included in the first screen. Therefore, referring to FIG8 , the image quality of the real-scene image 812 included in the second screen 810 is better than the image quality of the real-scene image 803 included in the first screen 801. That is, the electronic device can enhance the visual effect of the real-scene image included in the first screen. The electronic device can render the adjusted first screen to display the second screen. Compared with the user looking at the first screen, the user has a better experience when looking at the real-scene image included in the second screen.

[0145] In some embodiments, the first rendering strategy includes reducing the size of the virtual image included in the to-be-rendered screen. That is, the first rendering strategy includes reducing the size of the virtual image included in the first screen. Therefore, referring to FIG8 , the area of ​​the virtual image 811 included in the second screen 810 is smaller than the area of ​​the virtual image 802 included in the first screen 801. That is, the electronic device can weaken the visual effect of the virtual image included in the first screen. The electronic device can render the adjusted first screen to display the second screen. Compared to looking at the first screen, the user's experience is better when looking at the real image included in the second screen.

[0146] In some embodiments, the first rendering strategy includes adjusting the virtual image included in the to-be-rendered screen toward the edge of the to-be-rendered screen. That is, the first rendering strategy includes adjusting the virtual image included in the first screen toward the edge of the first screen. Therefore, referring to FIG8 , the distance between the virtual image 811 included in the second screen 810 and the edge of the second screen 810 is less than the distance between the virtual image 802 in the first screen 801 and the edge of the first screen 801. That is, the electronic device can weaken the visual effect of the virtual image included in the first screen by adjusting the virtual image included in the first screen to a position close to the edge of the first screen. The electronic device can render the adjusted first screen to display the second screen. Compared with the user looking at the first screen, the user has a better experience when looking at the real image included in the second screen.

[0147] In some embodiments, because the first rendering strategy includes increasing the capture frame rate and / or shutter speed of the first camera, the first camera can capture a greater number of environmental images. Because the first camera captures a greater number of environmental images, more image resources are available for generating real-world images, and the quality of the generated real-world images is improved. Compared to viewing the first screen, the user experiences a better experience when viewing the real-world images included in the second screen.

[0148] In some embodiments, because the first rendering strategy includes optimizing the environment image captured by the first camera, the image quality of the real scene image included in the to-be-rendered screen based on the optimized environment image is also higher. This provides a better user experience when viewing the real scene image included in the second screen compared to when viewing the first screen.

[0149] In some embodiments, if the electronic device's rendering strategy before obtaining the user's gaze point information is not the second rendering strategy, then after obtaining the user's gaze point information, if the electronic device determines that the user's gaze point is located at the location of the virtual scene image included in the first screen, it indicates that the user is paying more attention to the virtual scene image included in the first screen. The electronic device may switch the rendering strategy to the second rendering strategy and adjust the first screen based on the second rendering strategy to display the second screen.

[0150] In some embodiments, since the second rendering strategy includes reducing the rendering parameters of the real scene image included in the to-be-rendered screen, that is, the second rendering strategy includes reducing the rendering parameters of the real scene image included in the first screen. Therefore, referring to FIG9 , the image quality of the real scene image 902 included in the first screen 901 is better than the image quality of the real scene image 911 included in the second screen 910. That is, the electronic device can highlight the visual effect of the virtual scene image included in the adjusted first screen by weakening the visual effect of the real scene image included in the first screen. The electronic device can render the adjusted first screen to display the second screen. Compared with the user looking at the first screen, the user has a better experience when looking at the virtual scene image included in the second screen.

[0151] In some embodiments, the second rendering strategy includes increasing the rendering parameters of the virtual image included in the to-be-rendered screen. That is, the second rendering strategy includes increasing the rendering parameters of the virtual image included in the first screen. Therefore, referring to FIG9 , the image quality of the virtual image 912 included in the second screen 910 is superior to the image quality of the virtual image 903 included in the first screen 901. That is, the electronic device can enhance the visual effect of the virtual image included in the first screen. The electronic device can render the adjusted first screen to display the second screen. Compared to the user looking at the first screen, the user's experience is better when looking at the virtual image included in the second screen.

[0152] In some embodiments, the second rendering strategy includes amplifying the virtual image included in the to-be-rendered screen. That is, the second rendering strategy includes amplifying the virtual image included in the first screen. Therefore, referring to FIG9 , the area of ​​the virtual image 912 included in the second screen 910 is larger than the area of ​​the virtual image 903 included in the first screen 901. That is, the electronic device can enhance the visual effect of the virtual image included in the first screen. The electronic device can render the adjusted first screen to display the second screen. Compared to the user looking at the first screen, the user's experience is better when looking at the virtual image included in the second screen.

[0153] In some embodiments, the second rendering strategy includes adjusting the virtual image included in the to-be-rendered screen toward the center of the to-be-rendered screen. That is, the second rendering strategy includes adjusting the virtual image included in the first screen toward the center of the first screen. Therefore, referring to FIG9 , the distance between the virtual image 903 included in the first screen 901 and the edge of the first screen 901 is less than the distance between the virtual image 912 in the second screen 910 and the edge of the second screen 910. That is, the electronic device can enhance the visual effect of the virtual image included in the first screen by adjusting the virtual image included in the first screen to be close to the center of the first screen. The electronic device can render the adjusted first screen to display the second screen. Compared with the user looking at the first screen, the user has a better experience when looking at the virtual image included in the second screen.

[0154] In some embodiments, since the second rendering strategy includes reducing the shooting frame rate and / or shutter rate of the first camera, the computing resources required by the first camera when capturing images of the environment around the electronic device will also be reduced, thereby saving the computing resources of the first camera.

[0155] In some embodiments, the user's posture information includes the user's head rotation information. The user's head rotation information includes the user's head rotation angle in the horizontal direction and the user's head rotation angle in a direction perpendicular to the horizontal direction (also referred to as the vertical direction in embodiments of this application). The following describes in detail a method for an electronic device to automatically switch the electronic device's rendering strategy based on the user's head rotation information.

[0156] In some embodiments, the electronic device may obtain the user's head rotation information based on a sensor for detecting the user's head rotation angle (e.g., an accelerometer sensor). The electronic device may also determine the user's head rotation information based on an image of the user's head captured by a third camera. The embodiments of the present application do not limit the method by which the electronic device obtains the user's head rotation information.

[0157] In some scenarios, users may turn their heads to obtain a real-world image corresponding to an image of the environment in the real world. For example, a user may look down at a physical object such as a mobile phone, tablet, or paper document. A user may look up at the sky. A user may walk with their head down. A user may turn their head left and right to view the surrounding environment. The embodiments of the present application do not restrict the scenarios in which a user obtains an image of an environment corresponding to the real world.

[0158] In some embodiments, a first preset condition can be set based on a scenario in which the user obtains a real-life image corresponding to an environmental image in the real world. After the electronic device displays the first screen, if the electronic device determines that the user's head rotation information meets the first preset condition, it means that the user is paying more attention to the real-life image corresponding to the environmental image in the real world. That is, the user will pay more attention to the real-life image in the first screen. Therefore, the rendering strategy of the electronic device can be determined as the first rendering strategy. Based on the first rendering strategy, the electronic device can adjust the first screen to display the second screen. Among them, the description of the first rendering strategy can be found in the description in the aforementioned embodiment, and will not be repeated here.

[0159] In some embodiments, the first preset condition includes the user's head rotating at an angle greater than a first threshold in the horizontal direction, and / or the user's head rotating at an angle greater than a second threshold in the vertical direction. The first threshold and the second threshold may be the same or different. For example, the first threshold and the second threshold are both 45°. For another example, the first threshold is 60° and the second threshold is 45°. The embodiments of the present application do not limit the specific values ​​of the first threshold and the second threshold.

[0160] In some embodiments, a second preset condition can be set based on the scenario in which the user obtains a real-scene image corresponding to an environmental image in the real world. After the electronic device displays the first screen, if the electronic device determines that the user's head rotation information meets the second preset condition, it means that the user is more concerned about the virtual image corresponding to the virtual content. That is, the user will pay more attention to the virtual image in the first screen. Therefore, the rendering strategy of the electronic device can be determined as the second rendering strategy. Based on the second rendering strategy, the electronic device can adjust the first screen to display the second screen. Among them, the description of the second rendering strategy can be found in the description in the aforementioned embodiment, and will not be repeated here.

[0161] In some embodiments, the second preset condition includes that the rotation angle of the user's head in the horizontal direction is less than or equal to the first threshold, and the rotation angle of the user's head in the vertical direction is less than or equal to the second threshold.

[0162] In some embodiments, the user's posture information includes the user's movement information. The user's movement information is used to represent the user's head movement trajectory. When a user walks, stands up, jumps, sits down, squats, or performs other actions, their head moves, thereby generating a head movement trajectory. The following describes in detail a method for an electronic device to automatically switch its rendering strategy based on the user's rotation information.

[0163] In some embodiments, the electronic device may obtain the user's movement information based on a sensor (e.g., an accelerometer sensor) for detecting the movement trajectory of the user's head. The electronic device may also determine the user's movement information based on an image of the user's head captured by a third camera. The embodiments of the present application do not limit the method by which the electronic device obtains the user's head rotation information.

[0164] In some embodiments, the electronic device can determine the distance the user's head moves based on the user's movement information. If the electronic device determines that the distance the user's head moves is greater than a preset distance, it means that the user is walking, standing up, jumping, sitting down, squatting, and other actions. When the user performs these actions, he or she will pay more attention to the real-scene image corresponding to the environmental image in the real world. That is, the user will pay more attention to the real-scene image in the first screen. Therefore, the rendering strategy of the electronic device can be determined as the first rendering strategy. Based on the first rendering strategy, the electronic device can adjust the first screen to display the second screen. Among them, the description of the first rendering strategy can be found in the description in the aforementioned embodiment, which will not be repeated here.

[0165] If the electronic device determines that the distance the user's head has moved is less than or equal to the preset distance, the electronic device may determine the rendering strategy of the electronic device to be the second rendering strategy. Based on the second rendering strategy, the electronic device may adjust the first screen to display the second screen. For a description of the second rendering strategy, please refer to the description in the aforementioned embodiment and will not be repeated here.

[0166] It should be noted that those skilled in the art can set the value corresponding to the preset distance according to the actual scenario, for example, setting the preset distance to 0.4 meters. The embodiment of the present application does not limit the value of the preset distance.

[0167] In an embodiment of the present application, the electronic device can determine whether to switch the rendering strategy of the electronic device based on the user's posture information.

[0168] If the electronic device switches the rendering strategy to the first rendering strategy, the electronic device adjusts the first screen based on the first rendering strategy, which can enhance the visual effect of the real scene image in the first screen and weaken the visual effect of the virtual scene image in the first screen. This meets the differentiated needs of users for the display content of the electronic device and is beneficial to the user's experience when viewing the real scene image in the second screen. In addition, the electronic device can save computing resources of the electronic device by reducing the resources allocated to the virtual scene image included in the first screen.

[0169] If the electronic device switches the rendering strategy to the second rendering strategy, the electronic device adjusts the first screen based on the second rendering strategy, which can enhance the visual effect of the virtual image in the first screen and weaken the visual effect of the real image in the first screen. This meets the differentiated needs of users for the display content of the electronic device and is beneficial to the user's experience when viewing the virtual image in the second screen. In addition, the electronic device can save computing resources of the electronic device by reducing the resources allocated to the real image included in the first screen.

[0170] In some schemes, multiple embodiments of the present application can be combined and the combined scheme can be implemented. Optionally, some operations in the process of each method embodiment are optionally combined, and / or the order of some operations is optionally changed. In addition, the execution order between the steps of each process is only exemplary and does not constitute a limitation on the execution order between the steps. There can also be other execution orders between the steps. It is not intended to indicate that the execution order is the only order in which these operations can be performed. Ordinary technicians in this field will think of many ways to reorder the operations described herein. In addition, it should be noted that the process details involved in a certain embodiment of this article are also applicable to other embodiments in a similar manner, or different embodiments can be used in combination.

[0171] Furthermore, some steps in the method embodiments may be equivalently replaced with other possible steps. Alternatively, some steps in the method embodiments may be optional and may be deleted in certain usage scenarios. Alternatively, other possible steps may be added to the method embodiments.

[0172] Furthermore, the various method embodiments may be implemented separately or in combination.

[0173] It is understandable that in order to implement the above functions, the electronic device includes hardware and / or software modules corresponding to the execution of each function. In combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in combination with the embodiments, but such implementation should not be considered to be beyond the scope of this application.

[0174] This embodiment can divide the electronic device into functional modules based on the above-mentioned method example. For example, each functional module can be divided according to its function, or two or more functions can be integrated into a single processing module. The above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that the module division in this embodiment is illustrative and only represents a logical functional division. In actual implementation, other division methods may be used.

[0175] Exemplarily, referring to FIG. 10 , the electronic device provided in an embodiment of the present application includes a perception module 1001 , a decision module 1002 , and an adjustment module 1003 .

[0176] The perception module includes a posture recognition module and a scene recognition module. The scene recognition module is used to identify the scenario in which the user is using the electronic device. For example, the user is using the audio and video application on the electronic device, or the user is using the camera application on the electronic device. The posture recognition module is used to identify the user's posture and thus obtain the user's posture information. For example, the posture recognition module can identify the user's gaze point information, head rotation information, and movement information.

[0177] The decision module is used to determine the rendering strategy of the electronic device based on the scenario in which the user uses the electronic device or the user's posture information obtained by the perception module. For example, the decision module can determine the application used by the user based on the scenario in which the user uses the electronic device obtained by the perception module, and determine the rendering strategy for the application screen corresponding to the application. For another example, the decision module can determine the content that the user is paying attention to in the displayed screen based on at least one of the user's gaze point information, head rotation information, and movement information obtained by the perception module. Based on the content that the user is paying attention to in the displayed screen, the decision module can determine the rendering strategy corresponding to the displayed screen.

[0178] The adjustment module is used to adjust the image to be rendered based on the rendering strategy determined by the decision module, and render the adjusted image to be rendered on the display panel (or display screen).

[0179] For example, before the electronic device starts the target application and displays the third screen corresponding to the target application, the adjustment module can adjust the third screen based on the rendering strategy for the target application determined by the decision module to obtain the first screen. Thereafter, the adjustment module can render the first screen on the display panel (or display screen) of the electronic device.

[0180] For another example, after the electronic device displays a first image, if the decision module determines based on the user's posture information that the electronic device's rendering strategy has switched, the adjustment module adjusts the first image based on the switched rendering strategy determined by the decision module to obtain a second image. The adjustment module can then render the second image on the display panel (or screen) of the electronic device.

[0181] In some embodiments, the decision module may further adjust the camera's shooting parameters based on the rendering strategy. The camera is used to capture a real-world image corresponding to the environment image included in the image to be rendered. The camera may be included in the electronic device or may be externally connected to the electronic device. The decision module may increase the camera's shooting parameters to improve the quality of the real-world image captured by the electronic device. The decision module may also reduce the camera's shooting parameters to conserve computing resources used by the camera to capture the environment image.

[0182] In some embodiments, the decision module may further perform optimization processing on the environment image captured by the camera based on the rendering strategy, such as noise reduction, ghost removal, dynamic range enhancement, hole filling, etc. The decision module may improve the quality of the real scene image corresponding to the environment image by optimizing the environment image captured by the camera.

[0183] An embodiment of the present application further provides an electronic device, as shown in FIG11 , which may be a display device. The display device includes one or more processors 1101 , a memory 1102 , and a communication interface 1103 .

[0184] The memory 1102 and the communication interface 1103 are coupled to the processor 1101. For example, the memory 1102, the communication interface 1103 and the processor 1101 may be coupled together via a bus 1104.

[0185] The communication interface 1103 is used to transmit data with other devices. The memory 1102 stores computer program code. The computer program code includes computer instructions. When the computer instructions are executed by the processor 1101, the electronic device executes the method of the embodiment of the present application.

[0186] Among them, the processor 1101 can be a processor or a controller, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute the various exemplary logic blocks, modules and circuits described in conjunction with the contents of this disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The processor can be used to support the electronic device to execute the method steps provided in the above embodiments.

[0187] The bus 1104 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 1104 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, FIG11 shows only one thick line, but this does not necessarily mean that there is only one bus or only one type of bus.

[0188] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0189] Optionally, there may be one or more processors in the chip system. The processor may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0190] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be provided separately from the processor, which is not limited in the embodiments of the present application. For example, the memory may be a non-transient processor, such as a read-only memory (ROM), which may be integrated with the processor on the same chip or provided on different chips. The embodiments of the present application do not specifically limit the type of memory or the configuration of the memory and the processor.

[0191] Exemplarily, the chip system may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0192] It should be understood that each step in the above method embodiment can be completed by hardware integrated logic circuits in a processor or by software instructions. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0193] An embodiment of the present application further provides a computer-readable storage medium, in which computer program code is stored. When the processor executes the computer program code, the electronic device executes the relevant method steps in the above method embodiment.

[0194] An embodiment of the present application further provides a computer program product, which, when executed on a computer, enables the computer to execute the relevant method steps in the above method embodiment.

[0195] In addition, embodiments of the present application further provide a device. Specifically, the device may be a component or module, and may include one or more processors and a memory connected to each other. The memory is configured to store a computer program. When the computer program is executed by one or more processors, the device performs the scenario configuration method described in each of the above method embodiments.

[0196] The apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods provided above, and will not be repeated here.

[0197] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, read-only compact discs (CD-ROMs), or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0198] Through the description of the above embodiments, those skilled in the art will clearly understand that for the sake of convenience and brevity, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed; that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working processes of the above-mentioned systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0199] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

Claims

1. A display method, characterized in that: Applied to electronic equipment, the method comprises: Displaying a first picture; the first picture includes a real scene image and a virtual scene image; the real scene image includes an environment image around the electronic device; the virtual scene image includes an image corresponding to the virtual content; Based on the user's posture information, a second picture is displayed; the second picture is obtained based on the adjustment of the first picture; the image quality of the virtual scene image in the first picture is better than the image quality of the virtual scene image in the second picture, or the image quality of the real scene image in the first picture is better than the image quality of the real scene image in the second picture.

2. The method according to claim 1, characterized in that The image quality of the virtual scene image is related to at least one of the following information: the area of ​​the virtual scene image, the distance between the virtual scene image and the edge of the picture where the virtual scene image is located, and the rendering parameters of the virtual scene image; The image quality of the real scene image is related to at least one of the following information: the area of ​​the real scene image, the distance between the real scene image and the edge of the picture where the real scene image is located, the rendering parameters of the real scene image, and the shooting parameters of the camera used to obtain the real scene image; The rendering parameters include at least one of a clarity parameter, a resolution parameter, a color saturation parameter, a brightness parameter, a color temperature parameter, and a contrast parameter; and the shooting parameters include at least one of a shooting frame rate and a shutter speed.

3. The method according to claim 2, characterized in that The second picture is obtained based on the adjustment of the first picture, and includes: adjusting the at least one item of information corresponding to the virtual scene image in the first picture to obtain the second picture; and / or, The at least one item of information corresponding to the real scene image in the first picture is adjusted to obtain the second picture.

4. The method according to any one of claims 1 to 3, characterized in that: When the image quality of the virtual scene image in the first picture is better than the image quality of the virtual scene image in the second picture, the area of ​​the virtual scene image in the first picture is larger than the area of ​​the virtual scene image in the second picture; When the image quality of the real scene image in the first picture is better than the image quality of the real scene image in the second picture, the area of ​​the virtual scene image in the first picture is smaller than the area of ​​the virtual scene image in the second picture.

5. The method according to any one of claims 1 to 4, characterized in that: When the image quality of the virtual scene image in the first picture is better than that of the virtual scene image in the second picture, the distance between the virtual scene image in the second picture and the edge of the second picture is smaller than the distance between the virtual scene image in the first picture and the edge of the first picture.

6. The method according to any one of claims 1 to 5, characterized in that: When the image quality of the real scene image in the first picture is better than that of the real scene image in the second picture, the distance between the virtual scene image in the first picture and the edge of the first picture is smaller than the distance between the virtual scene image in the second picture and the edge of the second picture.

7. The method according to any one of claims 1 to 6, characterized in that: When the image quality of the virtual scene image in the first picture is better than the image quality of the virtual scene image in the second picture, and the image quality of the real scene image in the second picture is better than the image quality of the real scene image in the first picture; When the image quality of the real scene image in the first picture is better than that of the real scene image in the second picture, the image quality of the virtual scene image in the second picture is better than that of the virtual scene image in the first picture.

8. The method according to any one of claims 1 to 7, characterized in that: The posture information includes the gaze point information of the user on the first screen; the gaze point information is used to represent the position of the gaze point of the user on the first screen; When the gaze point is located at the position where the real scene image in the first picture is located, the image quality of the virtual scene image in the first picture is better than the image quality of the virtual scene image in the second picture; Or, when the gaze point is located at the position where the virtual scene image in the first picture is located, the image quality of the real scene image in the first picture is better than the image quality of the real scene image in the second picture.

9. The method according to any one of claims 1 to 8, characterized in that: The posture information includes the head rotation information of the user; the head rotation information of the user includes the rotation angle of the user's head in the horizontal direction and the rotation angle of the user's head in the vertical direction; When the rotation angle of the user's head in the horizontal direction is greater than a first threshold, and / or the rotation angle of the user's head in the vertical direction is greater than a second threshold, the image quality of the virtual scene image in the first picture is better than the image quality of the virtual scene image in the second picture; When the rotation angle of the user's head in the horizontal direction is less than or equal to a first threshold, and the rotation angle of the user's head in the vertical direction is less than or equal to a second threshold, the image quality of the real-scene image in the first screen is better than the image quality of the real-scene image in the second screen.

10. The method according to any one of claims 1 to 9, characterized in that: The posture information includes movement information of the user; the movement information of the user is used to characterize the movement trajectory of the user's head; When the distance moved by the user's head is greater than a preset distance, the image quality of the virtual scene image in the first picture is better than the image quality of the virtual scene image in the second picture; When the distance moved by the user's head is less than or equal to the preset distance, the image quality of the real scene image in the first picture is better than the image quality of the real scene image in the second picture; The distance moved by the user's head is obtained based on the movement information of the user.

11. The method according to any one of claims 1 to 10, characterized in that: The target application in the electronic device corresponds to at least one application screen; The at least one application screen includes a third screen; before displaying the first screen, the method further includes: Displaying a settings interface; the settings interface is used to set a rendering strategy for the at least one application screen; the rendering strategy is used to adjust the screen to be rendered; Receiving a first trigger operation input by a user on the setting interface; the first trigger operation is used to instruct to set a rendering strategy for the at least one application screen as a target rendering strategy; In response to the first trigger operation, a rendering strategy for the at least one application screen is set as a target rendering strategy.

12. The method according to claim 11, characterized in that After setting the rendering strategy for the at least one application screen as the target rendering strategy, the method further includes: Receiving a second trigger operation input by a user; the second trigger operation is used to instruct to display the third screen; In response to the second trigger operation, adjusting the third picture based on the target rendering strategy; The first picture obtained by adjusting the third picture is displayed.

13. The method according to claim 11 or 12, characterized in that: The rendering strategy includes a first rendering strategy and a second rendering strategy; The second picture is obtained based on the adjustment of the first picture, including: the second picture is obtained based on the adjustment of the first picture by using the rendering strategy; The first rendering strategy includes reducing the rendering parameters of the virtual scene image included in the picture to be rendered; The second rendering strategy includes reducing the rendering parameters of the real scene image included in the picture to be rendered; The rendering parameters include at least one of a clarity parameter, a resolution parameter, a color saturation parameter, a brightness parameter, a color temperature parameter, and a contrast parameter.

14. The method according to claim 13, characterized in that The first rendering strategy also includes: increasing the rendering parameters of the real scene image included in the picture to be rendered; and / or, reducing the virtual scene image included in the picture to be rendered; and / or, Adjusting the virtual scene image included in the picture to be rendered toward the edge of the picture to be rendered; and / or, At least one of the shooting frame rate and shutter rate of the camera is increased; the real scene image included in the picture to be rendered is obtained based on the camera.

15. The method according to claim 13 or 14, characterized in that The second rendering strategy includes: increasing the rendering parameters of the virtual scene image included in the picture to be rendered; and / or, Enlarging the virtual scene image included in the picture to be rendered; and / or, Adjusting the virtual scene image included in the picture to be rendered toward the center of the picture to be rendered; and / or, At least one of the shooting frame rate and shutter rate of the camera is reduced; the real scene image included in the picture to be rendered is obtained based on the camera.

16. An electronic device, characterized in that: include: A memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the method as described in any one of claims 1-15.

17. A computer-readable storage medium comprising computer instructions, characterized in that: When the computer instructions are executed on an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 15.

18. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 15.

Images