Image display method, electronic equipment and readable storage medium

Abstract

The invention discloses an image display method, electronic equipment and a readable storage medium, and belongs to the field of display. The method comprises the following steps: acquiring a first type of image frames and a second type of image frames; the first type of image frames comprises an image frame of a target application program under a view angle of a first user, the second type of image frames comprises an image frame of the target application program under the view angle of the target user, and the view angle of the target user is different from the view angle of the first user; the first user and the target user are different users using the target application program; generating a target image based on the first type of image frames and the second type of image frames; the target image comprises the first type of image frames and the second type of image frames; and displaying the target image.

Description

Technical Field

[0001] This application belongs to the field of display technology, and specifically relates to an image display method, an electronic device, and a readable storage medium. Background Technology

[0002] Currently, many scenarios require the ability to view from different perspectives. For example, in game applications that support spectator mode, players can watch other players' game footage from their perspectives to appreciate their amazing performances and learn from their gameplay.

[0003] However, when using the different viewing angle function, the relevant technology can usually only switch to one of the multiple different perspectives for viewing, which requires frequent switching between multiple perspectives. Summary of the Invention

[0004] This application provides an image display method, an electronic device, and a readable storage medium, which can solve the problem of the need to frequently switch between multiple viewpoints in related technologies.

[0005] In a first aspect, embodiments of this application provide an image display method, including: Acquire a first type of image frame and a second type of image frame; the first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of a target user, wherein the perspective of the target user is different from that of the first user; the first user and the target user are different users using the target application. A target image is generated based on the first type of image frames and the second type of image frames; the target image includes the first type of image frames and the second type of image frames. Display the target image.

[0006] Secondly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores a program or instructions, and the program or instructions, when executed by the processor, implement the method described in the first aspect.

[0007] Thirdly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0009] In this embodiment, a first type of image frame and a second type of image frame are acquired. The first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of a target user, whose perspective is different from that of the first user. The first user and the target user are different users using the target application. A target image is generated based on the first type of image frame and the second type of image frame. The target image includes the first type of image frame and the second type of image frame. The target image is then displayed. Thus, by displaying the target image, first and second type image frames of the target application from multiple different perspectives can be displayed simultaneously. Compared to related technologies that can only view a single perspective at a time, the method provided in this embodiment can display image frames from multiple perspectives without switching perspectives, enabling multi-perspective viewing and solving the problem of frequent switching between multiple perspectives required in related technologies. Attached Figure Description

[0010] Figure 1 This is an example diagram illustrating a multi-view spectating method provided by related technologies; Figure 2 This is a flowchart of an image display method provided in an embodiment of this application; Figure 3 This is a schematic diagram illustrating an example of image display in a game application provided in an embodiment of this application; Figure 4 This is a flowchart of another image display method provided in the embodiments of this application; Figure 5 This is a schematic diagram of an image transmission process provided in an embodiment of this application; Figure 6 This is a schematic diagram of a layer processing process provided in an embodiment of this application; Figure 7 This is an example flowchart of a multi-view display process provided in an embodiment of this application; Figure 8 This is a flowchart of another image display method provided in the embodiments of this application; Figure 9 This is a schematic diagram illustrating an image processing example provided in an embodiment of this application; Figure 10 This is a schematic diagram illustrating a successful block alignment provided in an embodiment of this application; Figure 11 This is a schematic diagram illustrating an example of a block alignment failure provided in an embodiment of this application; Figure 12 This is an example flowchart of another multi-view display process provided in the embodiments of this application; Figure 13This is a flowchart illustrating a specific image display method provided in an embodiment of this application; Figure 14 This is a structural block diagram of an image display device provided in an embodiment of this application; Figure 15 This is a structural block diagram of an electronic device provided in an embodiment of this application; Figure 16 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0012] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0013] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0014] The following is a brief introduction to the image compositor (SurfaceFlinger), hardware compositor (HWComposer), display processing unit (DPU), and display panel involved in the embodiments of this application.

[0015] Generally, the display of an application's screen on an electronic device follows a path: application rendering - image compositor (compositing) - HWComposer (hardware compositing / mixing) - DPU - display panel. The image compositor, HWComposer, and DPU can all be integrated within the System-on-a-Chip (SoC) chip of the electronic device. The image compositor, the software compositing engine in the display system, is one of the core services of the Android system. Running within the system process of the display system, it has the highest graphics privileges and is responsible for receiving graphics data from various applications (such as games, the desktop, and the status bar), managing the Z-order, position, and transparency of all visible interface layers (Surfaces) on the electronic device. HWComposer is a standardized hardware abstraction layer interface, serving as the abstraction layer connecting software and hardware in the Android system. It can perform layer compositing through a dedicated channel of the display hardware. The DPU is a hardware module specifically responsible for handling display-related tasks, including image scaling, rotation, color correction, and frame rate conversion. A display panel, also known as a display screen, is a physical screen hardware component, such as a liquid crystal display (LCD) or an organic light-emitting diode (OLED) screen. At the software level, it usually interacts with the display panel through a driver to control parameters such as brightness, resolution, and refresh rate.

[0016] The following example uses a game application that supports multi-view spectating as an illustration. Figure 1 , Figure 1 This is an example diagram illustrating a multi-view spectating method provided by related technologies. For example... Figure 1 As shown, related technologies can output a display image from a specific perspective to a display panel through game applications, surface synthesizers, hardware synthesizers, and display processing units (DPUs). This allows players to choose one perspective from multiple viewpoints. However, since there are many exciting moments in a game, this method can easily cause players to miss crucial scenes. Users need to frequently switch between multiple perspectives to understand the overall situation, resulting in a suboptimal multi-view viewing experience.

[0017] The method provided in this application embodiment displays target images including first-type and second-type image frames from different viewpoints, meaning that image frames from multiple viewpoints can be displayed simultaneously. This allows players to obtain images from multiple perspectives without frequently switching viewpoints, facilitating analysis of game details and enhancing the player's gaming experience. However, it should be noted that this application embodiment is not limited to the aforementioned game spectating scenario; it can also be applied to other multi-view display scenarios, such as watching online concerts or sports competitions.

[0018] The image display method provided in this application embodiment can be executed by an electronic device, wherein the electronic device can be various types of display devices, such as mobile phones, tablets, laptops and other terminal devices.

[0019] The image display method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0020] Please refer to Figure 2 , Figure 2 This is a flowchart of an image display method provided in an embodiment of this application. For example... Figure 2 As shown, this method can be executed by an electronic device, such as a terminal, and includes the following steps: Step 210: Obtain the first type of image frames and the second type of image frames; the first type of image frames includes image frames of the target application from the perspective of the first user, and the second type of image frames includes image frames of the target application from the perspective of the target user, whose perspective is different from that of the first user; the first user and the target user are different users using the target application.

[0021] In this application embodiment, the target application can be an application that supports multi-view viewing, such as a game application or a video application. In specific application scenarios, this application embodiment can be applied to game spectating scenarios, where the target application is, for example, a game application that supports spectating functionality. This application embodiment can also be applied to online concert viewing scenarios, where a first type of image frame can be used to display the view from the camera position corresponding to the first user, and a second type of image frame can be used to display the view from the camera position corresponding to the second user. When using the target application, viewing from different perspectives is supported. The first user's perspective can be the currently locked perspective of the target application, i.e., the user's primary viewing perspective when using the target application. The first user's perspective can include one or more perspectives, and the first type of image frame can include one or more image frames of the target application from the first user's perspective. The target user's perspective can be other viewing perspectives of the target application. The target user can include one or more different second users, and the target user's perspective can include one or more second perspectives. The second type of image frame can include one or more image frames of the target application from the target user's perspective. Both the first user's perspective and the target user's perspective can be viewing perspectives, i.e., the first user's perspective and the target user's perspective can be different viewing perspectives of the target application. Acquiring the first type of image frame and the second type of image frame includes: acquiring the first type of image frame and the second type of image frame when the multi-view display function is enabled.

[0022] Taking a single viewpoint as an example, the first type of image frame may include one first image frame of the target application from that single viewpoint. Taking multiple viewpoints as an example, the first type of image frame may include multiple first image frames of the target application from multiple viewpoints; the following examples all use the first user's perspective including a single viewpoint. Taking a single second viewpoint as an example, the second type of image frame may include one second image frame of the target application from that single second viewpoint. Taking multiple second viewpoints as an example, the second type of image frame may include multiple second image frames of the target application from multiple second viewpoints.

[0023] Step 220: Generate a target image based on the first type of image frames and the second type of image frames; the target image includes the first type of image frames and the second type of image frames.

[0024] In this embodiment, a first type of image frame and a second type of image frame can be stitched together to obtain a target image. The first type of image frame and the second type of image frame on the target image can be displayed without overlapping. Taking the multi-view display of a game application that supports viewing functionality as an example, see [reference needed]. Figure 3 , Figure 3 This is a schematic diagram illustrating an example of image display in a game application provided in this application embodiment. For example... Figure 3As shown, the first type of image frame is, for example, Figure 3 The image frame that occupies the largest area in the middle, the second type of image frame is, for example, Figure 3 The other four image frames in the image are not overlapping with each other. This display method can avoid the image frames from different viewing angles from occluding each other and can provide a better display effect.

[0025] It should be noted that the embodiments of this application do not limit the first type of image frame and the second type of image frame to not overlapping. In the embodiments of this application, the first type of image frame and the second type of image frame can also be displayed overlappingly, that is, the second type of image frame can be displayed on top of the first type of image frame in the form of a small window. This display method can provide a larger display area and a clearer display effect.

[0026] Step 230: Display the target image.

[0027] In this embodiment, a target image is displayed on the display panel (also known as a display screen) of an electronic device. The electronic device may include a display chip, such as a discrete display chip. During the process of generating the target image based on a first type of image frame and a second type of image frame, the first type of image frame and the second type of image frame can be stitched together by the internal processing module of the discrete display chip to obtain the target image. A discrete display chip refers to an auxiliary chip added in addition to the integrated graphics processing unit (GPU) and DPU built into the SoC chip of an electronic device, which is dedicated to handling display tasks. Discrete display chips are usually not used to perform the main 3D rendering work, but focus on post-processing image processing and efficiency optimization. However, it should be noted that this embodiment is not limited to discrete display chips, but may also include other display chips, such as integrated display chips.

[0028] In this embodiment, a first type of image frame and a second type of image frame are acquired. The first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of a target user, whose perspective differs from that of the first user. The first user and the target user are different users using the target application. A target image is generated based on the first type of image frame and the second type of image frame. The target image includes the first type of image frame and the second type of image frame. The target image is then displayed. Thus, by displaying the target image, the first type of image frame and the second type of image frame of the target application from multiple different perspectives can be displayed simultaneously. Compared to related technologies that can only view a single perspective at a time, the method provided in this embodiment can display image frames from multiple perspectives without switching perspectives, enabling multi-perspective viewing and solving the problem of frequent switching between multiple perspectives required in related technologies.

[0029] In one embodiment of this application, step 210, obtaining the first type of image frame and the second type of image frame, includes: obtaining the first type of image frame and the second type of image frame from the storage space of the display chip of the electronic device.

[0030] In this embodiment, the display chip is, for example, a standalone display chip, and the storage space is, for example, the cache area of ​​the standalone display chip. The storage space of the display chip can be used to cache second-type image frames from other perspectives of the target application, facilitating the subsequent stitching of the first-type and second-type image frames. At the same time, caching the first-type and second-type image frames through the display chip facilitates the subsequent stitching of the first-type and second-type image frames by the display chip.

[0031] In one embodiment of this application, the second type of image frame is an image frame that assists the first type of image frame, and the display area occupied by the first type of image frame is larger than the display area occupied by the second type of image frame.

[0032] In this embodiment, since the second type of image frame has a different display perspective than the first type of image frame, the second type of image frame can display the target application's screen from other perspectives. While the user is viewing the first type of image frame from the first user's perspective, the second type of image frame can assist the user in simultaneously viewing multiple different perspectives while using the target application. Simultaneously, the first type of image frame, as the user's currently locked perspective (i.e., the perspective the user focuses on), can have a larger display area than the second type of image frame from other perspectives. This allows for a clearer viewing of the first type of image frame from the first user's perspective while providing multiple viewing angles.

[0033] In one embodiment of this application, the target user's perspective includes multiple different perspectives, and the second type of image frames includes multiple image frames of the target application under multiple different perspectives, with each image frame corresponding to one of the multiple different perspectives.

[0034] In this embodiment, the target user's perspective may include multiple different second perspectives, and the multiple image frames may include multiple second image frames of the target application under multiple second perspectives, with each second image frame corresponding to one of the multiple second perspectives. For example, if the electronic device supports displaying the target application from up to K perspectives, then the target user's perspective may include, for example, K-1 second perspectives. In this way, the target application can be displayed from all angles simultaneously through images from K perspectives.

[0035] In one embodiment of this application, the target application includes a game application that supports spectator functionality, and the perspectives of the first user and the target user can be different player perspectives. In other words, the image display method provided in this embodiment can be applied to game applications, allowing gamers to learn game controls and analyze game details to enhance their gaming experience through multi-perspective spectating.

[0036] For example, a game application could be a Multiplayer Online Battle Arena (MOBA) game application. The core gameplay of a MOBA game involves multiple players forming two teams on a fixed, symmetrical map, each controlling a "hero" unit with unique abilities (e.g., Figure 3 In MOBA games (either Hero 1 or Hero 2), players compete for victory through teamwork, strategic gameplay, and skillful maneuvering. Currently, MOBA games are very popular, with numerous tournaments and spectator features in various games. Players can enjoy watching skilled players or competitors' performances, and newcomers can learn from expert strategies. Alternatively, players can use multi-view functionality during gameplay. Specifically, after a player's hero dies, they can switch to a teammate's perspective to view their teammate's actions and analyze game details, enhancing the gaming experience.

[0037] Driven by technological iteration and user demand, MOBA game viewing systems have evolved from basic viewing functions to a comprehensive experience integrating learning, social interaction, and entertainment, with their competitive value and entertainment value continuously improving. In the future, with the widespread adoption of 5G technology and cloud gaming, game viewing systems are expected to further break through device limitations, becoming a core hub connecting players, tournaments, and content creators. The method provided in this application's embodiments can optimize MOBA game viewing systems and enable multi-view viewing functionality.

[0038] Taking a game that supports displaying the game screens of five players as an example, it can simultaneously display the game screens of five players, helping users to better learn game controls. Each player's game screen is shown below. Figure 3 As shown. The first type of image frame from the perspective of the first user is, for example, the game screen occupying the largest area at the bottom. The target user includes, for example, the following: Figure 3 Players 2, 3, 4, and 5 are represented in the second type of image frames from the target user's perspective, including the four game screens shown above. Taking the first type of image frames as an example, the top left corner of the image is the game's minimap, the bottom left corner is the perspective switching area (by clicking the player icon to switch to the corresponding player's perspective), the bottom of the image shows Player 1's (the first user's) skill panel, and the remaining area is the game screen, including the "hero's" control screen and detailed map information. Figure 3(As shown in the middle four-pointed star).

[0039] Please see Figure 4 , Figure 4 This is a flowchart of another image display method provided in an embodiment of this application. For example... Figure 4 As shown, the method includes the following steps: Step 410: Receive the first type of image through the first image transmission channel in the electronic device.

[0040] Step 420: Receive the second type of image through the second image transmission channel in the electronic device.

[0041] In this embodiment, the first screen transmission channel and the second screen transmission channel are different screen transmission channels in the electronic device. See also... Figure 5 , Figure 5 This is a schematic diagram of an image transmission process provided in an embodiment of this application. Figure 5 As shown, the SoC chip within an electronic device may include an image compositor (SurfaceFlinger), a hardware compositor (HWComposer), and a DPU. The image compositor can acquire the running screen of an application, which includes a first type of screen from the perspective of a first user and a second type of screen from the perspective of the target user. The first type of screen can be transmitted through the first screen transmission channel within the electronic device (i.e.,...). Figure 5 The original image channel on the right is used for transmission, while the second type of image can be transmitted through the second image transmission channel within the electronic device (i.e., the original image channel on the right). Figure 5 The transmission takes place through the other player's view channel on the left side of the screen. The first screen transmission channel is different from the second screen transmission channel.

[0042] Step 430: Store the first type of image frame and the second type of image frame in the storage space of the display chip of the electronic device. The first type of image frame is obtained based on the first type of image, and the second type of image frame is obtained based on the second type of image.

[0043] In this embodiment, the display chip is, for example, a standalone display chip. The first type of screen can be directly identified as the first type of image frame, and the second type of screen as the second type of image frame. Alternatively, the first and second types of screens can be marked with viewing angle identifiers to obtain the first and second type of image frames. After obtaining the first and second type of image frames, they can be stored in the storage space (cache) of the standalone display chip.

[0044] Step 440: Obtain the first type of image frame and the second type of image frame from the storage space of the display chip of the electronic device.

[0045] Step 450: Generate a target image based on the first type of image frames and the second type of image frames; the target image includes the first type of image frames and the second type of image frames.

[0046] In this embodiment, if the first type of image frame includes a single first image frame and the second type of image frame includes multiple second image frames, it is possible that after receiving the single first image frame, the multiple second image frames have not yet all been cached in the display chip's storage space. In this case, historical image frames from the target application's perspective as seen by the target user can be retrieved from the display chip's storage space. These historical image frames and the first image frames can be stitched together to generate the target image for display. This ensures multi-view display without affecting the display of the target application from the first user's perspective.

[0047] Step 460: Display the target image.

[0048] In this embodiment of the application, the first type of image and the second type of image are received through the first image transmission channel and the second image transmission channel in the electronic device, respectively. In this way, the transmission of the second type of image will not affect the transmission of the first type of image, so that image frames from other perspectives can be obtained without affecting the display of the first type of image.

[0049] In one embodiment of this application, before receiving the second type of screen through the second screen transmission channel in the electronic device in step 420, the image display method includes, in addition to steps 410-460 above, steps 410-460: obtaining the viewpoint identification information layer of the target application and the layer to be displayed of the target application from the viewpoint of the target user; overlaying and compositing the layer to be displayed and the viewpoint identification information layer to obtain the second type of screen; and transmitting the second type of screen to the display chip through the second screen transmission channel.

[0050] For reference Figure 6 , Figure 6 This is a schematic diagram of a layer processing procedure provided in an embodiment of this application. An image compositer in an electronic device can acquire the viewpoint identification information layer of the target application and the layer to be displayed from the viewpoint of the target user. The hardware compositer in the electronic device then overlays and composites the layer to be displayed and the viewpoint identification information layer to obtain a second type of image. The resolution of the viewpoint identification information layer is consistent with the resolution of the electronic device's display panel (i.e., the display screen). Figure 6As shown, the viewpoint identification information layer is set to transparent, the layer to be displayed is placed at the bottom, and the viewpoint identification information layer is placed on top. The hardware compositor can composite the layer to be displayed and the viewpoint identification information layer through a dedicated overlay channel to obtain a second type of image (i.e., the composited layer). The second type of image can be transferred to the storage space of the display chip (e.g., a discrete display chip) via the DPU.

[0051] Taking a game application as an example, the viewpoint identification information layer can include N different viewpoint identification information layers (e.g., Figure 6 The layer to be displayed can include N display layers with different perspectives (e.g., player 2, player 3, etc.). Figure 6 The second type of screen includes N different perspectives of the second screen (the layers to be displayed for other players in the game). There is a one-to-one correspondence between the N identification information layers and the N display layers, and a one-to-one correspondence between the N display layers and the N second screens. For the i-th display layer among the N display layers, the hardware compositor can overlay and composite the i-th display layer with the display layer corresponding to the i-th display layer to obtain the second screen corresponding to the i-th display layer. Here, i is a positive integer less than or equal to N. Based on the N display layers, N second screens are obtained, which are the second type of screens. After obtaining the second type of screens, the second type of screens (e.g., the layers to be displayed for other players in the game) can be used to display the second type of screens. Figure 6 The player 2 screen, player 3 screen, player 4 screen and player 5 screen are stored in the display chip's storage space.

[0052] In one embodiment of this application, before receiving the first type of image through the first image transmission channel in the electronic device in step 410, the image display method includes, in addition to the steps 410-460 described above, the following: acquiring the first type of image and the second type of image; transmitting the first type of image to the display chip through the first image transmission channel, and transmitting the second type of image to the display chip through the second image transmission channel.

[0053] In this embodiment, a first type of image is transmitted to the display chip via a first image transmission channel, and the first type of image can be stored in the display chip's storage space. A second type of image is transmitted to the display chip via a second image transmission channel, and the second type of image can be stored in the display chip's storage space. The second type of image is, for example, an image obtained by overlaying and synthesizing the layer to be displayed and the viewpoint identification information layer mentioned above, or it can be the layer to be displayed mentioned above; there is no limitation here. Taking the second type of image as an example of an image obtained by overlaying and synthesizing the layer to be displayed and the viewpoint identification information layer, after the hardware synthesizer obtains the second type of image, it can be stored in the display chip's storage space. Figure 5The left-hand channel transmits data to the display chip's memory via the DPU, the second Display Serial Interface (DSI), and the second Receive (RX) channel. The first type of image can be transmitted via... Figure 6 The right-hand channel transmits data to the display chip's memory space via the image synthesizer, hardware synthesizer, DPU, and the first DSI and first RX channels. The DSI interface is a high-speed, serial, low-power communication protocol interface specifically designed to connect the display controller in the SoC chip to the display panel.

[0054] In this embodiment of the application, the first type of screen and the second type of screen can be transmitted to the storage space of the display chip simultaneously via the first screen transmission channel and the second screen transmission channel, respectively. The first type of screen and the second type of screen can be cached using the internal caching mechanism of the display chip, and the two are transmitted via different screen transmission channels, so that the second type of screen can be transmitted without affecting the transmission of the first type of screen.

[0055] In one embodiment of this application, the target user includes multiple second users, and the target user's perspective includes multiple different perspectives corresponding to the multiple second users. The second type of screen includes multiple screens of the target application under multiple different perspectives, and each screen in the multiple screens corresponds to one of the multiple different perspectives. Obtaining the second type of screen includes: sequentially obtaining each of the multiple screens in the background. In this way, since the second type of screen is obtained in the background, it does not affect the display of the foreground screen of the electronic device.

[0056] In this embodiment, the image compositer in the electronic device can sequentially acquire each of multiple screens in the background by switching perspectives. Specifically, the second type of screen includes N second screens under N second perspectives. For the i-th second perspective among the N second perspectives, the image compositer can switch the display perspective of the target application to the i-th second perspective by switching perspectives in the background, and acquire the second screen of the target application in the i-th second perspective. By sequentially switching the display perspective of the target application to each of the N second perspectives, the second screen of the target application in each of the N second perspectives can be acquired sequentially in the background.

[0057] In addition, for ease of understanding, the following describes the complete workflow in the case where the viewpoint identifier of the image is obtained by the image synthesizer. (Refer to...) Figure 7 , Figure 7 This is an example flowchart of a multi-view display process provided in an embodiment of this application. It uses a game application that supports spectator functionality as the target application and an independent display chip as an example. See also... Figure 5 To understand the whole process. For example Figure 7As shown, after enabling multi-view viewing on an electronic device, such as during game streaming or via a game console switch, the image synthesizer in the device can receive the display layers from other players by switching between different player perspectives in the background, and add a corresponding perspective identification information layer. The resolution of the perspective identification information layer is the same as the screen display resolution, and the perspective identification information layer is set to transparent. The display layer is at the bottom, and the perspective identification information layer is on top (e.g., ...). Figure 6 As shown in the layer acquisition diagram, all layers are transmitted to the hardware compositor via the second screen transmission channel. After receiving the layers, the hardware compositor scales all layers to fit the screen resolution. Then, the hardware compositor can overlay and composite the layers to be displayed and the viewpoint information layers to obtain a composited layer (such as...). Figure 6 (Layer compositing in the process). Then, the composited layer can be transmitted to the independent display chip via the second screen transmission channel through the DPU, the second DSI, and the second RX. The independent display chip can store the corresponding player's screen in the same cache (i.e., the display chip's storage space) through the viewpoint identification information layer (e.g., ...). Figure 6 (Layer storage in the image). After the display screens from other players' perspectives have been cached, the independent display chip uses the display screen of the currently locked player's perspective (the first user's perspective) as the main interface. The display screens from other perspectives (second type of image frames) are stitched together into a target image in small windows, and then the target image can be transmitted to the display panel for display.

[0058] Please see Figure 8 , Figure 8 This is a flowchart of another image display method provided in an embodiment of this application. For example... Figure 8 As shown, the method includes the following steps: Step 810: Receive the first type of image through the first image transmission channel in the electronic device.

[0059] Step 820: Receive the second type of image through the second image transmission channel in the electronic device.

[0060] Step 830: Perform content detection on the second type of screen using the display chip to obtain the viewpoint identifier of the second type of screen.

[0061] In this embodiment, the display chip is, for example, a discrete display chip. After receiving the second type of image, the discrete display chip can use motion estimation to perform content detection on the second type of image to determine which viewpoint the second type of image belongs to in the target application's display, thus obtaining the viewpoint identifier of the second type of image. Motion estimation is an algorithm of the discrete display chip used to accurately analyze and calculate the motion vector of each object (or pixel block) in an image from a continuous sequence of image frames. The second type of image is, for example, the layer to be displayed described above, i.e., the image without layer overlay.

[0062] Step 840: Mark the viewpoint identifier on the second type of image to obtain the second type of image frame.

[0063] Taking a game application that supports spectator functionality as an example, see... Figure 9 , Figure 9 This is a schematic diagram illustrating an image processing example provided in an embodiment of this application. For example... Figure 9 As shown, the second type of image (i.e. Figure 9 Other player perspective layers can be transmitted via the second screen transmission channel through the image synthesizer, hardware synthesizer, and DPU to the storage space (cache) of the independent display chip. After the independent display chip receives the second type of screen, it can perform content detection on the second type of screen to obtain the perspective identifier of the second type of screen, and mark the perspective identifier on the second type of screen, for example... Figure 9 The screen shows the label "Player 2".

[0064] The second type of screen may include one or more second screens. For example, if the second type of screen includes N second screens from N second viewpoints, then the second type of image frames may include N second image frames from N second viewpoints, with a one-to-one correspondence between the N second image frames and the N second screens. For the i-th second screen among the N second screens, content detection can be performed on the i-th second screen to obtain its viewpoint identifier. This viewpoint identifier is then marked on the i-th second screen to obtain the i-th second image frame. Based on the N second screens, N second image frames are obtained. After obtaining the N second image frames, they can be stored in the storage space (cache) of the independent display chip. The following description focuses on the content detection of a single second screen.

[0065] Step 850: Store the first type of image frame and the second type of image frame in the storage space of the display chip of the electronic device. The first type of image frame is obtained based on the first type of image.

[0066] Step 860: Obtain the first type of image frame and the second type of image frame from the storage space of the display chip of the electronic device.

[0067] Step 870: Generate a target image based on the first type of image frame and the second type of image frame; the target image includes the first type of image frame and the second type of image frame.

[0068] Step 880: Display the target image.

[0069] In this embodiment, the display chip can be used to perform content detection on the second type of screen to obtain the viewpoint identifier of the second type of screen. Compared with the method described above of generating the viewpoint identifier information layer by the image synthesizer, the power consumption of the display chip for content detection is lower, which can reduce the power consumption of the display system of the electronic device and optimize the power consumption of the device to a certain extent.

[0070] Taking the detection of the viewpoint identifier of a single frame by the display chip as an example, in one embodiment of this application, step 830 involves performing content detection on the second type of frame using the display chip to obtain the viewpoint identifier of the second type of frame. This includes: comparing the second type of frame with M pre-stored reference frames; each of the M reference frames has a viewpoint identifier; and if a target reference frame matching the second type of frame exists among the M reference frames, determining the viewpoint identifier of the target reference frame as the viewpoint identifier of the second type of frame. In this way, the viewpoint identifier of the second type of frame is obtained after comparison with the M reference frames, resulting in a more accurate viewpoint identifier.

[0071] In this embodiment, the target reference frame can be one of M reference frames. The M reference frames can include M historical frames of the target application from M different perspectives. The electronic device supports displaying the target application from up to K different perspectives, where K perspectives include M perspectives, K is an integer greater than 1, and M is a positive integer less than K. That is, the electronic device can pre-store M reference frames of the target application from M different perspectives. The second type of screen serves as the real-time display screen of the target application, and can be compared one by one with the M reference frames. If a target reference frame exists among the M reference frames that matches the second type of screen, it indicates that the target reference frame and the second type of screen represent different perspectives of the target application from the same viewpoint. The perspective identifier of the target reference frame can then be determined as the perspective identifier of the second type of screen.

[0072] In the process of comparing the second type of image with the pre-stored M reference frames, block-by-block comparisons can be performed between the second type of image and the M reference frames. (See reference...) Figure 10 and Figure 11 , Figure 10 This is a schematic diagram illustrating a successful block alignment provided in an embodiment of this application. Figure 11This is an example illustration of a block alignment failure provided in an embodiment of this application. Specifically, for the j-th reference frame among M reference frames, during the block alignment process between the second type of image and the j-th reference frame, the second type of image can be divided into multiple non-overlapping first blocks, for example... Figure 10 and Figure 11 The image is divided into multiple blocks using a grid. Since the resolution of electronic devices (such as mobile phones) is fixed, the position coordinates of any one of the first blocks obtained from the second type of image division can be acquired. Correspondingly, the j-th reference frame can also be divided into multiple non-overlapping second blocks, and the position coordinates of any one of the second blocks can be acquired. Then, each first block can be compared with multiple second blocks in the j-th reference frame that are close to (within a certain range) the first block in position, and the block in the display chip's storage space that is most similar to that first block can be found; this is block matching. Here, j is a positive integer less than or equal to M.

[0073] If each first block obtained from the second type of screen division can be matched with a similar second block in the j-th reference frame, then it can be determined that the second type of screen and the j-th reference frame have been successfully matched. Taking a game application that supports spectator functionality as an example, such as... Figure 10 As shown, Figure 10 The two images in the diagram represent the second type of screen and the j-th reference frame, respectively. If the second type of screen and the j-th reference frame are successfully matched, it means that the content displayed by the second type of screen and the j-th reference frame should be roughly the same, with only slight differences in the display position. The second type of screen and the j-th reference frame are screens displayed at different points in time from the same player's perspective. This method can be used to determine the player's perspective corresponding to the second type of screen.

[0074] If, among the multiple first blocks obtained from the second type of screen division, there exists a target first block that cannot be matched with a similar second block in the j-th reference frame, that is, the displayed content of the target first block is inconsistent with the displayed content of a similar position in the j-th reference frame, for example... Figure 11 If the content corresponding to the starting point of the middle arrow is inconsistent with the content pointed to by the arrow, then the block comparison between the second type of screen and the j-th reference frame fails, indicating that the second type of screen and the j-th reference frame belong to display screens from different perspectives.

[0075] If no reference frame among the M reference frames matches the second type of image, it can be determined that the viewpoint of the second type of image is different from all M viewpoints of the M reference frames. The viewpoint identifier of the second type of image can be directly obtained. Since the viewpoint identifier of the second type of image is different from all M viewpoint identifiers of the M reference frames, the viewpoint identifier of the second type of image can be set according to the actual application, such as "Viewpoint 1" or "Player 1". It should be noted that the embodiments of this application are not limited to determining the viewpoint identifier of the second type of image by comparing it with the pre-stored M reference frames in the above manner. In fact, if the electronic device does not store reference frames internally, the viewpoint identifier of the second type of image can be directly set, and the second type of image can be stored as a reference frame.

[0076] In one embodiment, if the target user's perspective includes N different second perspectives, the second type of screen includes N second screens of N second perspectives, where N is an integer greater than 1. Step 830 involves performing content detection on the second type of screen using the display chip to obtain the perspective identifier of the second type of screen. This includes: for the i-th second screen among the N second screens, comparing the i-th second screen with M pre-stored reference frames, each of the M reference frames having a perspective identifier; if a target reference frame matching the i-th second screen exists among the M reference frames, determining the perspective identifier of the target reference frame as the perspective identifier of the i-th second screen. The perspective identifier of each of the N second screens can be obtained in the above manner, and the corresponding perspective identifier can be marked on each of the N second screens. Specifically, for the i-th second screen among the N second screens, the perspective identifier of the i-th second screen is marked on the i-th second screen. The specific process of comparing the i-th second screen with the M reference frames can be referred to the previous text and will not be repeated here.

[0077] In one embodiment of this application, after comparing the second type of image with the pre-stored M reference frames, the image display method includes, in addition to the steps 810-880 described above, the following: if there is a target reference frame in the M reference frames that matches the second type of image, remove the target reference frame from the M reference frames and use the second type of image as a reference frame in the M reference frames; if there is no target reference frame in the M reference frames that matches the second type of image, determine the second type of image as a reference frame.

[0078] In this embodiment, if the second type of image matches the target reference frame, it indicates that the second type of image and the target reference frame are display images from the same viewpoint but at different times. Furthermore, since the second type of image is a newly acquired image through the second image transmission channel, it is more real-time than the target reference frame. The target reference frame can be replaced with the second type of image; that is, the target reference frame can be removed from the M reference frames, and the second type of image can be used as a new reference frame. In this way, the reference frames among the M reference frames are more real-time, and the accuracy of image matching using the M reference frames is higher.

[0079] Furthermore, if no target reference frame matching the second type of image is found among the M reference frames, the viewpoint identifier of the second type of image can be directly determined, and this second type of image can be designated as a reference frame. The viewpoint identifier of the second type of image is different from the M viewpoint identifiers of the M reference frames. In this way, various display images from different viewpoints can be supplemented as reference frames, improving the matching success rate of subsequent images.

[0080] The following describes the screen matching process using a specific example. Let's take a game application that supports spectator functionality as the target application and a dedicated display chip as the display chip. Initially, the dedicated display chip's storage space does not contain a reference frame. At this time, the dedicated display chip receives screen 1 through the second screen transmission channel. Since there is no reference frame, screen 1 can be directly stored as a reference frame and marked as "Player 1". The dedicated display chip continues to receive screen 2 through the second screen transmission channel and compares screen 2 with the reference frame (screen 1). If screen 2 matches successfully with the reference frame, the viewpoint of screen 2 is identified as "Player 1", and screen 1 in the reference frame is replaced with screen 2; if screen 2 fails to match, screen 2 can be stored as a new reference frame and marked as "Player 2". The dedicated display chip can continue to receive screen 3 through the second screen transmission channel and compare screen 3 with the reference frame; subsequent comparison processes will not be elaborated here.

[0081] Furthermore, for ease of understanding, the following describes the complete workflow in the case where content detection is performed by a discrete display chip. (See also...) Figure 12 , Figure 12 This is an example flowchart of another multi-view display process provided in an embodiment of this application. Taking a game application that supports spectator functionality as an example, see also... Figure 5 To understand the whole process. For example Figure 12As shown, after the electronic device enables multi-view viewing, such as enabling multi-view viewing in game live streaming or through a game console switch, the image synthesizer in the electronic device can receive the viewpoints of other players (i.e., the second type of view) by switching between different player perspectives in the background. This is then transmitted to the independent display chip via the second view transmission channel through HWComposer, DPU, the second DSI, and the second RX. Upon receiving the second type of view, the independent display chip uses a motion estimation algorithm to detect the content of the image, determine which player the received image belongs to, and mark the corresponding viewpoint identifier (also called player identifier). The marked image frame is then saved to the corresponding location in the independent display chip's storage space. After the image frames from each player's perspective are cached, the independent display chip can lock the current player's viewpoint image frame as the main interface (i.e., the first image frame transmitted through the first view transmission channel). Image frames from other perspectives are stitched together as small windows to form a target image, which is then transmitted to the display panel (also called the display screen) for display.

[0082] The motion estimation algorithm divides the image into many non-overlapping blocks. Since the phone's resolution is fixed, the coordinate information corresponding to each block can be determined. Then, each block is compared with blocks in the reference frame in the cache that have the same coordinate information within a certain range. The block in the cache that is most similar to the current block is found, i.e., block matching, which allows the determination of the corresponding player information sent.

[0083] Please see Figure 13 , Figure 13 This is a flowchart illustrating a specific image display method provided in an embodiment of this application. Figure 13 As shown, the method includes the following steps: Step 1310: Obtain the first type of screen.

[0084] Step 1320: In the background, retrieve each type of screen in the second type of screen sequentially. The second type of screen includes multiple screens of the target application from multiple different perspectives. Each screen in the multiple types of screens corresponds to one of the multiple different perspectives.

[0085] Step 1330: Transmit the first type of image to the display chip through the first image transmission channel.

[0086] Step 1340: Transmit the second type of image to the display chip through the second image transmission channel.

[0087] Here, the first image transmission channel and the second image transmission channel are different image transmission channels in the electronic device. After transmitting the second type of image to the display chip through the second image transmission channel in step 1340, the image display method, in addition to steps 1310-1370, also includes: receiving the first type of image through the first image transmission channel in the electronic device; and receiving the second type of image through the second image transmission channel in the electronic device.

[0088] Step 1350: Store the first type of image frame and the second type of image frame in the storage space of the display chip of the electronic device. The first type of image frame is obtained based on the first type of image, and the second type of image frame is obtained based on the second type of image.

[0089] The first type of image frame includes image frames of the target application from the perspective of a first user. The second type of image frame includes image frames of the target application from the perspective of a target user, whose perspective differs from that of the first user. The target application includes game applications that support spectator functionality. The second type of image frame serves as an auxiliary to the first type of image frame. The first type of image frame occupies a larger display area than the second type of image frame. The target user includes multiple second users, and the target user's perspective includes multiple different perspectives corresponding to these second users. The second type of image frame includes multiple image frames of the target application from these multiple different perspectives, with each image frame corresponding to one of the multiple different perspectives.

[0090] Step 1360: Generate a target image based on the first type of image frame and the second type of image frame; the target image includes the first type of image frame and the second type of image frame.

[0091] Step 1370: Display the target image.

[0092] In this application embodiment, two methods for marking the viewpoint of the displayed screen are provided. The first method: the viewpoint identification information layer corresponding to the layer to be displayed is obtained through an image compositer. Specifically, before obtaining each type of screen in the second type of screen in the background, the viewpoint identification information layer of the target application and the layer to be displayed of the target application from the perspective of the target user can be obtained; the layer to be displayed and the viewpoint identification information layer are superimposed and composited to obtain the second type of screen.

[0093] The second method involves performing content detection on the second type of image using the display chip to obtain the viewpoint identifier for the second type of image. Specifically, before storing the first and second type of image frames in the storage space of the display chip of the electronic device, the display chip performs content detection on the second type of image to obtain the viewpoint identifier for the second type of image; the viewpoint identifier is then marked on the second type of image to obtain the second type of image frame.

[0094] In one example, the display chip performs content detection on the second type of screen to obtain the viewpoint identifier of the second type of screen, including: comparing the second type of screen with M pre-stored reference frames; each of the M reference frames has a viewpoint identifier; if there is a target reference frame among the M reference frames that matches the second type of screen, the viewpoint identifier of the target reference frame is determined as the viewpoint identifier of the second type of screen.

[0095] In one example, after comparing the second type of image with M pre-stored reference frames, if there is a target reference frame in the M reference frames that matches the second type of image, the target reference frame is removed from the M reference frames and the second type of image is used as the reference frame in the M reference frames; if there is no target reference frame in the M reference frames that matches the second type of image, the second type of image is determined as the reference frame.

[0096] In this embodiment, a first type of image frame and a second type of image frame are acquired. The first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of a target user, whose perspective differs from that of the first user. The first user and the target user are different users using the target application. A target image is generated based on the first type of image frame and the second type of image frame. The target image includes the first type of image frame and the second type of image frame. The target image is then displayed. Thus, by displaying the target image, the first type of image frame and the second type of image frame of the target application from multiple different perspectives can be displayed simultaneously. Compared to related technologies that can only view a single perspective at a time, the method provided in this embodiment can display image frames from multiple perspectives without switching perspectives, enabling multi-perspective viewing and solving the problem of frequent switching between multiple perspectives required in related technologies.

[0097] It is important to understand that Figures 1 to 13 The explanations of the same or corresponding steps can be cross-referenced. For example, Figure 2 The explanation of step 220 is applicable to Figure 4 Step 460 and Figure 8 Step 870 in the process.

[0098] Meanwhile, it should be understood that this application embodiment utilizes the caching mechanism within the independent display chip to dynamically synthesize multi-view images, meaning the final image presented to the user is dynamic. This allows users to watch the dynamic situation of each hero in real time, displaying multiple exciting scenes simultaneously on the screen, enhancing the viewing experience. During the match, the image synthesizer (SurfaceFlinger) can acquire images from several other players' perspectives outside the viewing angle and transmit them to the independent display chip for caching via a second image transmission channel. After all the images from all perspectives have been received, the independent display chip can synthesize the second type of image frames stored in the cache with the first type of image frames transmitted via the first image transmission channel, and transmit the synthesized target image to the display screen for display, allowing the user to view the images from all players' perspectives at the same time.

[0099] Furthermore, this application provides two methods for obtaining the viewpoint identifier of the displayed image. In the first method, the present invention acquires images from several other player perspectives outside the viewing perspective using an image synthesizer, adds a player information layer to the acquired image layer, and synthesizes the other player information through a second image transmission channel before sending it to an independent display chip for caching (e.g., ...). Figure 6 Once all the required viewpoints are transmitted to the dedicated display chip for caching, the chip's internal processing module combines the first and second types of image frames before transmitting them to the display screen. This method allows users to enjoy the game from multiple player perspectives simultaneously without constantly switching viewing angles.

[0100] The second method uses a dedicated display chip to identify the player's perspective, eliminating the need for an image compositer to create a perspective identification layer. The dedicated display chip uses content detection to label the acquired second-type frames as different player perspectives. Then, when a new second-type frame is received, the dedicated display chip automatically matches the corresponding player perspective using a content detection algorithm and composites the content into the corresponding area (e.g., ...). Figure 9 Once all the required viewpoints are transmitted to the dedicated display chip's cache, the chip's internal processing module can synthesize the first and second types of image frames before transmitting them to the display screen. This method, by having the dedicated display chip perform layer recognition, reduces system load and optimizes power consumption. Furthermore, it allows viewers to observe multiple players' perspectives without constantly switching between them while watching game matches.

[0101] Please see Figure 14 , Figure 14 This is a structural block diagram of an image display device provided in an embodiment of this application. For example... Figure 14As shown, this application embodiment provides an image display device 1400, which includes: an acquisition module 1410, a generation module 1420, and a display module 1430.

[0102] The acquisition module 1410 is used to acquire a first type of image frame and a second type of image frame; the first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of a target user, the perspective of the target user is different from that of the first user; the first user and the target user are different users using the target application. The generation module 1420 is used to generate a target image based on a first type of image frame and a second type of image frame; the target image includes the first type of image frame and the second type of image frame. Display module 1430 is used to display the target image.

[0103] In this embodiment, a first type of image frame and a second type of image frame are acquired. The first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of a target user, whose perspective differs from that of the first user. The first user and the target user are different users using the target application. A target image is generated based on the first type of image frame and the second type of image frame. The target image includes the first type of image frame and the second type of image frame. The target image is then displayed. Thus, by displaying the target image, the first type of image frame and the second type of image frame of the target application from multiple different perspectives can be displayed simultaneously. Compared to related technologies that can only view a single perspective at a time, the method provided in this embodiment can display image frames from multiple perspectives without switching perspectives, enabling multi-perspective viewing and solving the problem of frequent switching between multiple perspectives required in related technologies.

[0104] The image display device provided in this application embodiment can implement the various processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0105] like Figure 15As shown in the illustration, this application also provides an electronic device 1500. The electronic device 1500 includes a processor 1510 and a memory 1520. The memory 1520 stores programs or instructions, which, when executed by the processor 1510, implement the steps of any of the methods described above. For example, when the program is executed by the processor 1510, it implements the following process: acquiring a first type of image frame and a second type of image frame; the first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of the target user, whose perspective differs from that of the first user; the first user and the target user are different users using the target application; generating a target image based on the first type of image frame and the second type of image frame; the target image includes the first type of image frame and the second type of image frame; and displaying the target image. Thus, by displaying the target image, the first type of image frames and the second type of image frames of the target application can be displayed simultaneously from multiple different perspectives. Compared with the related technologies that can only view a single perspective at a time, the method provided by this application embodiment can display image frames from multiple perspectives without switching perspectives, enabling multi-perspective viewing and solving the problem of the related technologies that require frequent switching between multiple perspectives.

[0106] For reference Figure 16 , Figure 16 This is a schematic diagram of the hardware structure of an electronic device that implements various embodiments of the present invention. For example... Figure 16 As shown, the electronic device 1600 includes, but is not limited to, components such as: radio frequency unit 1601, network module 1602, audio output unit 1603, input unit 1604, sensor 1605, display unit 1606, user input unit 1607, interface unit 1608, memory 1609, and processor 1610.

[0107] Those skilled in the art will understand that the electronic device 1600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 16 The electronic device structures shown are not intended to limit the electronic device. An electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements; these will not be elaborated further here. In embodiments of the present invention, the electronic device includes, but is not limited to, mobile phones, tablet computers, laptops, PDAs, in-vehicle terminals, wearable devices, and pedometers.

[0108] The processor 1610 is used to acquire a first type of image frame and a second type of image frame; the first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of a target user, the perspective of the target user is different from that of the first user; the first user and the target user are different users using the target application; a target image is generated based on the first type of image frame and the second type of image frame; the target image includes the first type of image frame and the second type of image frame; and the target image is displayed.

[0109] The electronic device provided in this application acquires a first type of image frame and a second type of image frame. The first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of the target user, whose perspective is different from that of the first user. The first user and the target user are different users using the target application. A target image is generated based on the first type of image frame and the second type of image frame. The target image includes the first type of image frame and the second type of image frame. The target image is then displayed. Thus, by displaying the target image, the first type of image frame and the second type of image frame of the target application from multiple different perspectives can be displayed simultaneously. Compared to related technologies that can only view a single perspective at a time, the method provided in this application can display image frames from multiple perspectives without switching perspectives, achieving a multi-view viewing effect and solving the problem of frequently switching between multiple perspectives required in related technologies.

[0110] It should be understood that, in this embodiment of the invention, the radio frequency unit 1601 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink data from the base station and processes it with the processor 1610; additionally, it transmits uplink data to the base station. Typically, the radio frequency unit 1601 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc. Furthermore, the radio frequency unit 1601 can also communicate with networks and other devices through a wireless communication system.

[0111] Electronic devices provide users with wireless broadband internet access through network module 1602, such as helping users send and receive emails, browse web pages, and access streaming media.

[0112] The audio output unit 1603 can convert audio data received by the radio frequency unit 1601 or the network module 1602 or stored in the memory 1609 into audio signals and output them as sound. Furthermore, the audio output unit 1603 can also provide audio output related to specific functions performed by the electronic device 1600 (e.g., call signal reception sound, message reception sound, etc.). The audio output unit 1603 includes a speaker, a buzzer, and a receiver, etc.

[0113] Input unit 1604 is used to receive audio or video signals. Input unit 1604 may include a graphics processing unit (GPU) 16041 and a microphone 16042. GPU 16041 processes image data of still images or videos acquired by an image capture device (such as a camera) in video capture mode or image capture mode. The processed image frames can be displayed on display unit 1606. The image frames processed by GPU 16041 can be stored in memory 1609 (or other storage medium) or transmitted via radio frequency unit 1601 or network module 1602. Microphone 16042 can receive sound and process such sound into audio data. The processed audio data can be converted into a format that can be transmitted to a mobile communication base station via radio frequency unit 1601 in telephone call mode.

[0114] The electronic device 1600 also includes at least one sensor 1605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 16061 according to the ambient light level, and the proximity sensor can turn off the display panel 16061 and / or backlight when the electronic device 1600 is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used to identify the posture of the electronic device (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc. The sensor 1605 may also include a fingerprint sensor, pressure sensor, iris sensor, molecular sensor, gyroscope, barometer, hygrometer, thermometer, infrared sensor, etc., which will not be described in detail here.

[0115] The display unit 1606 is used to display information input by the user or information provided to the user. The display unit 1606 may include a display panel 16061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0116] User input unit 1607 can be used to receive input numerical or character information, and generate key signal inputs related to user settings and function control of electronic devices. Specifically, user input unit 1607 includes a touch panel 16071 and other input devices 16072. Touch panel 16071, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near touch panel 16071). Touch panel 16071 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to processor 1610, which receives and executes commands from processor 1610. In addition, touch panel 16071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 16071, the user input unit 1607 may also include other input devices 16072. Specifically, other input devices 16072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0117] Furthermore, the touch panel 16071 can cover the display panel 16061. When the touch panel 16071 detects a touch operation on or near it, it transmits the information to the processor 1610 to determine the type of touch event. Subsequently, the processor 1610 provides corresponding visual output on the display panel 16061 based on the type of touch event. Although in Figure 16 In this embodiment, the touch panel 16071 and the display panel 16061 are two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 16071 and the display panel 16061 can be integrated to realize the input and output functions of the electronic device. The specific implementation is not limited here.

[0118] Interface unit 1608 serves as an interface for connecting external devices to electronic device 1600. For example, external devices may include a wired or wireless headphone port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, and so on. Interface unit 1605 may be used to receive input from external devices (e.g., data, power, etc.) and transmit the received input to one or more components within electronic device 1600, or it may be used to transmit data between electronic device 1600 and external devices.

[0119] The memory 1609 can be used to store software programs and various data. The memory 1609 may primarily include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc.; the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory 1609 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0120] Processor 1610 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 1609, and by calling data stored in memory 1609, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Processor 1610 may include one or more processing units; preferably, processor 1610 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 1610.

[0121] In addition, the electronic device 1600 includes some functional modules not shown, which will not be described in detail here.

[0122] This application also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of various embodiments of the image display method and achieve the same technical effect. To avoid repetition, these will not be described again here.

[0123] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0124] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0125] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0126] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0127] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0128] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image display method, characterized in that, include: Acquire a first type of image frame and a second type of image frame; the first type of image frame includes image frames of the target application from the perspective of a first user, and the second type of image frame includes image frames of the target application from the perspective of a target user, wherein the perspective of the target user is different from that of the first user. The first user and the target user are different users using the target application; A target image is generated based on the first type of image frames and the second type of image frames; the target image includes the first type of image frames and the second type of image frames. Display the target image.

2. The method according to claim 1, characterized in that, The acquisition of the first type of image frame and the second type of image frame includes: Obtain the first type of image frame and the second type of image frame from the storage space of the display chip of the electronic device.

3. The method according to claim 2, characterized in that, Before retrieving the first type of image frame and the second type of image frame from the storage space of the display chip of the electronic device, the method further includes: The first type of image is received through the first image transmission channel in the electronic device; The second type of image is received through the second image transmission channel in the electronic device; wherein, the first image transmission channel and the second image transmission channel are different image transmission channels in the electronic device; The first type of image frame and the second type of image frame are stored in the storage space of the display chip of the electronic device; The first type of image frame is obtained based on the first type of image, and the second type of image frame is obtained based on the second type of image.

4. The method according to claim 3, characterized in that, Before receiving the second type of image through the second image transmission channel in the electronic device, the method further includes: Obtain the viewpoint identification information layer of the target application and the layer of the target application to be displayed from the viewpoint of the target user; The layer to be displayed and the viewpoint identification information layer are overlaid and combined to obtain the second type of image; The second type of image is transmitted to the display chip through the second image transmission channel.

5. The method according to claim 3, characterized in that, Before receiving the first type of image through the first image transmission channel in the electronic device, the method further includes: Acquire the first type of image and the second type of image; The first type of image is transmitted to the display chip through the first image transmission channel; The second type of image is transmitted to the display chip through the second image transmission channel.

6. The method according to claim 5, characterized in that, The target user includes multiple second users, and the perspective of the target user includes multiple different perspectives corresponding to the multiple second users; the second type of screen includes multiple screens of the target application under the multiple different perspectives, and each of the multiple screens corresponds to one of the multiple different perspectives; The second type of image is obtained, including: In the background, each of the various screens is retrieved sequentially.

7. The method according to any one of claims 3, 5-6, characterized in that, Before storing the first type of image frame and the second type of image frame in the storage space of the display chip of the electronic device, the method further includes: The display chip performs content detection on the second type of image to obtain the viewpoint identifier of the second type of image; Mark the viewpoint identifier on the second type of screen to obtain the second type of image frame.

8. The method according to claim 7, characterized in that, The step of performing content detection on the second type of image using the display chip to obtain the viewpoint identifier of the second type of image includes: The second type of image is compared with M pre-stored reference frames; each of the M reference frames has a viewpoint identifier. If there is a target reference frame among the M reference frames that matches the second type of image, the viewpoint identifier of the target reference frame is determined as the viewpoint identifier of the second type of image.

9. The method according to claim 8, characterized in that, After comparing the second type of image with the pre-stored M reference frames, the method further includes: If a target reference frame matching the second type of image exists among the M reference frames, the target reference frame is removed from the M reference frames, and the second type of image is used as a reference frame among the M reference frames; If there is no target reference frame among the M reference frames that matches the second type of image, the second type of image is determined as the reference frame.

10. The method according to claim 1, characterized in that, The second type of image frame is an image frame that assists the first type of image frame; the display area occupied by the first type of image frame is greater than the display area occupied by the second type of image frame.

11. The method according to claim 1, characterized in that, The target applications include game applications that support spectator functionality.

12. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 11.

13. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the method as described in any one of claims 1 to 11.

Images