Hybrid display method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a hybrid display method and device, electronic equipment and storage medium, wherein the method comprises: in response to a display interface change event, detecting whether a group of native views in a native view tree exists a target native view, in response to the existence of the target native view, performing drawing processing on the target native view to generate a proxy texture corresponding to the target native view; based on the display level order of the native view tree, in the rendering context of the surface view, the proxy texture and the display content corresponding to the surface view are synthesized and displayed, wherein the synthesis display processing is used to display the target native view in the form of the proxy texture inside the display content corresponding to the surface view. Through the present application, the problem of display abnormalities between the native view and the surface view carrying content in the related art is solved, and the stability of interface display and the consistency of interface display are improved.

Description

Technical Field

[0001] This application relates to the field of rendering technology, and more specifically, to a hybrid display method, apparatus, electronic device, and storage medium. Background Technology

[0002] In related technologies, SurfaceView (also known as Surface Control) is commonly used in native Android development to carry high-performance graphical content, such as the host's video screen in a live streaming room, special effects bullet screen systems (such as rockets, fireworks, and gift animations), 3D scenes rendered by game engines, and real-time video filters or AI beautification processing streams. The advantage of SurfaceView lies in its independent drawing surface, which can bypass the drawing pipeline of the Android View (also known as native control) tree and directly perform efficient rendering through the system compositor (SurfaceFlinger), thereby achieving low latency, high frame rate, and low power consumption performance. However, because SurfaceView uses an independent drawing surface, it cannot participate in the unified hierarchical composition of the Android View tree. This leads to display anomalies when it is mixed with native controls (such as buttons, avatar frames, input boxes, etc.), such as occlusion anomalies, holes (due to the independent mechanism of SurfaceView, they are invisible and displayed as black holes), hierarchical disorder, clipping failure, and visual fragmentation. Therefore, in related technologies, there are technical problems that easily lead to display anomalies between the content carried by native views (View) and SurfaceView. Summary of the Invention

[0003] This application provides a hybrid display method, apparatus, electronic device, and storage medium to at least solve the technical problem in the related art where display anomalies easily occur between the content carried by the native view and the surface view.

[0004] According to one aspect of the embodiments of this application, a hybrid display method is provided, comprising: in response to a display interface change event, detecting whether a target native view exists in a set of native views in a native view tree, wherein the target native view is a native view that overlaps with a surface view, and the native view tree includes the surface view; in response to the existence of the target native view, performing drawing processing on the target native view to generate a proxy texture corresponding to the target native view; and, based on the display hierarchy order of the native view tree, performing composite display processing on the proxy texture and the display content corresponding to the surface view in the rendering context of the surface view, wherein the composite display processing is used to display the target native view in the form of the proxy texture inside the display content corresponding to the surface view.

[0005] According to another aspect of the embodiments of this application, a hybrid display device is also provided, comprising: a detection unit, configured to detect, in response to a display interface change event, whether a target native view exists in a set of native views in a native view tree, wherein the target native view is a native view that overlaps with a surface view, and the native view tree includes the surface view; a drawing unit, configured to, in response to the existence of the target native view, perform drawing processing on the target native view to generate a proxy texture corresponding to the target native view; and a display unit, configured to, based on the display hierarchy order of the native view tree, perform composite display processing on the proxy texture and the display content corresponding to the surface view in the rendering context of the surface view, wherein the composite display processing is used to display the target native view in the form of the proxy texture inside the display content corresponding to the surface view.

[0006] In an exemplary embodiment, the detection unit includes: an acquisition module, configured to acquire the visible area of ​​each native view in the set of native views and the display area of ​​the surface view; a first determination module, configured to determine native views in the set of native views whose visible areas overlap with the display area of ​​the surface view as candidate views; and a detection module, configured to perform the following conflict detection on the candidate views and the surface view based on the display layer of the candidate views and the display layer of the surface view: determining the relative display order between the candidate views and the surface view based on the display layer of the candidate views and the display layer of the surface view; and determining the candidate view as the target native view if the relative display order between the candidate views and the surface view is inconsistent with a specified display order, wherein the specified display order is used to indicate the system display order between the candidate views and the surface view.

[0007] In one exemplary embodiment, the drawing unit includes: a drawing module, configured to perform drawing processing on the target native view in a preset buffer based on the display information of the target native view and the drawing environment of the target native view, to generate a proxy texture of the target native view, wherein the display information is used to indicate the drawing content of the target native view.

[0008] In an exemplary embodiment, the hybrid display device further includes: a first acquisition unit, configured to acquire transformation parameters between a proxy texture of the target native view and the target native view, wherein the transformation parameters are used to indicate the mapping relationship between the proxy texture of the target native view and the target native view; and a first determination unit, configured to determine the transformation parameters and the display information as metadata of the proxy texture, and establish a binding relationship between the proxy texture and the metadata.

[0009] In one exemplary embodiment, the hybrid display device includes: a second determining unit, configured to determine a specified overlapping region, wherein the specified overlapping region is an overlapping region of the visible area of ​​the target native view and the display area of ​​the surface view; and a third determining unit, configured to determine the specified overlapping region as a region to be processed in the target native view, wherein the display information of the target native view is used to indicate the drawing content of the region to be processed.

[0010] In one exemplary embodiment, the display unit includes: a second determining module, configured to determine the relative drawing order of the target native view in the native view tree based on the display hierarchy order of the native view tree, wherein the relative drawing order is used to indicate the insertion position of the target native view in the rendering queue of the surface view; and an insertion module, configured to insert the proxy texture into the rendering queue of the surface view in the rendering context of the surface view based on the relative drawing order, so as to perform composite display processing of the proxy texture and the display data corresponding to the surface view, wherein the rendering queue of the surface view is used to indicate the rendering order of the display content corresponding to the surface view in the rendering context of the surface view.

[0011] In one exemplary embodiment, the hybrid display device further includes: a second acquisition unit, configured to acquire coordinate information of a specified interaction event in response to a specified interaction event, wherein the specified interaction event is receiving an interaction instruction on a proxy display area, and the proxy display area is the area where a proxy texture is located in the area where the surface view is located; a conversion unit, configured to perform coordinate system conversion processing on the coordinate information of the specified interaction event based on a coordinate mapping relationship to obtain converted coordinate information, wherein the coordinate mapping relationship is used to indicate the mapping relationship between the first coordinate system where the surface view is located and the second coordinate system where the target native view is located; and an execution unit, configured to control the instance corresponding to the target native view to execute the specified interaction instruction based on the converted coordinate information.

[0012] In one exemplary embodiment, the hybrid display device further includes: a second acquisition unit, configured to acquire coordinate information of the specified update event in response to the specified update event, wherein the specified update event is an update event for a proxy display area, and the proxy display area is the area where a proxy texture is located in the area where the surface view is located; and an update unit, configured to determine an area to be updated based on the coordinate information of the specified update event, and update the texture area in the proxy texture corresponding to the area to be updated, wherein the coordinate information of the area to be updated includes the coordinate information of the specified update event.

[0013] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed by a processor.

[0014] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in any of the method embodiments described above.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to perform the steps of any of the above method embodiments through the computer program.

[0016] This application addresses the issue of display anomalies caused by detecting overlap between native and surface views in response to display interface change events. It identifies the target native view and generates a proxy texture corresponding to it. Within the rendering context of the surface view, the texture is composited according to the hierarchical order of the native view tree, embedding its content into the internal display of the surface view. This solves the problem of display anomalies easily occurring between the content carried by the native view and the surface view in related technologies, achieving the technical effect of improving the stability and consistency of the interface display. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating an application scenario of a hybrid display method according to an embodiment of this application;

[0018] Figure 2 This is a flowchart illustrating an optional hybrid display method according to an embodiment of this application;

[0019] Figure 3 This is a flowchart illustrating another optional hybrid display method according to an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of an optional hybrid display method according to an embodiment of this application;

[0021] Figure 5 This is a structural block diagram of an optional hybrid display device according to an embodiment of this application;

[0022] Figure 6This is a computer system architecture block diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] According to one aspect of the embodiments of this application, a hybrid display method is provided. Optionally, in this embodiment, the above-described hybrid display method may be applied, but is not limited to, to applications such as... Figure 1 The hardware environment shown includes terminal device 102 and server 104. Server 104 can be connected to terminal device 102 via a network and can be used to provide services (e.g., application services, etc.) to terminal device 102 or clients installed on terminal device 102. A database can be set up on server 104 or independently of server 104 to provide data storage services for server 104.

[0026] The aforementioned network may include, but is not limited to, at least one of the following: wired network and wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network (WAN), metropolitan area network (MAN), and local area network (LAN). The aforementioned wireless network may include, but is not limited to, at least one of the following: Wireless Fidelity (WIFI) and Bluetooth. Terminal device 102 may be, but is not limited to, a personal computer (PC), mobile phone, tablet computer, etc. Server 104 may be, but is not limited to, a cloud server, server cluster, or other server types.

[0027] The hybrid display method of this application embodiment can be executed by the terminal device 102. Alternatively, the execution of the hybrid display method of this application embodiment by the terminal device 102 can also be executed by a client installed on it.

[0028] Taking the hybrid display method in this embodiment executed by terminal device 102 as an example, Figure 2 This is a flowchart illustrating an optional hybrid display method according to an embodiment of this application, such as... Figure 2 As shown, the process of this method may include the following steps:

[0029] Step S202: In response to the display interface change event, detect whether there is a target native view in a group of native views in the native view tree, wherein the target native view is a native view that overlaps with the surface view, and the native view tree includes the surface view;

[0030] Step S204: In response to the existence of a target native view, the target native view is drawn to generate a proxy texture corresponding to the target native view;

[0031] Step S206: Based on the display hierarchy order of the native view tree, in the rendering context of the surface view, the proxy texture and the display content corresponding to the surface view are composited for display processing. The composited display processing is used to display the target native view in the form of a proxy texture inside the display content corresponding to the surface view.

[0032] The hybrid display method in this embodiment can be applied to the field of rendering technology, and can be applied to scenarios such as live streaming platforms, short video interaction, gamified UI, and real-time video overlay, which require cross-level natural fusion display of high-performance rendered content (such as video streams, Cocos effects, and OpenGL particles) and native interactive controls (such as buttons, avatar frames, input boxes, and bullet screen entrances).

[0033] In related technologies, SurfaceView is commonly used in native Android development to host high-performance graphical content, such as: the host's video screen in a live streaming room, special effects bullet screen systems (such as rockets, fireworks, and gift animations), 3D scenes rendered by game engines, and real-time video filters or AI beautification processing streams. The advantage of SurfaceView lies in its independent drawing surface, which can bypass the Android View (native view) tree's drawing pipeline and directly perform efficient rendering through the system compositor (SurfaceFlinger), thus achieving low latency, high frame rate, and low power consumption performance. However, because SurfaceView uses an independent drawing surface, it is prone to display anomalies when mixed with native controls (such as buttons, avatar frames, input boxes, etc.). These anomalies include issues such as occlusion anomalies, holes (due to SurfaceView's independent mechanism, they are invisible and displayed as black holes), layer misalignment, clipping failure, and visual fragmentation. Therefore, related technologies often present technical problems related to display anomalies between content carried by native views (View) and SurfaceView.

[0034] To at least partially solve the above-mentioned technical problems, in this embodiment, in response to a display interface change event, the target native view is determined by detecting the overlap relationship between the native view and the surface view, and a proxy texture corresponding to the target native view is generated. In the rendering context of the surface view, the native view is composited according to the hierarchical order of the native view tree, thereby embedding its drawing content into the internal display of the surface view. This solves the problem that display anomalies are prone to occur between the content carried by the native view (View) and the surface view (SurfaceView) in related technologies, and achieves the technical effect of improving the stability and consistency of the interface display.

[0035] It should be noted that display interface change events can refer to system events that trigger the redraw or relayout of the Android view tree, including but not limited to layout change events, scrolling events, show / hide events, animation change events, and SurfaceView area change events. The native view tree can refer to the hierarchical tree structure in the Android system composed of ViewGroups and Views, used to manage the layout, drawing, and event dispatch of UI elements. In this embodiment, the surface view (also called a surface control) is a node in the native view tree. Optionally, the native view tree may include at least one native view (also called a native control) and at least one surface view; in practical application scenarios, the native view tree may contain multiple native views and one or more surface views. For each native view, it is necessary to detect whether there is an overlap relationship with at least one surface view to determine whether a target native view exists. Both native views and surface views can be understood as view controls.

[0036] Optionally, a set of native views can be all native views in the native view tree, or native views that are to change based on a display interface change event. Specifically, when a display interface change event is detected, the native views to be detected can be determined based on the changed area indicated by the event. Each native view to be detected is traversed to determine if a target native view exists among them. Here, the native view to be detected is the native view whose display area intersects with the changed area. The target native view can refer to a native view in the native view tree whose visible area overlaps with the display area of ​​the surface view.

[0037] Optionally, the presence of a target native view can be determined by obtaining the display area of ​​each native view to be detected and, based on the display areas of the native views and the surface view, by determining whether there is an overlap. Of course, the native view to be detected can also be any native view in the native view tree.

[0038] Optionally, during the process of detecting whether a target native view exists, it is also possible to determine whether the display state of the native view is visible and whether the transparency of the native view is greater than a preset threshold, so that the native view is marked as a target native view only when it actually participates in the rendering of user-visible content.

[0039] It should be noted that rendering processing can refer to rendering the visual display information of the target native view into a texture image, i.e., a proxy texture, by calling the drawing method of the target native view. A proxy texture can refer to the graphic texture data generated by the content drawn by the target native view that can be received by the SurfaceView rendering pipeline. It can include image pixel data, alpha channel, original size, local clipping region, transformation matrix (translation / scaling / rotation), etc., and is used to simulate the visual appearance of the native view on the surface.

[0040] Optionally, the drawing process may include drawing only the local area in the target native view that overlaps with the surface view, rather than drawing the entire control; the proxy texture may contain only the pixel data of the overlapping area and carry its offset information relative to the original control coordinate system, so as to reduce texture memory usage and drawing overhead.

[0041] It should be noted that the display hierarchy order of the native view tree refers to the drawing order of each view in the native view tree, which is determined by the order of the child elements of the parent container. In this embodiment, the display hierarchy order of the native view tree can be used as the basis for the insertion order of proxy textures in the surface view. The rendering context of the surface view is used to indicate the rendering environment and rendering pipeline of the surface view. It is usually an OpenGL ES (Open Graphics Library for Embedded Systems) or Skia (an open source 2D graphics rendering engine) rendering pipeline, which includes: the currently bound Surface, GL (Graphics Library) context, viewport matrix, blending mode, clipping region, depth buffer, etc.

[0042] Composite display processing refers to overlaying proxy textures as layers onto the original content (such as videos or effects) of the SurfaceView within the rendering context of the SurfaceView, performing pixel-level blending to achieve the visual effect that makes it appear as if the native controls are displayed inside the SurfaceView.

[0043] Optionally, based on the display hierarchy order of the native view tree, the drawing order of each native view in the native view tree is determined. Then, based on the drawing order of each native view, all generated proxy textures are processed sequentially. For each proxy texture, the position of the proxy texture in the coordinate system of the surface view can be located based on the metadata of the proxy texture, and the proxy texture can be mixed based on the metadata of the proxy texture to draw the proxy texture inside the display content corresponding to the surface view. The inside of the display content corresponding to the surface view can refer to the frame buffer of the surface view for output display.

[0044] Optionally, the drawing order of proxy textures can also be dynamically adjusted based on the hierarchical relationship of the target native view corresponding to the proxy texture in the native view tree and the preset rendering level of the content inside the SurfaceView, so that the proxy texture can be inserted at a specified position between multiple sub-layers in the SurfaceView, achieving precise interleaving display of multi-layer content.

[0045] The embodiments provided in this application, in response to a display interface change event, detect whether a target native view exists in a set of native views in the native view tree, wherein the target native view is a native view that overlaps with the surface view, and the native view tree includes the surface view; in response to the existence of the target native view, perform drawing processing on the target native view to generate a proxy texture corresponding to the target native view; based on the display hierarchy order of the native view tree, in the rendering context of the surface view, perform composite display processing with the display content corresponding to the surface view, wherein the composite display processing is used to display the target native view as a proxy texture inside the display content corresponding to the surface view. By detecting the overlap relationship between the native view and the surface view, the target native view is determined, and a corresponding proxy texture is generated. In the rendering context of the surface view, composite processing is performed according to the hierarchy order of the native view tree, thereby embedding its drawing content into the internal display of the surface view. This solves the problem of display anomalies that easily occur between the content carried by the native view (View) and the surface view (SurfaceView) in related technologies, achieving the technical effect of improving the stability and consistency of the interface display.

[0046] In one exemplary embodiment, in order to reduce the computational overhead of processing, it is possible to determine whether there is an overlap based on the visible area of ​​the native view, thereby determining whether a target native view exists.

[0047] Correspondingly, detecting whether a target native view exists in a group of native views in the native view tree includes: obtaining the visible area of ​​each native view in the group of native views and the display area of ​​the surface view; identifying native views whose visible areas overlap with the display area of ​​the surface view as candidate views; and performing the following conflict detection on the candidate views and the surface view based on the display hierarchy of the candidate views and the surface view: determining the relative display order between the candidate views and the surface view based on the display hierarchy of the candidate views and the surface view; and determining the candidate view as the target native view if the relative display order between the candidate views and the surface view is inconsistent with the specified display order, wherein the specified display order is used to indicate the system display order between the candidate views and the surface view.

[0048] It should be noted that the visible area of ​​each native view can refer to the visible pixel range of each native view after image processing by the parent container. Image processing can include cropping, scrolling, and transparency processing.

[0049] The display hierarchy refers to the drawing order index of a view (native view or surface view) within its parent container. A larger index indicates a later drawing position, resulting in a later display position on the interface. Optionally, the relative display order between the candidate view and the surface view can be determined based on their respective display hierarchy levels. This relative display order can be understood as the order in which the candidate view and surface view, as indicated in the native view tree, are expected by the business logic. For example, the relative display order could mean that the candidate view is displayed above or below the surface view. Specifying the display order can also indicate the system display order between the candidate view and the surface view; that is, in the operating system, the surface view is usually located above the native view by default.

[0050] Optionally, a candidate view can be determined based on the visible area of ​​each native view in a set of native views and the display area of ​​the surface view. Specifically, the visible rectangular area of ​​each native view in a set of native views is determined, i.e., the visible area of ​​each native view, and the display area of ​​the surface view is determined. The display area of ​​the surface view is used to reflect the final rendering range in the system compositor and may deviate from its layout position in the view tree due to the scaling, offset, or independent hierarchy of the SurfaceControl. A two-dimensional planar operation is performed on the visible area of ​​each candidate view and the display area of ​​the surface view. An overlap is determined only if there is an overlap (i.e., a non-empty intersection) between the visible area and the display area of ​​the surface view in the two-dimensional plane.

[0051] Optionally, based on the native view tree, the display hierarchy of each native view and each surface view can be determined. Then, after determining the candidate views, the display hierarchy of the candidate views and the surface views can be obtained to determine the relative display order between the candidate views and the surface views. It should be noted that the native view tree records the index of each view (i.e., the index expected by the business logic). Based on the index of each view, the display hierarchy of the native views and surface views can be determined. If the index of a candidate view is greater than the index of the SurfaceView, then the business logic expects the candidate view to be above the SurfaceView; if the index of a candidate view is less than the index of the SurfaceView, then the business logic expects the candidate view to be below the SurfaceView.

[0052] It's important to note that in determining the existence of a target native view, the native view's position area, hierarchy, display state, and visible area can be obtained. The native view's position area and hierarchy are used to determine whether there is spatial overlap (corresponding to overlap relationship) and hierarchy conflict (i.e., whether the relative display order matches the specified display order), respectively. The display state determines whether the native view is actually participating in the display, such as whether it is hidden, transparent, or not currently in a valid display state. The visible area can be used to determine the area of ​​the target space that is still visible to the user after being clipped, obscured, or scrolled by the parent container, thus avoiding proxying invalid areas.

[0053] This embodiment determines overlap based on the visible area of ​​each native view and checks for display order conflicts when overlap is detected. This allows for proxy processing only for controls that are incorrectly suppressed due to the SurfaceView's independent mechanism (such as the like button and avatar frame in a live stream), avoiding accidental texturing of non-conflicting controls and significantly reducing CPU / GPU overhead. Simultaneously, by reusing the hierarchical information of the native view tree, it ensures that proxy textures are synthesized within the SurfaceView in the original design order.

[0054] In one exemplary embodiment, to ensure the display effect of the proxy texture to a certain extent, the target native view can be drawn based on its drawing method to obtain the corresponding proxy texture. Correspondingly, the drawing process of the target native view to generate the proxy texture includes: in a preset buffer, drawing the target native view based on its display information and drawing environment to generate the proxy texture of the target native view, wherein the display information is used to indicate the drawing content of the target native view.

[0055] It should be noted that the preset buffer can refer to an off-screen graphics buffer allocated for generating proxy textures, such as OpenGL's Framebuffer Object (FBO) or Android's HardwareLayer. The off-screen graphics buffer is used to temporarily store the drawing results of the target native view, avoiding direct rendering to the screen. The display information of the target native view can include visual attributes such as the target native view's size, text content, background color, image resources, transparency, and margins. Of course, the display information of the target native view can also include the clipping region, which can be used to indicate the area where the target native view overlaps with the surface view. To reduce the computational overhead required for rendering to a certain extent, proxy texture drawing can be performed only on the differences in overlap between the target native view and the surface view.

[0056] The drawing environment of the target native view can be used to indicate the context state of the target native view in the native view tree. This can include the clipping region of the parent container, the transformation matrix (including scaling / rotation), the drawing order (z-order), the animation state (such as gradient / translation), and the current theme style (such as font and color theme), ensuring that the proxy texture accurately reproduces the original visual performance.

[0057] Optionally, the rendering result of the target native view is converted into a readable texture image, i.e., a proxy texture. The proxy texture can be a texture image corresponding to a clipped region in the target native view, a texture image corresponding to a specified region in the target native view, or a texture image corresponding to the entire region in the target native view. The clipped region can be the overlapping area of ​​the target native view and the surface view. If a proxy texture corresponding to the target native view exists in advance, the clipped region can also be the region that has changed based on the display interface change event. The specified region can include the clipped region, but is less than or equal to the entire region in the target native view. The specified region can be an area of ​​a specified size set based on actual needs.

[0058] Optionally, in the preset buffer, the drawing method of the target native view can be invoked, and drawing can be performed based on the display information of the target native view in the drawing environment of the target native view to obtain the proxy texture.

[0059] In this embodiment, by reusing the real drawing environment, the proxy texture completely retains the display content of the original view (i.e., the corresponding target native view), accurately restores visual consistency, and thus ensures the display effect of the proxy texture.

[0060] In an exemplary embodiment, in order to accurately reproduce the display content of the target native view, the method further includes: obtaining transformation parameters between the proxy texture of the target native view and the target native view, wherein the transformation parameters are used to indicate the mapping relationship between the proxy texture of the target native view and the target native view; determining the transformation parameters and display information as metadata of the proxy texture, and establishing a binding relationship between the proxy texture and the metadata.

[0061] It should be noted that transformation parameters describe the mapping relationship between the original display space of the target native view and the display space of the proxy texture, such as translation, scaling, rotation, and coordinate offset. Display information can include visual attributes such as the target view's size, transparency (alpha), clipping region, drawing order (z-index), and background style, which guide the rendering behavior of the proxy texture during the compositing stage.

[0062] The metadata of a proxy texture can serve as a description of its behavior and properties, used to reconstruct the visual information of the proxy texture within the rendering context of the surface view. Optionally, by binding the generated proxy texture to its corresponding metadata, an association can be established to quickly query and apply the correct parameters each time the proxy texture is composed, avoiding misalignment or distortion.

[0063] Optionally, the metadata of the proxy texture may also include the hierarchy information of the corresponding target native view in the native view tree, that is, the display hierarchy of the target native view in the native view tree.

[0064] Optionally, while generating the proxy texture, the real-time change status of the target native view can be collected to obtain the display information of the target native view. If the target native view is in an animation state, the change value of each frame of the target native view can be obtained to ensure that dynamic changes are captured. Subsequently, these parameters (such as original view coordinates, scaling ratio, rotation angle, clipping rectangle, transparency, and layer index (layer information)) are encapsulated and bound to the generated proxy texture.

[0065] Within the rendering context of the surface view, before each composition, the metadata of the target proxy texture is obtained based on the layout state of the current frame. This allows the proxy texture to be precisely drawn onto the specified position of the surface view according to its original parameters. For example, if the original view is scaled by 1.5 times or rotated by 30 degrees, the proxy texture will apply the same transformation synchronously to ensure complete visual consistency. For partial update scenarios (such as text changes), only the texture content is updated, and the metadata is reused to avoid rebinding.

[0066] In this embodiment, by binding the metadata of the proxy texture to the proxy texture, the original spatial pose of the view can be restored by transforming the parameters, thus achieving accurate reproduction of the display content of the target native view.

[0067] In an exemplary embodiment, in order to reduce resource overhead, only the overlapping area with the composite can be drawn. Specifically, a specified overlapping area is determined, wherein the specified overlapping area is the overlapping area of ​​the visible area of ​​the target native view and the display area of ​​the surface view; the specified overlapping area is determined as the area to be processed in the target native view, wherein the display information of the target native view is used to indicate the drawing content of the area to be processed.

[0068] It should be noted that the specified overlapping area can refer to the two-dimensional intersection area of ​​the visible area of ​​the target native view and the display area of ​​the SurfaceView in the screen coordinate system. That is, the pixel range in which the target native view and the surface area actually overlap in space visually. In order to reduce resource consumption, only the specified overlapping area can be drawn. Then, the display information of the corresponding target native view can include all the visual content that the target native view should present within the specified overlapping area, including background, text, icons, animation frames, transparency, clipping shape, etc., which can ensure that the proxy texture only accurately reproduces the visual performance of the overlapping part.

[0069] Optionally, after determining the target native view, the actual visible area of ​​the target native view in the screen coordinate system can be obtained, and the display area of ​​the surface view can be obtained. The intersection between the visible area of ​​the target native view and the display area of ​​the surface view can be calculated to obtain the specified overlapping area, which may be a rectangle, an empty set, or an incomplete rectangle (such as an irregular shape caused by clipping of the parent container).

[0070] Optionally, a clipping boundary for off-screen rendering is constructed based on the specified overlapping area. Then, within a preset buffer, the rendering method of the target native view is called with this overlapping area as the rendering range, and only the overlapping portion is rendered. After rendering, only the pixel data of the overlapping area is uploaded as a texture, not the entire view, while the coordinates, size, and relative offset of the overlapping area are bound to the texture as part of the metadata. During subsequent updates, the system only listens for changes in the dirty areas of this region (such as text updates or button state toggles) to trigger incremental rendering, avoiding a full rebuild.

[0071] This embodiment reduces texture memory usage and texture upload bandwidth by drawing only the overlapping areas with the composition, and avoids full-view drawing, thus shortening the drawing time.

[0072] In an exemplary embodiment, to restore the true visual hierarchy, a proxy texture corresponding to the target native view can be displayed according to the display hierarchy order of the original view tree. Correspondingly, based on the display hierarchy order of the native view tree, in the rendering context of the surface view, the proxy texture and the display content corresponding to the surface view are composited for display processing. This includes: determining the relative drawing order of the target native view in the native view tree based on the display hierarchy order of the native view tree, wherein the relative drawing order is used to indicate the insertion position of the target native view in the rendering queue of the surface view; and inserting the proxy texture into the rendering queue of the surface view in the rendering context of the surface view based on the relative drawing order, so as to composite the proxy texture and the display data corresponding to the surface view for display processing. The rendering queue of the surface view is used to indicate the rendering order of the display content corresponding to the surface view in the rendering context of the surface view.

[0073] It's important to note that relative drawing order refers to the priority of the target native view relative to other nodes in the native view tree. This priority is determined by the child element index of the target native view within its parent container; a higher child element index results in a later drawing position (i.e., a higher visual position). The rendering queue refers to an ordered list of all layers to be composited within the rendering context of the surface view. This could include video frames, special effects particles, bullet screen layers, and custom objects. Proxy textures are inserted into the corresponding positions in this queue according to their relative drawing order, rather than simply being stacked at the top or bottom. The insertion position refers to the specific insertion index of the proxy texture in the rendering queue, mapped from the relative hierarchy of the target view in the native view tree, ensuring that its layout within the surface view maintains the same front-to-back relationship as the original layout.

[0074] In one example, within the rendering context of the SurfaceView, a rendering queue is used to pre-define content types (e.g., background video, special effects particles, bullet comments, gift buttons) according to business logic. Upon detecting the existence of a target native view, its child element index in the parent container (e.g., index=5) is read, and all other candidate views overlapping with the SurfaceView in the native view tree are traversed. The relative relationship between their indices and the SurfaceView's own index is calculated to construct a virtual hierarchy graph. Based on this, it is calculated that the target view should be located after the bullet comments and before the gift effects in the complete visual hierarchy, corresponding to the 3rd insertion position in the rendering queue. Subsequently, the system inserts the proxy texture (containing metadata) of the target view as an independent layer into the corresponding position of the SurfaceView's rendering queue. During compositing, the rendering engine draws sequentially according to the queue order: first the video background, then the bullet comments, and then the proxy texture (restoring its original transparency, clipping, and transformation).

[0075] Optionally, when there are multiple target native views, they can be inserted sequentially according to the index of each target native view in the multiple target native views to keep the relative hierarchy of the native layout unchanged.

[0076] In this embodiment, the proxy texture corresponding to the target native view is mapped to the rendering queue of the surface view by mapping the display hierarchy order in the native view tree, thereby realizing the restoration of the real visual hierarchy.

[0077] In an exemplary embodiment, to ensure the consistency of interaction of the native view, interaction events on the target native control can be executed through coordinate mapping relationships and the instance corresponding to the target native view. Correspondingly, the above method further includes: responding to a specified interaction event, obtaining the coordinate information of the specified interaction event, wherein the specified interaction event is receiving an interaction instruction on a proxy display area, and the proxy display area is the area where the proxy texture is located in the area where the surface view is located; performing coordinate system transformation processing on the coordinate information of the specified interaction event based on the coordinate mapping relationship to obtain the transformed coordinate information, wherein the coordinate mapping relationship is used to indicate the mapping relationship between the first coordinate system where the surface view is located and the second coordinate system where the target native view is located; and controlling the instance corresponding to the target native view to execute the specified interaction instruction based on the transformed coordinate information.

[0078] It's important to note that specified interaction events can refer to input events such as clicks, swipes, long presses, and double-clicks triggered on native controls (like buttons, input fields, and avatar frames) rendered by proxy textures within the SurfaceView area. Although these events occur on the surface view, the proxy display area is the area where the proxy texture resides within the surface view's area, and therefore this area can be handled by the native view instance. The proxy display area can refer to the two-dimensional area that the proxy texture is actually mapped and displayed in the SurfaceView rendering context. It is not necessarily equivalent to the entire range of the proxy texture; its boundaries are dynamically calculated from the proxy texture's metadata (position, scaling, rotation, clipping), and are not the original size of the native control, but rather its visual projection area in the surface view. The coordinate mapping relationship can refer to the affine transformation function between the first coordinate system of the surface view and the second coordinate system of the target native view. This is constructed by the transformation parameters of the proxy texture (translation, scaling, rotation) and the absolute position of the original control in the native view tree, ensuring that the input coordinates can be accurately back-derived to the original control's local coordinate space.

[0079] Optionally, in response to a received interaction command, the screen coordinates of the interaction command are captured, and the location of the screen coordinates of the interaction command is checked against the proxy display area of ​​the proxy texture. Optionally, the boundary rectangle (including the transformed coordinates) in the metadata of each proxy texture is queried for determination. If the screen coordinates of the interaction command are located within the proxy display area of ​​one of the proxy textures, coordinate mapping is initiated. Based on the metadata bound to the corresponding proxy texture, the coordinate transformation between the surface view and the target native view corresponding to the proxy texture is performed. For example, if the proxy texture is scaled by 1.2 times, rotated by 15°, and offset by (100, 80), the event coordinates are first reverse-rotated, then reverse-scaled, and finally the offset is subtracted to restore the point in the local coordinate system of the original control. After the coordinate transformation, it can be directly distributed to the retained native control instance, enabling it to fully respond to native logic such as click, focus, long press, and drag.

[0080] Through this embodiment, interactive events on the target native control can be executed by using coordinate mapping relationships and the corresponding instance of the target native view, ensuring the consistency of interaction of the native view, and the occurrence of accidental touches can be avoided to a certain extent by using coordinate mapping relationships.

[0081] In an exemplary embodiment, to further reduce resource overhead, proxy texture processing can be performed only on the area to be updated. Correspondingly, in response to a specified update event, the coordinate information of the specified update event is obtained, wherein the specified update event is an update event for the proxy display area, and the proxy display area is the area where the proxy texture is located in the area where the surface view is located; based on the coordinate information of the specified update event, the area to be updated is determined, and the texture area in the proxy texture corresponding to the area to be updated is updated, wherein the coordinate information of the area to be updated includes the coordinate information of the specified update event.

[0082] It should be noted that the specified update time can refer to a system event that triggers a change in the content or state of the target native view. This can include, but is not limited to, text changes, animation frame updates, scroll offsets, changes in opacity, and selection state toggling. Its core characteristic is that it only affects a local part of the control, not the entire control. The proxy display area can refer to the two-dimensional area that the proxy texture of the target native control is actually mapped and displayed in the surface view. Its extent is determined by the transformation parameters in the metadata and the overlapping area, and may be an irregular rectangle or a dynamically changing sub-region.

[0083] The area to be updated can refer to the valid pixel area that needs to be redrawn due to a specified update event. It can be calculated from the intersection of the dirty area of ​​the target native view and the proxy display area, or it can be an area of ​​a specified size. The coordinate information of the area of ​​a specified size includes the coordinate information of the specified update event.

[0084] Optionally, when the target native control is updated due to business logic (such as text update or button highlighting), the local dirty area coordinates (relative to the control's own coordinate system) are captured. These dirty area coordinates are then transformed to the coordinate system of the surface view using an established coordinate mapping relationship to obtain its projection range within the proxy display area. Next, the intersection of this projection range and the proxy display area itself is calculated to determine the final area to be updated. This area may be only the width of a single character in the original control (such as a button text change), an icon area (such as an avatar update), or an area containing coordinate information related to the specified update event.

[0085] Optionally, off-screen redrawing can be performed only on the area to be updated in the target native view to generate corresponding local proxy texture data, so as to update the corresponding part of the original proxy texture. The whole process skips full image redrawing, does not rebuild texture buffer, does not trigger complete compositing process, and only updates the changed parts.

[0086] Optionally, if an excessively high update frequency of the target native view is detected (e.g., continuous triggering during scrolling), multiple small dirty areas can be merged into a single batch update instruction to further reduce GPU overhead. Optionally, the update frequency, animation state, interaction activity level, and current system rendering load of the target native view can be used to adjust the refresh frequency, update range, and composition priority of the proxy texture. When the system load exceeds a preset threshold, the proxy refresh frequency can be reduced, static control updates can be paused, or some target controls can be switched back to normal native display mode to ensure overall rendering stability.

[0087] This embodiment reduces resource overhead by reconstructing only the area to be updated, compared to reconstructing the entire proxy texture.

[0088] The following explanation, using optional examples, illustrates the hybrid display method in this application, primarily providing a method and system for cross-layer hybrid display of SurfaceView and native view based on texture proxy nodes. This aims to solve problems such as layer misalignment, abnormal occlusion, display fragmentation, and interaction failure caused by independent rendering of SurfaceView and native View on the Android platform. This hybrid display method is suitable for business scenarios requiring cross-layer fusion display of native controls and SurfaceView-borne content, such as live streaming rooms, special effects bullet comments, interactive gameplay, and gamified interfaces. SurfaceView can be used to carry video content, OpenGL rendered content, game engine content, or other independent surface-rendered content; the native view can be a button, avatar frame, input box, text control, prompt control, container control, or other Android native View controls, such as... Figure 3 and Figure 4 As shown, it may specifically include the following:

[0089] Step S302, Interface initialization;

[0090] Step S304: Load the native View tree and SurfaceView content;

[0091] Specifically, the native View tree of the Android interface (including controls such as buttons, text, and containers) and SurfaceView (carrying video, OpenGL rendering, or special effects content) are loaded, and the current proxy texture relationship between the native view and the surface view is obtained. That is, the proxy texture relationship between the target native control and the surface view that has been identified can include the original area information, visible area information, proxy texture display area information, and corresponding transformation parameters of the target native control that has been recorded. A correspondence between control instances and proxy textures is formed so that the mapping relationship can be updated synchronously when the layout changes, scrolls, animations, or the area is refreshed.

[0092] Step S306: Detect whether there is a cross-level relationship between the native view and the SurfaceView;

[0093] Specifically, in response to UI change events (such as layout changes, scrolling, animations, control showing / hiding, and SurfaceView size changes), the native views in the native view tree are traversed to obtain their position, size, visible area, hierarchy order, and display status. This determines whether there is overlap between native views and surface views. If overlap exists, the overlapping native views are identified as candidate views. After identifying candidate views, the relative display order between candidate views and surface views can be determined. The relative display order is then compared with the specified display order to determine whether there is a cross-hierarchical relationship. It should be noted that the native view tree records the index of each view (i.e., the index expected by the business). Based on the index of each view, the display hierarchy of native views and surface views can be determined. If the index of a candidate view is greater than the index of the SurfaceView, then the business expects the candidate view to be above the SurfaceView; if the index of a candidate view is less than the index of the SurfaceView, then the business expects the candidate view to be below the SurfaceView.

[0094] Step S308: If there is a cross-level relationship, obtain the target native view;

[0095] Step S310: Retain the instance of the target native view;

[0096] Step S312: Switch the target native view (target native control) to a texture proxy node and obtain the target space drawing results and display attributes;

[0097] Step S314: Generate proxy texture;

[0098] Specifically, the drawing path of the target native view is switched to an off-screen buffer; the current visual state of the target native view is collected, and proxy texture data containing the following attributes is generated: image content (pixel data), transparency (alpha), cropping region (the part overlapping with SurfaceView), transformation parameters (translation, scaling, rotation), and level index (for subsequent composition sorting). The generated proxy texture and its metadata (coordinates, transformation, level) are cached for use by the composition module.

[0099] Step S316: Perform secondary composite display based on the hierarchy of the original View tree;

[0100] Specifically, in the rendering context of the surface view, all generated proxy textures are received; according to the drawing order in the original native view tree, the proxy textures are inserted as virtual layers into the rendering queue of the original rendering content of the SurfaceView. By combining transparency, clipping, and transformation matrices, they are uniformly mixed and synthesized with native SurfaceView content such as videos and special effects, realizing cross-level visual fusion of native controls embedded inside the SurfaceView.

[0101] Step S318: In response to the interaction event, post the interaction event back to the instance of the target native control;

[0102] Specifically, it captures user input events (clicks, swipes, etc.) within the SurfaceView area; determines whether the event coordinates fall within any proxy display area; if so, it converts the screen coordinates back to the native control's local coordinates based on a preset coordinate mapping relationship (constructed by the transformation parameters of the proxy texture and the original control position); and distributes the converted events to the reserved native control instances, which then handle the click, state change, and other logic in their entirety, ensuring seamless interaction.

[0103] Step S320: Control the state of the target native view and update the displayed content;

[0104] Specifically, it listens for property change events of the target native control and detects changed areas (dirty areas), and only redraws and updates the proxy texture for local areas that overlap with the SurfaceView.

[0105] Step S322: In the absence of any cross-level relationship, maintain the original display;

[0106] Step S324: Output and display the results.

[0107] It should be noted that, Figure 4The cross-level detection module can be used to execute steps S302 to S308, the texture proxy switching module can be used to execute steps S310 to S312, the proxy texture generation module can be used to execute step S314, the secondary compositing display module can be used to execute step S316, the event postback module can be used to execute step S318, and the state synchronization update module can be used to execute step S320. Specifically, the performance control module can be used to optimize the generation and update process of proxy textures, ensuring system performance and rendering stability. Through the on-demand proxy mechanism, it only textures target native controls that have a cross-level with SurfaceView, avoiding the overhead of full-page texturing; combined with dirty area detection, it implements local incremental updates to reduce off-screen drawing and texture uploading; for static controls, it reduces the refresh rate or reuses cached textures, and dynamically improves the update priority for animated / high-frequency controls; when the system load exceeds the limit, it executes dynamic degradation strategies, such as pausing the update of non-critical controls, reducing the refresh rate, or temporarily switching back to native display, to ensure the smoothness of the main thread and user experience.

[0108] This optional example dynamically detects the spatial overlap and business hierarchy between native controls (native views) and SurfaceView, triggering on-demand texture delegation only for target controls (target native views) that need to be displayed together, preserving their native instances to maintain complete event handling and state management capabilities. Based on the original View tree hierarchy index, it achieves precise secondary compositing of proxy textures in the SurfaceView rendering context, allowing native controls to be naturally embedded within high-performance content such as videos and special effects, achieving seamless cross-level integration. Simultaneously, through local dirty area updates, static control cache reuse, dynamic frequency adjustment of animated controls, and intelligent degradation strategies when the system load is too high, it significantly reduces CPU / GPU overhead and memory usage, avoiding the performance loss caused by full-page texturing. This achieves complete preservation of the interactivity and state consistency of native controls without sacrificing SurfaceView rendering performance, significantly improving visual continuity, display stability, and user experience in complex scenarios such as live streaming rooms, special effects bullet comments, and game UIs.

[0109] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0111] According to another aspect of the embodiments of this application, a hybrid display device is also provided, which can be used to implement the hybrid display method provided in the above embodiments, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0112] Figure 5 This is a structural block diagram of an optional hybrid display device according to an embodiment of this application, such as... Figure 5 As shown, the hybrid display device includes:

[0113] The detection unit 502 is used to detect whether a target native view exists in a set of native views in the native view tree in response to a display interface change event. The target native view is a native view that overlaps with the surface view. The native view tree includes the surface view.

[0114] The drawing unit 504 is used to draw the target native view in response to the existence of the target native view, so as to generate the proxy texture corresponding to the target native view;

[0115] Display unit 506 is used to perform composite display processing on proxy textures and the display content corresponding to the surface view in the rendering context of the surface view based on the display hierarchy order of the native view tree. The composite display processing is used to display the target native view as a proxy texture inside the display content corresponding to the surface view.

[0116] It should be noted that the detection unit 502 in this embodiment can be used to perform the above step S202, the drawing unit 504 in this embodiment can be used to perform the above step S204, and the display unit 506 in this embodiment can be used to perform the above step S206.

[0117] The embodiments provided in this application, in response to a display interface change event, detect whether a target native view exists in a set of native views in the native view tree, wherein the target native view is a native view that overlaps with the surface view, and the native view tree includes the surface view; in response to the existence of the target native view, perform drawing processing on the target native view to generate a proxy texture corresponding to the target native view; based on the display hierarchy order of the native view tree, in the rendering context of the surface view, perform composite display processing with the display content corresponding to the surface view, wherein the composite display processing is used to display the target native view as a proxy texture inside the display content corresponding to the surface view. By detecting the overlap relationship between the native view and the surface view, the target native view is determined, and a corresponding proxy texture is generated. In the rendering context of the surface view, composite processing is performed according to the hierarchy order of the native view tree, thereby embedding its drawing content into the internal display of the surface view. This solves the problem of display anomalies that easily occur between the content carried by the native view (View) and the surface view (SurfaceView) in related technologies, achieving the technical effect of improving the stability and consistency of the interface display.

[0118] In one exemplary embodiment, the detection unit 502 includes:

[0119] The acquisition module is used to acquire the visible area of ​​each native view and the display area of ​​the surface view in a set of native views;

[0120] The first determining module is used to determine the native views in a set of native views whose visible areas overlap with the display areas of the surface view as candidate views;

[0121] The detection module is used to perform the following conflict detection on the candidate view and the surface view based on the display hierarchy of the candidate view and the surface view: determine the relative display order between the candidate view and the surface view based on the display hierarchy of the candidate view and the surface view; if the relative display order between the candidate view and the surface view is inconsistent with the specified display order, determine the candidate view as the target native view, wherein the specified display order is used to indicate the system display order between the candidate view and the surface view.

[0122] In one exemplary embodiment, the drawing unit 504 includes:

[0123] The drawing module is used to draw the target native view in a preset buffer based on the display information and drawing environment of the target native view, so as to generate a proxy texture of the target native view. The display information is used to indicate the drawing content of the target native view.

[0124] In an exemplary embodiment, the hybrid display device further includes: a first acquisition unit, configured to acquire transformation parameters between a proxy texture of a target native view and the target native view, wherein the transformation parameters are used to indicate the mapping relationship between the proxy texture of the target native view and the target native view; and a first determination unit, configured to determine the transformation parameters and display information as metadata of the proxy texture, and establish a binding relationship between the proxy texture and the metadata.

[0125] In one exemplary embodiment, the hybrid display device includes: a second determining unit, configured to determine a specified overlapping region, wherein the specified overlapping region is the overlapping region of the visible area of ​​the target native view and the display area of ​​the surface view;

[0126] The third determining unit is used to determine the specified overlapping area as the processing area of ​​the target native view, wherein the display information of the target native view is used to indicate the drawing content of the processing area.

[0127] In one exemplary embodiment, the display unit 506 includes:

[0128] The second determining module is used to determine the relative drawing order of the target native view in the native view tree based on the display hierarchy order of the native view tree, wherein the relative drawing order is used to indicate the insertion position of the target native view in the rendering queue of the surface view.

[0129] The insertion module is used to insert proxy textures into the rendering queue of the surface view in the rendering context of the surface view based on the relative drawing order, so as to composite the proxy textures with the display data corresponding to the surface view for display processing. The rendering queue of the surface view is used to indicate the rendering order of the display content corresponding to the surface view in the rendering context of the surface view.

[0130] In an exemplary embodiment, the hybrid display device further includes: a second acquisition unit, configured to acquire coordinate information of a specified interaction event in response to a specified interaction event, wherein the specified interaction event is receiving an interaction instruction on a proxy display area, and the proxy display area is the area where a proxy texture is located in the area where the surface view is located;

[0131] The transformation unit is used to perform coordinate system transformation on the coordinate information of a specified interactive event based on the coordinate mapping relationship to obtain the transformed coordinate information. The coordinate mapping relationship is used to indicate the mapping relationship between the first coordinate system where the surface view is located and the second coordinate system where the target native view is located.

[0132] The execution unit is used to control the instance corresponding to the target native view to execute specified interactive instructions based on the transformed coordinate information.

[0133] In one exemplary embodiment, the hybrid display device further includes: a second acquisition unit, configured to acquire coordinate information of a specified update event in response to a specified update event, wherein the specified update event is an update event for a proxy display area, and the proxy display area is the area where a proxy texture is located in the area where the surface view is located;

[0134] An update unit is used to determine the area to be updated based on the coordinate information of a specified update event, and to update the texture area in the proxy texture corresponding to the area to be updated, wherein the coordinate information of the area to be updated includes the coordinate information of the specified update event.

[0135] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0136] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program executes the steps in any of the above method embodiments when it is run.

[0137] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0138] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is configured to perform the steps of any of the method embodiments described above via the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0139] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0140] According to another aspect of the embodiments of this application, a computer program product is also provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit 601, it performs various functions provided in the embodiments of this application. The sequence numbers of the embodiments of this application above are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0141] Figure 6 A schematic block diagram of a computer system architecture for implementing embodiments of the present application is shown. Figure 6 As shown, the computer system 600 includes a Central Processing Unit (CPU) 601, which performs various appropriate actions and processes based on programs stored in ROM 602 or loaded into RAM 603 from storage section 608. Random access memory 603 also stores various programs and data required for system operation. The CPU 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0142] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card, such as a local area network card or modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0143] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit 601, it performs various functions defined in the system of this application.

[0144] It should be noted that, Figure 6 The computer system 600 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0145] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0146] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A hybrid display method, characterized in that, include: In response to a display interface change event, detect whether a target native view exists in a set of native views in the native view tree, wherein the target native view is a native view that overlaps with the surface view, and the native view tree includes the surface view; In response to the existence of the target native view, the target native view is drawn to generate a proxy texture corresponding to the target native view; Based on the display hierarchy order of the native view tree, in the rendering context of the surface view, the proxy texture and the display content corresponding to the surface view are composited for display processing, wherein the composited display processing is used to display the target native view in the form of the proxy texture inside the display content corresponding to the surface view.

2. The method according to claim 1, characterized in that, The detection of whether a target native view exists in a set of native views in the native view tree includes: Obtain the visible area of ​​each native view in the set of native views and the display area of ​​the surface view; Native views whose visible areas overlap with the display area of ​​the surface view are identified as candidate views; Based on the display hierarchy of the candidate view and the display hierarchy of the surface view, the following conflict detection is performed on the candidate view and the surface view: Based on the display hierarchy of the candidate view and the display hierarchy of the surface view, the relative display order between the candidate view and the surface view is determined; If the relative display order between the candidate view and the surface view is inconsistent with the specified display order, the candidate view is determined to be the target native view, wherein the specified display order is used to indicate the system display order between the candidate view and the surface view.

3. The method according to claim 1, characterized in that, The step of drawing the target native view to generate a proxy texture corresponding to the target native view includes: In a preset buffer, based on the display information and drawing environment of the target native view, the target native view is drawn to generate a proxy texture of the target native view, wherein the display information is used to indicate the drawing content of the target native view.

4. The method according to claim 3, characterized in that, The method further includes: Obtain the transformation parameters between the proxy texture of the target native view and the target native view, wherein the transformation parameters are used to indicate the mapping relationship between the proxy texture of the target native view and the target native view; The transformation parameters and the display information are determined as the metadata of the proxy texture, and a binding relationship is established between the proxy texture and the metadata.

5. The method according to claim 3, characterized in that, The method further includes: Determine a specified overlapping region, wherein the specified overlapping region is the overlapping region of the visible area of ​​the target native view and the display area of ​​the surface view; The specified overlapping area is determined as the area to be processed in the target native view, wherein the display information of the target native view is used to indicate the drawing content of the area to be processed.

6. The method according to claim 3, characterized in that, Based on the display hierarchy order of the native view tree, in the rendering context of the surface view, the process of compositing the proxy texture with the display content corresponding to the surface view includes: Based on the display hierarchy order of the native view tree, the relative drawing order of the target native view in the native view tree is determined, wherein the relative drawing order is used to indicate the insertion position of the target native view in the rendering queue of the surface view; Based on the relative drawing order, in the rendering context of the surface view, the proxy texture is inserted into the rendering queue of the surface view to synthesize the proxy texture with the display data corresponding to the surface view for display processing. The rendering queue of the surface view is used to indicate the rendering order of the display content corresponding to the surface view in the rendering context of the surface view.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: In response to a specified interaction event, the coordinate information of the specified interaction event is obtained, wherein the specified interaction event is receiving an interaction instruction on the proxy display area, and the proxy display area is the area where the proxy texture is located in the area where the surface view is located; Based on the coordinate mapping relationship, the coordinate information of the specified interactive event is transformed by the coordinate system to obtain the transformed coordinate information. The coordinate mapping relationship is used to indicate the mapping relationship between the first coordinate system where the surface view is located and the second coordinate system where the target native view is located. Based on the transformed coordinate information, the instance corresponding to the target native view is controlled to execute the specified interaction command.

8. The method according to any one of claims 1 to 6, characterized in that, The method further includes: In response to a specified update event, obtain the coordinate information of the specified update event, wherein the specified update event is an update event for the proxy display area, and the proxy display area is the area where the proxy texture is located in the area where the surface view is located; Based on the coordinate information of the specified update event, the region to be updated is determined, and the texture region in the proxy texture corresponding to the region to be updated is updated, wherein the coordinate information of the region to be updated includes the coordinate information of the specified update event.

9. A hybrid display device, characterized in that, include: A detection unit is used to detect, in response to a display interface change event, whether a target native view exists in a set of native views in the native view tree, wherein the target native view is a native view that overlaps with a surface view, and the native view tree includes the surface view; A drawing unit is configured to, in response to the existence of the target native view, perform drawing processing on the target native view to generate a proxy texture corresponding to the target native view; The display unit is used to perform composite display processing on the proxy texture and the display content corresponding to the surface view in the rendering context of the surface view, based on the display hierarchy order of the native view tree. The composite display processing is used to display the target native view in the form of the proxy texture inside the display content corresponding to the surface view.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images