Method for filtering android display picture layer

Abstract

The invention discloses a method for filtering an android display picture layer. The method comprises the following steps that: a SurfaceFlinger transmits prepared image display data to an image drawing function of an image synthesis engine (Compostion Engine); when the presentation of the Compostion Engine is executed, an output stack is traversed firstly, an Output calls a preparation function presentation and transmits a screen synthesis refreshing parameter, and Layer elements displayed by the current Output are drawn and rendered in sequence; the input Layer stack is traversed by the Output, in addition, a screen Output object provides a name attribute and serves as a mark for distinguishing the screen, in the whole traversing process, the affiliation relation between the Layer and the current Output needs to be judged, and after it is confirmed that the Layer belongs to the current screen Output, the Layer is drawn and rendered; when the screen output filtering scheme needs to be locked, the Output filters the drawing Layer of the specified screen Output; and after all Output display contents of the screen of the current system are drawn one by one, the drawn data are synthesized and displayed on the corresponding screen through HWC.

Description

Technical Field

[0001] The present invention belongs to the technical field of Android image display, and in particular relates to a method for filtering Android display screen layers. Background Art

[0002] As one of the IoT technologies, wireless screen projection breaks the traditional physical screen projection scheme. Whether it is the construction of transmission links or the encapsulation and transmission of video streams, wireless screen projection is more advanced than traditional screen projection. In recent years, developers have gradually increased their work on wireless screen projection technology on the Android system. Wireless screen projection technology is used in video conferencing, office development, and cross-terminal screen sharing. Although wireless screen projection technology brings a lot of convenience to users, users' demand for privacy protection and rejection of pop-up boxes in wireless screen projection is becoming more and more obvious, especially in conference occasions. Wireless screen projection in the absence of privacy protection and rejection of pop-up boxes will not only interfere with the progress and explanation of the meeting, but also lead to the exposure of personal sensitive information. In addition, when Android devices use virtual display solutions to record screens, the automatic pop-up boxes of the software will block some important content, resulting in the screen content to be recorded being obscured or even exposing some sensitive information from the pop-up box content, which ultimately leads to users not being able to achieve the desired screen recording effect. Summary of the invention

[0003] In view of the above problems, the present invention provides a method for filtering Android display screen layers, which effectively limits the behavior of automatic pop-up windows of applications, meets the needs of privacy protection and rejection of pop-up windows, and provides users with a more user-friendly functional scenario in the field of Android screen projection office.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A method for filtering an Android display screen layer comprises the following steps:

[0006] The display sends a vertical synchronization signal Vsync to the hardware composition component HWC of the Android system display layer, and HWC calls back the synchronization method onComposerHalVsync of the primitive processing server SurfaceFlinger to prepare for screen display content synchronization;

[0007] After the Android system receives the Vsync signal, SurfaceFlinger recalculates the display content of all current screens and obtains the primitive transfer unit Surface and Surface handle SurfaceControl created by the application display window; SurfaceFlinger uses the obtained Surface and SurfaceControl to construct the Layer, and adds the constructed Layer to the current system screen Z-axis Layer management stack;

[0008] At the same time, SurfaceFlinger registered in the screen drawing scheduler Scheduler generates Vsync processing signals at fixed intervals and calls its signal processing method onTransact to notify the message queue MessageQueue to process the screen display refresh event. MessageQueue calls back SurfaceFlinger's message receiving function onMessageReceived to send the message back to SurfaceFlinger;

[0009] After receiving the screen display refresh message, SurfaceFlinger executes the screen refresh processing function onMessageRefresh. The refresh processing function creates a screen composition refresh parameter object, then collects all the drawing data output terminals of the current Android system and adds them to the output stack of the screen composition refresh parameter object, then traverses all the layers in the screen Z-axis Layer management stack and adds them to the Layer stack of the composition refresh parameter object; when it is necessary to filter the layer in a targeted manner, it traverses the elements of the screen Z-axis Layer management stack; SurfaceFlinger passes the prepared image display data to the image drawing function present of the image composition engine CompositionEngine;

[0010] When executing the present of CompositionEngine, the output stack is traversed first. Output calls the prepare function and passes in the screen composition refresh parameters, and draws and renders the current Output display Layer elements in sequence;

[0011] Output traverses the passed-in Layer stack. In addition, the screen Output object provides a name attribute as a mark to distinguish the screen. During the entire traversal process, the relationship between the Layer and the current Output must be determined. After confirming that the Layer belongs to the current screen Output, it will be drawn and rendered. When the screen output filtering scheme needs to be locked, Output filters the drawing Layer of the specified screen Output.

[0012] After drawing all the output display contents of the current system screen one by one, the drawn data is synthesized and displayed on the corresponding screen through HWC.

[0013] In a possible implementation manner, the screens among all current screens include a main screen, a secondary screen, and a virtual screen.

[0014] In a possible implementation, when a layer needs to be directional filtered, traversing the screen Z-axis layer management stack elements includes:

[0015] Take out the Layer element from the screen Z-axis Layer management stack;

[0016] Get the Layer name information and compare it with the target string;

[0017] When the layer name is equal to or contains the target field, the layer is not added to the layer stack of the synthesis refresh parameter object and the layer element is taken out from the screen Z-axis layer management stack to take out the next layer element;

[0018] Adds the Layer to the Layer stack of the Composition Refresh Parameters object.

[0019] In a possible implementation, the logic of the Output filtering specifying the drawing layer of the screen Output includes:

[0020] Compare the screen Output object name. If it is the target screen Output that needs to be filtered by Layer, proceed to the next step. Otherwise, draw normally.

[0021] Compare the name of the layer that needs to be drawn. If it is the target layer that needs to be filtered, do not draw it. Otherwise, draw it normally.

[0022] Follow the above steps to traverse the next Layer.

[0023] The present invention has the following beneficial effects:

[0024] (1) Directional layer filtering: After MessageQueue sends a screen refresh message to SurfaceFlinger, SurfaceFlinger will automatically perform a screen image refresh operation and use the created Surface and SurfaceControl to construct a new Layer. It will then reorganize all parameters of the current system screen synthesis refresh content, including all outputs of the current system and all layers on the screen Z axis. When SurfaceFlinger iteratively adds layers to the screen synthesis refresh layer stack, it compares the name attribute of the Layer object to filter all layers whose names are equal to or contain the target field to ensure that they are not added to the layer stack, so that the image display system bottom layer cannot draw the filtered layers, thereby achieving the effect that all layers with matching names on all outputs of the current system are hidden or cannot pop up.

[0025] (2) Accurate output filtering: The image display layer will register all the current system screen outputs to SurfaceFlinger. Faced with Android's multi-screen usage scenarios, different screens will have different display requirements. The unified layer filtering method cannot block content on a specified screen. SurfaceFlinger collects all the current system outputs and constructs a layer stack by creating a screen composition refresh parameter object, and then passes all the screen composition refresh parameters to CompositionEngine. CompositionEngine will first traverse all the current system screen outputs. The output will call the preparation function and pass in the screen composition refresh parameters, and then iterate the layer stack. During this period, the name attribute provided by the screen Output object can be used to determine whether it is the target screen that needs to filter the layer, and then match all layers whose names are equal to or contain the target field. When the corresponding layer is matched, the output's drawing processing logic for the layer is skipped, so that the target screen output display does not contain the drawing content of the layer that needs to be filtered, and the target screen display layer filtering effect is accurately achieved.

[0026] (3) Humanized functional design: Traditional screen projection technology generally adopts a solution similar to screen display cloning to present, whether it is critical content or relatively sensitive information, it will be clearly visible on the projection screen. In the design of the screen projection function of the Android system, the invention can enhance the user's screen projection experience by directional filtering of screen content. On the one hand, it controls the behavior of automatic pop-up boxes of applications, and on the other hand, it reduces the leakage of sensitive information. Whether the system projects the screen in the form of display image copying or by drawing the specified screen display content separately, the solution can accurately filter the Layer displayed on the screen, providing users with a more humanized user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The present invention is a flowchart of a method for filtering Android display screen layers according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] See also Figure 1 , shown is a method for filtering Android display screen layers implemented in one embodiment of the present invention, comprising the following steps:

[0030] S10, the display sends a vertical synchronization signal Vsync to the hardware composition component HWC of the display layer of the Android system, and HWC calls back the synchronization method onComposerHalVsync of the primitive processing server SurfaceFlinger to prepare for screen display content synchronization;

[0031] S20, after the Android system receives the Vsync signal, SurfaceFlinger recalculates the display content of all current screens and obtains the primitive transfer unit Surface and Surface handle SurfaceControl created by the application display window; SurfaceFlinger uses the obtained Surface and SurfaceControl to construct a Layer, and adds the constructed Layer to the current system screen Z-axis Layer management stack; the screens among all current screens include the main screen, secondary screen and virtual screen.

[0032] S30, SurfaceFlinger registered in the screen drawing scheduler Scheduler generates a Vsync processing signal at a fixed interval and calls its signal processing method onTransact to notify the message queue MessageQueue to process the screen display refresh event, and MessageQueue calls back the message receiving function onMessageReceived of SurfaceFlinger to send the message back to SurfaceFlinger;

[0033] S40, after receiving the screen display refresh message, SurfaceFlinger executes the screen refresh processing function onMessageRefresh. The refresh processing function creates a screen synthesis refresh parameter object, then collects all the drawing data output terminals of the current Android system and adds them to the output stack of the screen synthesis refresh parameter object, then traverses all the layers in the screen Z-axis Layer management stack and adds them to the Layer stack of the synthesis refresh parameter object; when it is necessary to filter the layer in a targeted manner, traverse the elements of the screen Z-axis Layer management stack; SurfaceFlinger passes the prepared image display data to the image drawing function present of the image synthesis engine CompositionEngine; the image display data includes all Output, Layer and Layer position management stacks. After passing them to CompositionEngine, the layer is drawn to the display of the corresponding output object through the image drawing function present.

[0034] S50, when executing the present of CompositionEngine, first traverse the output stack, Output calls the prepare function prepare and passes in the screen composition refresh parameter, and draws and renders the current Output display Layer elements in sequence;

[0035] S60, Output traverses the layer stack passed in. In addition, the screen Output object provides a name attribute as a mark to distinguish the screen. During the entire traversal process, the relationship between the layer and the current output must be determined. After confirming that the layer belongs to the current screen output, it will be drawn and rendered; when the screen output filtering scheme needs to be locked, the Output filters the drawing layer of the specified screen output. The relationship is that when the device supports multi-screen display (there are multiple Output objects), after collecting all the layers, the Android display system will determine which output it belongs to based on the layer display configuration information.

[0036] S70, after drawing all the output display contents of the current system one by one, the drawn data are synthesized and displayed on the corresponding screen through HWC.

[0037] In a method for filtering Android display screen layers in a specific application example, in S40, when a layer needs to be filtered in a targeted manner, the elements of the layer management stack on the screen Z axis are traversed, including:

[0038] Take out the Layer element from the screen Z-axis Layer management stack;

[0039] Get the Layer name information and compare it with the target string;

[0040] When the layer name is equal to or contains the target field, the layer is not added to the layer stack of the synthesis refresh parameter object and the layer element is taken out from the screen Z-axis layer management stack to take out the next layer element;

[0041] Adds the Layer to the Layer stack of the Composition Refresh Parameters object.

[0042] The target string is: Layer is the graphic element displayed by the Android system. Each Layer has a window associated with the corresponding application layer Activity interface. In Android, each Activity is actually associated with a window, which is responsible for presenting content on the screen and processing view and input events. All Activities of an application normally run in the same process, and the running Activity information is in the form of process name + Activity program class name. The application process name is the application package name. Each Layer contains the full name of the Activity to which it belongs, which also includes the package name. Therefore, you can filter by matching the layer name information according to the corresponding application package name, or more accurately match the layer by the package name + activity class name. The above application package name and package name + activity class name can be used as the target string to match the layer name information.

[0043] In a method for filtering Android display screen layers in a specific application example, in S60, the logic of the output filtering the drawing layer of the specified screen output includes:

[0044] Compare the screen Output object name. If it is the target screen Output that needs to be filtered by Layer, proceed to the next step. Otherwise, draw normally.

[0045] Compare the name of the layer that needs to be drawn. If it is the target layer that needs to be filtered, do not draw it. Otherwise, draw it normally.

[0046] Follow the above steps to traverse the next Layer.

[0047] In a specific application example, when the screen output filtering scheme needs to be locked, when the user sets the Layer information that needs to be filtered through the software program interface or the configuration file list, when the CompositionEngine executes the image drawing function Present, the Layer information that needs to be filtered is used to match the layer information that is about to be drawn and rendered. If a match is found, the current Layer object will not be rendered or drawn, and the current Layer will be skipped directly, and the next one will be judged.

[0048] The target screen Output is: Each output object in the system is assigned with attributes such as ID number, internal, name, etc. The output object that needs to be filtered by Layer can be distinguished based on the name, and the Output object can also be further determined by the ID number.

[0049] After the method of filtering the Android display screen layers set above introduces the layer filtering scheme into the underlying image display system of Android, the system can lock the target screen according to the Output name, and further determine the target layer to be filtered by comparing the Layer name, and then skip the Output drawing process of the Layer with the specified name, thereby filtering the Layer that needs to be drawn on the target screen, thereby achieving the effect of screen display layer filtering. After the Android system integrates the display screen layer filtering scheme, it better improves the user experience in the fields of wireless screen projection and screen recording. When HWC synthesizes the remaining layer data onto the corresponding Output screen, it not only effectively limits the behavior of automatic pop-up windows of the application, but also meets the needs of privacy protection and rejection of pop-up windows, providing users with a more user-friendly functional scenario in the field of Android screen projection office.

[0050] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A method for filtering Android display screen layers, characterized in that: The following steps are involved: The display sends a vertical synchronization signal Vsync to the hardware composition component HWC of the Android system display layer, and HWC calls back the synchronization method onComposerHalVsync of the primitive processing server SurfaceFlinger to prepare for screen display content synchronization; After the Android system receives the Vsync signal, SurfaceFlinger recalculates the display content of all current screens and obtains the primitive transfer unit Surface and Surface handle SurfaceControl created by the application display window; SurfaceFlinger uses the obtained Surface and SurfaceControl to construct the Layer, and adds the constructed Layer to the current system screen Z-axis Layer management stack; At the same time, SurfaceFlinger registered in the screen drawing scheduler Scheduler generates Vsync processing signals at fixed intervals and calls its signal processing method onTransact to notify the message queue MessageQueue to process the screen display refresh event. MessageQueue calls back SurfaceFlinger's message receiving function onMessageReceived to send the message back to SurfaceFlinger; After receiving the screen display refresh message, SurfaceFlinger executes the screen refresh processing function onMessageRefresh. The refresh processing function creates a screen composition refresh parameter object, then collects all the drawing data output terminals of the current Android system and adds them to the output stack of the screen composition refresh parameter object, then traverses all the layers in the screen Z-axis Layer management stack and adds them to the Layer stack of the composition refresh parameter object; when it is necessary to filter the layer in a targeted manner, it traverses the elements of the screen Z-axis Layer management stack; SurfaceFlinger passes the prepared image display data to the image drawing function present of the image composition engine CompositionEngine; When executing the present of CompositionEngine, the output stack is traversed first. Output calls the prepare function and passes in the screen composition refresh parameters, and draws and renders the current Output display Layer elements in sequence; Output traverses the passed-in Layer stack. In addition, the screen Output object provides a name attribute as a mark to distinguish the screen. During the entire traversal process, the relationship between the Layer and the current Output must be determined. After confirming that the Layer belongs to the current screen Output, it will be drawn and rendered. When the screen output filtering scheme needs to be locked, Output filters the drawing Layer of the specified screen Output. After drawing all the output display contents of the current system screen one by one, the drawn data is synthesized and displayed on the corresponding screen through HWC.

2. The method for filtering Android display screen layers according to claim 1, characterized in that: The screens among all current screens include a main screen, a secondary screen and a virtual screen.

3. The method for filtering Android display screen layers according to claim 1 or 2, characterized in that: When the Layer needs to be filtered in a directional manner, the Layer management stack elements of the screen Z axis are traversed, including: Take out the Layer element from the screen Z-axis Layer management stack; Get the Layer name information and compare it with the target string; When the layer name is equal to or contains the target field, the layer is not added to the layer stack of the synthesis refresh parameter object and the layer element is taken out from the screen Z-axis layer management stack to take out the next layer element; Adds the Layer to the Layer stack of the Composition Refresh Parameters object.

4. The method for filtering Android display screen layers according to claim 1 or 2, characterized in that: The logic of drawing the Layer of the Output filter specified screen Output includes: Compare the screen Output object name. If it is the target screen Output that needs to be filtered by Layer, proceed to the next step. Otherwise, draw normally. Compare the name of the layer that needs to be drawn. If it is the target layer that needs to be filtered, do not draw it. Otherwise, draw it normally. Follow the above steps to traverse the next Layer.

Images