A multi-screen display method, device, related medium and vehicle

Abstract

The embodiment of the application discloses a multi-screen display method, device, related medium and vehicle. The method is applied to a single-host display system, the single-host display system comprises a plurality of application running environments, the method can comprise: for any application running environment, synthesizing display data based on an authorized display process corresponding to the application running environment to obtain to-be-displayed image data; and calling a direct rendering manager to perform rendering processing on the to-be-displayed image data of the application running environment to display in a display screen corresponding to the application running environment. By implementing the method, each application running environment performs rendering processing on display data in the application running environment through a corresponding authorized process, and displays the rendered data on a display screen corresponding to the application running environment, which helps to improve the isolation between display processes corresponding to each display screen, thereby ensuring the stability of display data processing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular, relates to a multi-screen display method and device, related medium and vehicle. BACKGROUND

[0002] With the popularization of automobile intelligent technology and the rapid development of intelligent cockpits, there are more and more display screens in the automobile cockpit. At present, in the use scene of single host and multiple display screens, layers generated in SurfaceFlinger can classify data into content to be displayed by different display screens. However, these display screens are based on the same display process to process display content. When the display content in one of the display screens causes the display process to be abnormal, the display effect of other display screens will be affected, and there is a problem of poor display process isolation, which has adverse effects on data stability and security. SUMMARY

[0003] One purpose of the present application is to provide a multi-screen display method, device, related medium and vehicle, which can improve the isolation of the display process corresponding to each display screen, thereby ensuring the stability of the display data corresponding to each display screen.

[0004] In a first aspect, an embodiment of the present application provides a multi-screen display method. The method is applied to a single host display system, which can include multiple application running environments. The method can include:

[0005] For any application running environment, based on the authorized display process corresponding to the application running environment, display data is synthesized to obtain to-be-displayed image data;

[0006] A direct rendering manager is called to render and process the to-be-displayed image data of the application running environment, so as to display the to-be-displayed image data on the display screen corresponding to the application running environment.

[0007] In one possible implementation, each application running environment is a container system to realize the isolation of the application running environment.

[0008] In another possible implementation, any application running environment can include a display synthesis process.

[0009] Before the display data is synthesized based on the authorized display process corresponding to the application running environment to obtain the to-be-displayed image data, the method can further include:

[0010] The display synthesis process is authorized and verified. When the verification is passed, the display synthesis process is determined to be the authorized display process corresponding to the application running environment, and the authorized display process has the rendering resource calling permission.

[0011] In another possible implementation, the authorization check on the display composition process can include:

[0012] The display composition process sends an authorization request to the direct rendering manager;

[0013] The direct rendering manager determines the authority of the display composition process according to the authorization request.

[0014] In another possible implementation, the authorization request can include identification information of the display composition process and identity information of the target display screen.

[0015] In another possible implementation, the direct rendering manager determines the authority of the display composition process according to the authorization request can include:

[0016] When the direct rendering manager determines that the display resource corresponding to the target display screen exists, and the identification information of the display composition process matches the identification information of the preset display process of the target display screen, it is determined that the display composition process is an authorized display process.

[0017] In another possible implementation, the method can further include:

[0018] When the direct rendering manager determines that the display resource corresponding to the target display screen does not exist, or the identification information of the display composition process does not match the identification information of the preset display process of the target display screen, the authorization check is terminated.

[0019] In another possible implementation, each application running environment can include an authorized display process, and for all authorized display processes corresponding to all application running environments, all authorized display processes can include a first authorized display process and at least one second authorized display process, wherein the first authorized display process is the first process authorized by the direct rendering manager.

[0020] In another possible implementation, the first authorized display process has more rendering resource calling authority than the second authorized display process.

[0021] In another possible implementation, any application running environment can include a plurality of image data corresponding to a plurality of application programs;

[0022] Based on the authorized display process corresponding to the application running environment, the display data composition obtains the to-be-displayed image data, which can include:

[0023] Using the authorized display process corresponding to the application running environment, the plurality of image data is composed into the to-be-displayed image data.

[0024] In a second aspect, an apparatus for multi-screen display is provided. The apparatus is applied to a single-host display system, which can include a plurality of application running environments. The apparatus can include:

[0025] an application module configured to, for any application running environment, synthesize display data based on an authorized display process corresponding to the application running environment to obtain image data to be displayed;

[0026] a rendering module configured to invoke a direct rendering manager to render the image data to be displayed of the application running environment to be displayed on a display screen corresponding to the application running environment.

[0027] In a possible implementation, each application running environment is a container system to realize isolation of the application running environment.

[0028] In another possible implementation, any application running environment can include a display synthesis process.

[0029] The apparatus can further include a verification module.

[0030] The verification module is configured to verify the display synthesis process, and determine that the display synthesis process is an authorized display process corresponding to the application running environment when the verification is passed, and the authorized display process has a rendering resource invoking permission.

[0031] In another possible implementation, the apparatus can further include:

[0032] The rendering module is further configured to send an authorization request to the direct rendering manager.

[0033] The verification module is further configured to determine the permission of the display synthesis process according to the authorization request.

[0034] In another possible implementation, the authorization request can include identification information of the display synthesis process and identity information of a target display screen.

[0035] In another possible implementation, the apparatus can further include:

[0036] The verification module is further configured to determine that the display synthesis process is an authorized display process when the direct rendering manager determines that the display resource corresponding to the target display screen exists, and the identification information of the display synthesis process matches the identification information of a preset display process of the target display screen.

[0037] In another possible implementation, the method can further include:

[0038] The checking module can also be configured to terminate the authorization check when the direct rendering manager determines that the display resource corresponding to the target display screen does not exist, or the identification information of the display composition process does not match the identification information of the preset display process of the target display screen.

[0039] In another possible implementation, each application running environment can include an authorized display process, and all authorized display processes corresponding to all application running environments can include a first authorized display process and at least one second authorized display process, where the first authorized display process is the first process authorized by the direct rendering manager.

[0040] In another possible implementation, the first authorized display process has more rendering resource calling permissions than the second authorized display process.

[0041] In another possible implementation, any application running environment can include a plurality of image data corresponding to a plurality of application programs.

[0042] The apparatus can also include a computing module.

[0043] The computing module can be configured to synthesize the plurality of image data into the to-be-displayed image data by using the authorized display process corresponding to the application running environment.

[0044] In a third aspect, an embodiment of the present application provides a multi-screen display apparatus, which can include the following parts: a processor, a memory, and a bus.

[0045] The processor and the memory are connected through the bus, where the memory is configured to store a set of program codes, and the processor is configured to call the program codes stored in the memory and execute the method as described in the first aspect.

[0046] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which includes:

[0047] The computer readable storage medium stores instructions, and when the instructions are run on a computer, the method as described in the first aspect is implemented.

[0048] In a fifth aspect, an embodiment of the present application provides a vehicle, which can include the multi-screen display apparatus as described in the second aspect or the third aspect and the computer readable storage medium as described in the fourth aspect.

[0049] It can be seen that the application can assign corresponding display screens for different application running environments, and each application running environment can use a respective authorized display process to perform rendering operation on display image data, which helps to improve the isolation of the display processes corresponding to different display screens, thereby improving the stability of the display data corresponding to each display screen, and the embodiment method of the application can be applied to various system platforms, can meet various operation requirements of users, and improves the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0051] Figure 1 An architecture schematic diagram of a single-host display system provided by the embodiment of the present application;

[0052] Figure 2 A flowchart of a multi-screen display method provided by the embodiment of the present application;

[0053] Figure 3 A flowchart of a method for a display composition process to obtain resource calling permission provided by the embodiment of the present application;

[0054] Figure 4 A flowchart of a permission verification provided by the embodiment of the present application;

[0055] Figure 5 A multi-screen display device provided by the embodiment of the present application;

[0056] Figure 6 Another multi-screen display device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0058] The terms "first", "second", "third", and "fourth" and the like in the description and in the claims of the present application and the accompanying drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. The terms "comprises", "comprising", "includes", "including" and the like are to be construed open- ended, meaning that they include the listed steps or elements, but not excluding others. For example, a process, method, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those listed steps or elements, but can include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

[0059] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment can be included in at least one embodiment of the application. The appearances of the phrase "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that any of the embodiments described herein can be combined with any of the other embodiments unless specifically noted otherwise.

[0060] In order to better understand the technical solutions of the embodiments of the present application, a single-host display system that the embodiments of the present application can involve is introduced first. Please refer to Figure 1 , a schematic diagram of an architecture of a single-host display system provided by the embodiments of the present application. The single-host display system can include multiple application running environments (such as Figure 1 a first application running environment 1, a second application running environment 2, and a third application running environment 3), an application layer code library (libdrm library) 110, and a direct rendering manager (DRM) 120, to drive multiple display screens (such as Figure 1 a first display screen 1, a second display screen 2, and a third display screen 3) to display images.

[0061] Among them, each application running environment contains an application set and an authorized display process. For example, the first application running environment 1 can include a first application set 1 and a first authorized display process 1, the second application running environment 2 can include a second application set 2 and a second authorized display process 2, and the third application running environment 3 can include a third application set 3 and a third authorized display process 3. Each application set can include multiple applications, and it should be noted that the number of applications contained in each application set is not limited here. Further, each authorized display process is in a namespace (Namespace) and / or control group (Control groups, C groups) in its corresponding application running environment. Figure 1 Each authorized display process in Figure 1The first authorization request 1, the second authorization request 2, and the third authorization request 3 in the process obtain the corresponding rendering resource access permissions (such as...). Figure 1 The system obtains the first rendering resource access permission 1, the second rendering resource access permission 2, and the third rendering resource access permission 3 in the library, and uses these rendering resource access permissions to indirectly obtain the rendering resources of the direct rendering manager 120 through the libdrm library 110.

[0062] A display screen is a computer's I / O device, or output device, which is a tool that displays electronic files onto the screen via specific transmission equipment. Based on different display technologies, display screens can be categorized as follows: Cathode Ray Tube (CRT) screens, Plasma Display Panel (PDP) screens, Liquid Crystal Display (LCD) screens, Organic Light Emitting Diode (OLED) screens, and Micro Light Emitting Diode (MicroLED) screens; based on their shape, they can be categorized as flat panel screens, curved screens, and 3D screens; and based on their application, they can be categorized as gaming screens, office screens, and graphic design screens.

[0063] Direct Render Manager 120 is a display driver framework at the Linux operating system kernel level, which can include, for example... Figure 1 The rendering system includes multiple rendering resources such as the Display Output Context (CRTC) (e.g., CRTC1211, CRTC1212, and CRTC1213), Planes (e.g., Planes1221, Planes1222, and Planes1223), Encoders (e.g., Encoder1231, Encoder1232, and Encoder1233), and Connectors (e.g., Connector1241, Connector1242, and Connector1243). The CRTC reads pixel data from the scan buffer and generates video mode timing signals using a phase-locked loop (PLL) circuit. It internally connects to the Framebuffer address and externally connects to the Encoder. The CRTC scans the content on the Framebuffer, overlays the Planes' content, and then transmits it to the Encoder.

[0064] Planes and Framebuffer are memory addresses, and there can be multiple Planes. The final image provided to the CRTC for scanning is actually often a combination of Framebuffer and Planes.

[0065] Encoder, which can encode (or convert) the pixel of the memory into the signal required by the display screen. The interaction between it and the CRTC can be called modesetting, which contains display information such as color mode, image size and image display timing.

[0066] Connector, which will connect a physical display output device (such as VGA, DVI, FPD-Link, HDMI, DisplayPort or S-Video, etc. Corresponding to the display screen in the embodiment of the present application), and store information related to the currently physically connected display output device (such as connection status, EDID data, DPMS status or supported video format).

[0067] Furthermore, between the authorized display process and the direct rendering manager, there can also be a libdrm library 110, which can be used to encapsulate the rendering resource interface provided by the direct rendering manager. The authorized display process can indirectly call the rendering resource of the direct rendering manager through the libdrm library.

[0068] Next, the method of the present application will be described in detail. Figures 2-3 The method of the present application will be described in detail.

[0069] Please refer to Figure 2 A flowchart of a multi-screen display method provided by the embodiment of the present application is shown, which is applied to a single-host display system, which can include multiple application running environments.

[0070] The method can include the following steps:

[0071] S201, for any of the application running environments, based on the authorized display process corresponding to the application running environment, the display data synthesis is performed to obtain the to-be-displayed image data.

[0072] The display data can be corresponding display data of image information obtained by an application program in the application running environment, or can be corresponding display data of a page presented to a user and related to the application program in the application running environment. For example, when the application program in the application running environment is a camera, the display data can be corresponding display data of an image taken by the user; when the application program in the application running environment is a chat software, the display data can be corresponding display data of a login interface of the chat software, corresponding display data of a chat box page, corresponding display data of a chat list page, or corresponding display data of a contact page, and the like.

[0073] Optionally, the single-host display system can further include a plurality of display screens and a direct rendering manager, each of the plurality of application running environments corresponds to one of the plurality of display screens, each of the application running environments can include an application set and an authorized display process, and each of the application sets can include one or more application programs.

[0074] In a possible implementation, each of the application running environments is a container system to realize application running environment isolation.

[0075] For example, each of the application running environments is a Linux container system (LXC, Linux Container), and multiple systems are realized through Namespace and Cgroup.

[0076] In another possible implementation, any of the application running environments can include a display composition process (HWC, Hardware Composer).

[0077] Before the display data is synthesized based on the authorized display process of the application running environment to obtain the to-be-displayed image data, the method can further include:

[0078] The display composition process is authorized to check, so as to determine that the display composition process is the authorized display process of the application running environment when the check passes, and the authorized display process has a rendering resource calling permission.

[0079] For a specific authorization checking process, reference can be made to Figure 3 and related embodiments.

[0080] S202, calling the direct rendering manager to perform rendering processing on the to-be-displayed image data of the application running environment, so as to display the to-be-displayed image data in the display screen corresponding to the application running environment.

[0081] For example, the authorized display process can send the rendered image data to the display screen corresponding to the application running environment through a connector interface in the direct rendering manager, such as Figure 1The first application running environment 1 corresponds to the first display screen 1.

[0082] In another possible implementation, any application running environment can include a plurality of image data corresponding to a plurality of application programs;

[0083] Based on the authorized display process corresponding to the application running environment, the display data is synthesized to obtain the to-be-displayed image data, which can include:

[0084] The plurality of image data is synthesized into to-be-displayed image data by using the authorized display process corresponding to the application running environment.

[0085] Specifically, the display data can include a plurality of image layers (corresponding to the plurality of image data described above), which need to be integrated into a complete image data by the layer synthesis resource of the direct rendering manager, and the format of the complete image data is adjusted according to the display requirements (such as color mode and / or resolution of the display screen, etc.) of the display screen corresponding to the application running environment, and finally the to-be-displayed image data is obtained.

[0086] It can be seen that the embodiments of the present application can isolate the rendering operation of the display data corresponding to each application running environment and display screen, which helps to ensure that each display screen can normally present the target image data, and also ensures that the rendering process of the display data of each application in each application running environment is normally carried out, improves the data display stability of each display screen, and further improves the user experience.

[0087] Further, please refer to Figure 3 A flowchart of a method for obtaining a rendering resource call permission of a display synthesis process provided by the embodiments of the present application, wherein any application running environment can include a display synthesis process;

[0088] The method can include the following steps:

[0089] S301, the display synthesis process sends an authorization request to the direct rendering manager.

[0090] In another possible implementation, the authorization request can include identification information of the display synthesis process and identity information of the target display screen.

[0091] The target display screen is the display screen that the application running environment corresponding application set is expected to present image data. The identity information of the target display screen can include the serial number information of the target display screen in the plurality of display screens.

[0092] For example, if there are a first display composition process 1 (with identification information com.xxx.yyy::111), a second display composition process 2 (with identification information com.xxx.yyy::112), and a third display composition process 3 (with identification information com.xxx.yyy::113) in the first application running environment, and the serial number information of the target display screen is 1, if the first application set corresponding to the first application running environment is expected to present image data on the target display screen, an authorization request similar to token="displayId=1&&processName=com.xxx.yyy::111" can be generated, where displayId is used to specify the serial number information of the target display screen, displayId=1 indicates the display screen with serial number 1, processName is used to specify the identification information of the display composition process expected to apply for the rendering resource call permission of the direct rendering manager, and processName=com.xxx.yyy::111 indicates the display composition process with identification information com.xxx.yyy::111. It can be understood that the meaning of the authorization request is "the display composition process with identification information com.xxx.yyy::111 is expected to transmit each image data of the first application set corresponding to the first application running environment to the display screen with serial number 1".

[0093] In S302, the direct rendering manager determines the permission of the display composition process according to the authorization request.

[0094] It should be noted that before the direct rendering manager determines the permission of the display composition process according to the authorization request, it is necessary to establish the association relationship between the display composition process in the application running environment and the display screen corresponding to the application running environment. The display screen corresponding to the application running environment will identify the display composition process associated therewith as a preset display process, and save the identification information of the display composition process in the process white list. More specifically, the establishment step of the association relationship can be performed by the staff before the single-host display system is put into use, and the user can also customize the preset display process of each display screen. For example, if the first display composition process (with identification information com.xxx.yyy::111) in the existing first application running environment is associated with the first display screen corresponding to the first application running environment, the first display screen will identify the first display composition process as a preset display process, and save the identification information com.xxx.yyy::111 in the process white list.

[0095] In another possible implementation, the direct rendering manager determines the permission of the display composition process according to the authorization request, which can include:

[0096] When the direct rendering manager judges that the display resource corresponding to the target display screen exists, and the identification information of the display composition process matches the identification information of the preset display process of the target display screen, it is determined that the display composition process is an authorized display process.

[0097] In another possible implementation, the method can further include:

[0098] When the direct rendering manager judges that the display resource corresponding to the target display screen does not exist, or the identification information of the display composition process does not match the identification information of the preset display process of the target display screen, the authorization check is terminated.

[0099] Exemplarily, assuming that the existing first display screen 1 (the preset display process is the fourth display composition process 4, the identification information of the fourth display composition process 4 is com.xxx.yyy::114, and the fourth display composition process 4 belongs to the fourth application running environment 4), the second display screen 2 (the preset display process is the fifth display composition process 5, the identification information of the fifth display composition process 5 is com.xxx.yyy::115, and the fifth display composition process 5 belongs to the fifth application running environment 5), and the third display screen 3 (the preset display process is the sixth display composition process 6, the identification information of the sixth display composition process 6 is com.xxx.yyy::116, and the sixth display composition process 6 belongs to the sixth application running environment 6), if the authorization request generated by the sixth display composition process 6 is token=“displayId=3&&processName=com.xxx.yyy::116”, it is firstly confirmed that the display screen with the serial number 3 exists, and the display composition process with the identification information com.xxx.yyy::116 is the preset display process of the third display screen 3, so it is considered that the authorization request of the sixth display composition process 6 is legal, and the rendering resource calling permission of the rendering resource of the direct rendering manager is issued to the sixth display composition process 6. Further, it can be confirmed that the sixth display composition process 6 is the authorized display process of the sixth application running environment 6, and the third display screen 3 is the display screen corresponding to the application set image data expected to be presented by the sixth application running environment 6; if the authorization request generated by the fifth display composition process 5 is token=“displayId=6&&processName=com.xxx.yyy::115”, it is firstly confirmed that the display screen with the serial number 6 does not exist, so the authorization verification about token=“displayId=6&&processName=com.xxx.yyy::115” can be terminated; if the authorization request generated by the fifth display composition process 5 is token=“displayId=3&&processName=com.xxx.yyy::115”, it is firstly confirmed that the display screen with the serial number 3 exists, but the display composition process with the identification information com.xxx.yyy::115 is not the preset display process of the third display screen 3, so the authorization verification about token=“displayId=6&&processName=com.xxx.yyy::115” can be terminated.

[0100] In another possible implementation, each application running environment can include one authorized display process, and for all authorized display processes corresponding to all application running environments, all authorized display processes can include one first authorized display process and at least one second authorized display process, wherein the first authorized display process is the first process that passes the authorization verification of the direct rendering manager.

[0101] In another possible implementation, the first authorized display process has more rendering resource invocation permissions than the second authorized display process.

[0102] Exemplarily, in the single-host display system, multiple application running environments each have an authorized display process, but the time nodes at which the authorized display processes obtain rendering resource invocation permissions from the direct rendering manager have a sequence. The first process that passes the authorization check of the direct rendering manager can be regarded as the first authorized display process, and the other authorized display processes can be regarded as the second authorized display processes. The first authorized display process can have DRM-Master permissions, and the second authorized display processes can have DRM-Auth permissions. The direct rendering manager corresponding to the DRM-Master permissions has more rendering resource interfaces than the direct rendering manager corresponding to the DRM-Auth permissions. For specific permission differences, refer to the permission differences between DRM-Master and DRM-Auth in the prior art, which are not described herein. Possibly, the direct rendering manager also needs to determine whether the instruction for invoking the rendering resource by the authorized display process is within the rendering resource invocation permissions of the authorized display process. If it is within the rendering resource invocation permissions, the authorized display process is allowed to invoke the corresponding rendering resource. If it is not within the rendering resource invocation permissions, the resource invocation request of the authorized display process is rejected.

[0103] Exemplarily, assume that the sixth display composition process 6, the seventh display composition process 7, the eighth display composition process 8, and the ninth display composition process 9 have rendering resource invocation permissions of the direct rendering manager, and the direct rendering manager issues the rendering resource invocation permissions in the following sequence: the eighth display composition process 8→the ninth display composition process 9→the sixth display composition process 6→the seventh display composition process 7. Then, the eighth display composition process 8 has DRM-Master permissions, and the sixth display composition process 6, the seventh display composition process 7, and the ninth display composition process 9 each have DRM-Auth permissions.

[0104] It should be noted that in the embodiments of the present application, each application running environment includes an authorized display process, and the display data of all applications in each application running environment is rendered by the authorized display process in the application running environment. Possibly, the authorized display process can send an authorization permission to the direct rendering manager each time the vehicle (or the application running environment) is started, and the direct rendering manager also checks the authorization request of the authorized display process. Alternatively, after the direct rendering manager checks the permissions of the display processes, the display processes (or the authorized display processes) remain in the same authorization state. For a specific permission checking process, refer to the corresponding content. Figure 3 The corresponding content.

[0105] It can be seen that the method of the embodiment of the application can determine whether the authorization request of each display process is legal according to the identity information of the existing display screen and the preset display process of the display screen, which helps to reduce the communication interaction between processes, thereby improving the efficiency of the direct rendering manager in verifying the authorization request of the display process. The method of the embodiment of the application can also issue different rendering resource call permissions to the display process according to the order in which the display process obtains the rendering resource call permission, which helps to improve the security of the display data corresponding to each application running environment.

[0106] For example, please refer to Figure 4 A flowchart of permission verification provided by the embodiment of the application. The flow of permission verification can be as follows:

[0107] For the display composition process, there can be:

[0108] S401, generating an authorization request according to the identity information of the target display screen and the identity information of itself.

[0109] S402, sending the authorization request to the direct rendering manager.

[0110] For the direct rendering manager, there can be:

[0111] S403, determining whether the identity information of the target display screen in the authorization request matches the identity information of the existing display screen.

[0112] If the identity information of the target display screen does not match the identity information of the existing display screen, S404 considers that the authorization request of the display composition process is not legal, and rejects the authorization request of the display composition process.

[0113] If the identity information of the target display screen matches the identity information of the existing display screen, S405 determines whether the display composition process is a preset display process of the target display screen.

[0114] If the display composition process is not a preset display process of the target display screen, S406 considers that the authorization request of the display composition process is not legal, and rejects the authorization request of the display composition process.

[0115] If the display composition process is a preset display process of the target display screen, S407 determines whether the display composition process is the first display process to which the direct rendering manager issues a rendering resource call permission.

[0116] If the display composition process is the first display process to which the direct rendering manager issues a rendering resource call permission, S408 issues DRM-Master permission to the display composition process.

[0117] If the display compositing process is not the first display process to which the direct rendering manager issues the rendering resource call permission, S409 issues a DRM-Auth permission to the display compositing process.

[0118] The device related to the embodiments of the present application is described below with reference to the drawings.

[0119] Please refer to Figure 5 The device for multi-screen display provided by the embodiments of the present application can be included in a single-host display system, the single-host display system can include a plurality of application running environments, and the device can include an application module 501 and a rendering module 502.

[0120] Optionally, the single-host display system can further include a plurality of display screens and a direct rendering manager, each of the plurality of application running environments corresponds to one of the plurality of display screens, and each application running environment can include an application set and an authorized display process, and each application set can include one or more application programs.

[0121] The application module 501 can be configured to, for any application running environment, perform display data composition based on the authorized display process corresponding to the application running environment to obtain to-be-displayed image data.

[0122] The rendering module 502 can be configured to call the direct rendering manager to perform rendering processing on the to-be-displayed image data of the application running environment, so as to display the to-be-displayed image data on the display screen corresponding to the application running environment.

[0123] In a possible implementation, each application running environment is a container system to realize application running environment isolation.

[0124] In another possible implementation, any application running environment can include a display compositing process.

[0125] The device can further include a verification module 503.

[0126] The verification module 503 can be configured to perform authorization verification on the display compositing process, so as to determine that the display compositing process is the authorized display process corresponding to the application running environment when the verification passes, and the authorized display process has rendering resource call permission.

[0127] In another possible implementation, the device can further include:

[0128] The rendering module 502 can be further configured to send an authorization request to the direct rendering manager.

[0129] The verification module 503 can be further configured to determine the permission of the display compositing process according to the authorization request.

[0130] In another possible implementation, the authorization request can include identification information of the display composition process and identity information of the target display screen.

[0131] In another possible implementation, the apparatus can further include:

[0132] The checking module 503 can be further configured to determine that the display composition process is an authorized display process when the direct rendering manager determines that the display resource corresponding to the target display screen exists and the identification information of the display composition process matches the identification information of the preset display process of the target display screen.

[0133] In another possible implementation, the method can further include:

[0134] The checking module 503 can be further configured to terminate the authorization check when the direct rendering manager determines that the display resource corresponding to the target display screen does not exist or the identification information of the display composition process does not match the identification information of the preset display process of the target display screen.

[0135] In another possible implementation, each application running environment can include one authorized display process, and for all authorized display processes corresponding to all application running environments, all authorized display processes can include one first authorized display process and at least one second authorized display process, wherein the first authorized display process is the first process that passes the authorization check by the direct rendering manager.

[0136] In another possible implementation, the first authorized display process has more rendering resource calling permissions than the second authorized display process.

[0137] In another possible implementation, any application running environment can include multiple image data corresponding to multiple application programs.

[0138] In another possible implementation, the application module 501 can be further configured to synthesize the multiple image data into the to-be-displayed image data by using the authorized display process corresponding to the application running environment.

[0139] Please refer to Figure 6 Another composition diagram of an apparatus for multi-screen display is provided in the embodiments of the present application, and the apparatus can include:

[0140] The processor 610, the memory 620 and the I / O interface 630. The processor 610, the memory 620 and the I / O interface 630 can be communicatively connected, the memory 620 is configured to store instructions, and the processor 610 is configured to execute the instructions stored in the memory 620 to implement the above Figure 2 and Figure 3 corresponding method steps.

[0141] The processor 610 is configured to execute instructions stored in the memory 620 to control the I / O interface 630 to receive and send signals, and complete the steps in the above method. The memory 620 can be integrated in the processor 610, or can be arranged separately from the processor 610.

[0142] The memory 620 can further include a storage system 621, a cache 622 and a RAM 623. The cache 622 is a first-level memory between the RAM 623 and the CPU, composed of static storage chips (SRAM), with a small capacity but much higher speed than the main memory, close to the speed of the CPU; the RAM 623 is an internal memory directly exchanging data with the CPU, which can be read and written at any time (except for refreshing), and has very fast speed, and is usually used as a temporary data storage medium for operating systems or other programs running. The three realize the function of the memory 620.

[0143] As an implementation manner, the function of the I / O interface 630 can be realized by a transceiver circuit or a dedicated transceiver chip. The processor 610 can be realized by a dedicated processing chip, a processing circuit, a processor or a general-purpose chip.

[0144] As another implementation manner, the device provided by the embodiment of the present application can be realized by using a general-purpose computer. That is, the program codes realizing the functions of the processor 610 and the I / O interface 630 are stored in the memory 620, and the general-purpose processor realizes the functions of the processor 610 and the I / O interface 630 by executing the codes in the memory 620.

[0145] The concepts, explanations, detailed descriptions and other steps related to the technical solutions provided by the embodiments of the present application involved in the device are described in the foregoing method or the description of the method steps performed by the device in other embodiments, and will not be repeated here.

[0146] As another implementation manner of the embodiment, a computer readable storage medium is provided, which stores instructions, and the instructions are executed to perform the method in the method embodiment.

[0147] As another implementation manner of the embodiment, a computer program product is provided, which includes instructions, and the instructions are executed to perform the method in the method embodiment.

[0148] Those skilled in the art can understand that, in order to facilitate the description, Figure 6 Only one memory and one processor are shown in the terminal or server. In actual terminal or server, there can be multiple processors and memories. The memory can also be referred to as a storage medium or a storage device, and the embodiments of the present application do not limit this.

[0149] It should be understood that, in the embodiments of the present application, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.

[0150] It should also be understood that the memory mentioned in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0151] It should be noted that when the processor is a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, the memory (storage module) is integrated in the processor.

[0152] It should be noted that the memory described herein is intended to include, but not be limited to, the above types and any other suitable type of memory.

[0153] The bus can include a data bus, a power bus, a control bus, a state signal bus, and the like, in addition to the data bus. However, for the purpose of clear illustration, all the buses are marked as bus in the figure.

[0154] It should also be understood that the first, second, third, fourth, and various numerical numbers involved herein are only for the convenience of differentiation, and do not limit the scope of the application.

[0155] It should be understood that the term "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after it.

[0156] In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor or the instruction in the form of software. The steps of the method disclosed in combination with the embodiments of the application can be directly embodied as hardware processor execution completion, or executed by hardware and software modules in the processor. The software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0157] In various embodiments of the application, the size of the serial number of each process described above does not mean the execution order, and the execution order of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0158] Those of ordinary skill in the art can realize that the various illustrative logical blocks (ILB) and steps described in combination with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether the function is executed in hardware or software mode depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the application.

[0159] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other manners. For example, the above-described device embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and in actual implementation, another division manner can be adopted, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0160] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0161] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0162] In the above embodiments, all or part can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk) and the like.

[0163] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement part or all steps of any one of the methods for multi-screen display described in the above method embodiments.

[0164] The embodiment of the present application further provides a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program. The computer program is operable to cause a computer to implement part or all steps of any one of the methods for multi-screen display described in the above method embodiments.

[0165] The above describes only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for multi-screen display, characterized in that, The method is applied to a single-host display system, which includes multiple application runtime environments. The method includes: For any of the application runtime environments, display data is synthesized based on the authorized display process corresponding to the application runtime environment to obtain the image data to be displayed. Any of the application runtime environments includes a display synthesis process. The direct rendering manager is invoked to render the image data to be displayed in the application runtime environment, so as to display it on the display screen corresponding to the application runtime environment; Before the display data to be displayed is synthesized based on the authorized display process corresponding to the application runtime environment, the method further includes: The display compositing process is authorized for verification. If the verification passes, the display compositing process is determined to be an authorized display process corresponding to the application runtime environment. The authorized display process has the permission to call rendering resources. The authorization verification of the display synthesis process includes: The display compositing process sends an authorization request to the direct rendering manager, the authorization request including the identification information of the display compositing process and the identity information of the target display screen; The direct rendering manager determines the permissions for the display compositing process based on the authorization request.

2. The method according to claim 1, characterized in that, Each of the aforementioned applications runs in a container system to achieve application runtime environment isolation.

3. The method according to claim 2, characterized in that, The direct rendering manager determines the permissions of the display compositing process based on the authorization request, including: When the direct rendering manager determines that the display resources corresponding to the target display screen exist, and the identification information of the display compositing process matches the identification information of the preset display process of the target display screen, then the display compositing process is determined to be an authorized display process.

4. The method according to claim 3, characterized in that, The method further includes: When the direct rendering manager determines that the display resource corresponding to the target display screen does not exist, or the identification information of the display compositing process does not match the identification information of the preset display process of the target display screen, the authorization verification is terminated.

5. The method according to claim 4, characterized in that, Each application runtime environment includes an authorized display process. For all authorized display processes corresponding to all application runtime environments, all authorized display processes include a first authorized display process and at least one second authorized display process, wherein the first authorized display process is the first process to pass the authorization verification of the direct rendering manager.

6. The method according to claim 5, characterized in that, The first authorized display process has more permissions to call rendering resources than the second authorized display process.

7. The method according to claim 6, characterized in that, Any of the application runtime environments mentioned above includes multiple image data corresponding to multiple applications; The process of synthesizing display data based on the authorized display process corresponding to the application runtime environment to obtain the image data to be displayed includes: The authorized display process corresponding to the application runtime environment combines multiple image data into image data to be displayed.

8. A multi-screen display device, characterized in that, The device is applied to a single-host display system, which includes multiple application operating environments. The device includes: The application module is used to synthesize display data based on the authorized display process corresponding to any of the application runtime environments to obtain image data to be displayed, wherein any of the application runtime environments includes a display synthesis process; The rendering module is used to call the direct rendering manager to render the image data to be displayed in the application runtime environment, so as to display it on the display screen corresponding to the application runtime environment. The verification module is used to perform authorization verification on the display compositing process, so as to determine that the display compositing process is an authorized display process corresponding to the application runtime environment when the verification passes, and the authorized display process has the right to call rendering resources; The rendering module is also used to send an authorization request to the direct rendering manager, the authorization request including the identification information of the display compositing process and the identity information of the target display screen; The verification module is also used to determine the permissions of the display synthesis process based on the authorization request.

9. A multi-screen display device, characterized in that, The device includes: The processor, the memory, and the I / O interface are communicatively connected, wherein the memory is used to store a set of program code, and the processor is used to call the program code stored in the memory to execute the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes: The computer-readable storage medium stores instructions that, when executed on a computer, implement the method as described in any one of claims 1-7.

11. A vehicle, characterized in that, The vehicle includes a multi-screen display device as described in claim 8 or 9, or a computer-readable storage medium as described in claim 10.

Images