Multi-screen display method and apparatus, related medium and vehicle

Abstract

A multi-screen display method and apparatus, a related medium, and a vehicle. The method is executed by a single-host display system, and the single-host display system comprises a plurality of application runtime environments. The method comprises: for any application runtime environment, on the basis of an authorization display process corresponding to the application runtime environment, performing display data synthesis to obtain image data to be displayed; and calling a direct rendering manager to perform rendering processing on said image data of the application runtime environment, so as to display same in a display screen corresponding to the application runtime environment.

Description

Multi-screen display method, device, related media and vehicle

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on November 30, 2023, with application number 202311638911.2 and invention name “A method, device, related media and vehicle for multi-screen display”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a method, device, related media, and vehicle for multi-screen display. Background Art

[0003] With the popularization of intelligent automobile technology and the rapid development of smart cockpits, there are more and more display screens in car cockpits. At present, in the use scenario of single host and multiple display screens, data can be classified into content that needs to be displayed on different display screens through the layer generated in SurfaceFlinger. However, these display screens all process display content based on the same display process. When the display content on one of the display screens causes the display process to be abnormal, it will affect the display effect of other display screens. This means that there is a problem of poor isolation of the display process, which has adverse effects on data stability and security.

[0004] Summary of the Invention

[0005] In a first aspect, an embodiment of the present application provides a multi-screen display method, which is performed by a single-host display system. The single-host display system may include multiple application runtime environments. The method may include:

[0006] For any application running environment, display data synthesis is performed based on the authorized display process corresponding to the application running environment to obtain image data to be displayed;

[0007] The direct rendering manager is called to render the image data to be displayed in the application running environment so as to be displayed on the display screen corresponding to the application running environment.

[0008] In one possible implementation, each application runtime environment is a container system to achieve isolation of the application runtime environment.

[0009] In another possible implementation, any application execution environment may include a display composition process;

[0010] Before synthesizing display data based on the authorized display process corresponding to the application running environment to obtain image data to be displayed, the method may further include:

[0011] The display composition process is authorized and checked, so that when the check passes, it is determined that the display composition process is the authorized display process corresponding to the application running environment, and the authorized display process has the rendering resource calling permission.

[0012] In another possible implementation, performing authorization verification on the display composition process may include:

[0013] The display compositing process sends an authorization request to the direct rendering manager;

[0014] The Direct Rendering Manager determines the permissions for the display composition process based on the authorization request.

[0015] In another possible implementation, the authorization request may include identification information of the display synthesis process and identity information of the target display screen.

[0016] In another possible implementation, the direct rendering manager determines the permission for displaying the composition process according to the authorization request, which may include:

[0017] When the direct rendering manager determines that there are display resources corresponding to the target display screen and the identification information of the display composition process matches the identification information of the preset display process of the target display screen, the display composition process is determined to be an authorized display process.

[0018] In another possible implementation, the method may further include:

[0019] When the direct rendering manager determines that there is no display resource corresponding to the target display screen, or the identification information of the display composition process does not match the identification information of the preset display process of the target display screen, the authorization check is terminated.

[0020] In another possible implementation, each application runtime environment may include an authorization display process. For all authorization display processes corresponding to all application runtime environments, all authorization display processes may include a first authorization display process and at least one second authorization display process, wherein the first authorization display process is the first process to pass the direct renderer authorization check.

[0021] In another possible implementation, the first authorized display process has more rendering resource calling permissions than the second authorized display process.

[0022] In another possible implementation, any application execution environment may include multiple image data corresponding to multiple application programs;

[0023] Synthesizing display data based on the authorized display process corresponding to the application running environment to obtain image data to be displayed may include:

[0024] The plurality of image data are synthesized into image data to be displayed using an authorized display process corresponding to the application running environment.

[0025] In a second aspect, an embodiment of the present application provides a multi-screen display device, which is executed by a single-host display system. The single-host display system may include multiple application running environments. The device may include:

[0026] The application module can be used to synthesize display data for any application running environment based on the authorized display process corresponding to the application running environment to obtain image data to be displayed;

[0027] The rendering module can be used to call the direct rendering manager to render the image data to be displayed in the application running environment so as to display it on the display screen corresponding to the application running environment.

[0028] In a possible implementation, the multi-screen display device may further include:

[0029] The verification module can be used to perform authorization verification on the display synthesis process, so as to determine that the display synthesis process is the authorized display process corresponding to the application running environment when the verification passes, and the authorized display process has the rendering resource calling permission.

[0030] In a possible implementation, each of the multiple application execution environments corresponds to one of the multiple display screens included in the single-host display system.

[0031] In a possible implementation, each of the application execution environments includes an application set and an authorization display process, and each application set includes one or more application programs.

[0032] In a third aspect, an embodiment of the present application provides a multi-screen display device, which may include:

[0033] A processor, a memory and an I / O interface, wherein the processor, the memory and the I / O interface are communicatively connected, wherein the memory is used to store a set of program codes, and the processor is used to call the program codes stored in the memory to execute the method as described in the first aspect.

[0034] In a fourth aspect, an embodiment of the present application provides a non-volatile computer-readable storage medium:

[0035] The non-volatile computer-readable storage medium stores instructions, which, when executed on a computer, implement the method described in the first aspect.

[0036] In a fifth aspect, an embodiment of the present application provides a vehicle, which may include a multi-screen display device as described in the second aspect or the third aspect and a non-volatile computer-readable storage medium as described in the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0038] FIG1 is a schematic diagram of the architecture of a single-host display system provided in an embodiment of the present application;

[0039] FIG2 is a schematic flow chart of a multi-screen display method provided in an embodiment of the present application;

[0040] FIG3 is a flow chart of a method for displaying a synthesis process obtaining resource calling authority provided by an embodiment of the present application;

[0041] FIG4 is a schematic diagram of a permission verification process according to an embodiment of the present application;

[0042] FIG5 is a multi-screen display device provided in an embodiment of the present application;

[0043] FIG6 is another multi-screen display device provided in an embodiment of the present application.

[0044] Description of reference numerals:

[0045] 500 - multi-screen display device, 501 - application module, 502 - rendering module, 503 - verification module;

[0046] 600 - multi-screen display device, 610 - processor, 620 - memory, 630 - I / O interface, 621 - storage system, 622 - cache, 623 - RAM. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, rather than to describe a specific order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0049] References herein to "embodiments" mean that a particular feature, result, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0050] One purpose of the present application is to provide a multi-screen display method, device, related media and vehicle, which can improve the isolation of the display processes corresponding to each display screen, thereby ensuring the stability of the display data corresponding to each display screen.

[0051] It can be seen that the present application can allocate corresponding display screens to different application running environments, and each application running environment will use its own authorized display process to render the display image data, which helps to improve the isolation of the display processes corresponding to different display screens, thereby improving the stability of the display data corresponding to each display screen, and the embodiment method of the present application can be applied to multiple system platforms, can meet the user's various operation needs, and improve the user experience.

[0052] In order to better understand the technical solutions of the embodiments of the present application, the single-host display system that may be involved in the embodiments of the present application is first introduced. Please refer to Figure 1, which is a schematic diagram of the architecture of a single-host display system provided by the embodiments of the present application. The single-host display system may include multiple application runtime environments (such as the first application runtime environment 1, the second application runtime environment 2, and the third application runtime environment 3 in Figure 1), an application layer code library (libdrm library) 110, and a direct rendering manager (Direct Rendering Manager, DRM) 120 to drive multiple display screens (such as the first display screen 1, the second display screen 2, and the third display screen 3 in Figure 1) to display images.

[0053] Each application runtime environment includes an application set and an authorized display process. For example, the first application runtime environment 1 may include the first application set 1 and the first authorized display process 1, the second application runtime environment 2 may include the second application set 2 and the second authorized display process 2, and the third application runtime environment 3 may include the third application set 3 and the third authorized display process 3. Each application set may include multiple applications. It should be noted that the number of applications included in each application set is not limited. Furthermore, each authorized display process may be in a namespace (Namespace) and / or control group (C group) in its corresponding application runtime environment. Each authorized display process in FIG1 may send an authorization request (such as the first authorization request 1, the second authorization request 2, and the third authorization request 3 in FIG1 ) to the direct rendering manager 120 to obtain corresponding rendering resource call permissions (such as the first rendering resource call permission 1, the second rendering resource call permission 2, and the third rendering resource call permission 3 in FIG1 ), and use the rendering resource call permissions to indirectly obtain rendering resources of the direct rendering manager 120 through the libdrm library 110.

[0054] A display screen is a computer's I / O device, or output device. It is a display tool that can display certain electronic files on the screen through a specific transmission device. Depending on the display technology, display screens can be divided into cathode ray tube (CRT) display screens, plasma display panel (PDP) display screens, liquid crystal display (LCD) display screens, organic light emitting diode (OLED) display screens, and micro light emitting diode (Micro LED) display screens. Depending on the appearance, display screens can be divided into flat panel display screens, curved display screens, and 3D display screens. Depending on the application scenario, display screens can be divided into gaming display screens, office display screens, and art design display screens.

[0055] The direct rendering manager 120 is a display driver framework in the Linux operating system kernel layer. It can include multiple rendering resources such as the display output context (CRTC) (such as the first CRTC 1211, the second CRTC 1212, and the third CRTC 1213), layers (Planes) (such as the first Planes 1221, the second Planes 1222, and the third Planes 1223), encoders (such as the first Encoder 1231, the second Encoder 1232, and the third Encoder 1233), and connectors (such as the first connector 1241, the second connector 1242, and the third connector 1243) as shown in Figure 1. Among them, the CRTC can read the pixel data in the scan buffer and generate a video mode timing signal from it with the help of a phase locked loop (PLL) circuit. It is connected to the canvas (Framebuffer) address internally and to the encoder externally. The CRTC scans the content on the Framebuffer and transmits it to the encoder after superimposing the content of the Planes.

[0056] Planes, like Framebuffer, are memory addresses. There can be multiple Planes, and the picture (image) finally provided to CRTC for scanning is actually often a combination of Framebuffer and Planes.

[0057] The encoder encodes (or converts) the pixels in the memory into the signals required by the display screen. The interaction between it and the CRTC is called modesetting, which includes display information such as color mode, image size, and image display timing.

[0058] Connector connects to a physical display output device (such as VGA, DVI, FPD-Link, HDMI, DisplayPort or S-Video, etc., which corresponds to the display screen in the embodiment of this application) and stores information related to the currently physically connected display output device (such as connection status, EDID data, DPMS status or supported video formats).

[0059] Furthermore, a libdrm library 110 may be provided between the authorized display process and the direct rendering manager. The libdrm library 110 may be used to encapsulate the rendering resource interface provided by the direct rendering manager. The authorized display process may indirectly call the rendering resources of the direct rendering manager through the libdrm library.

[0060] Below, the method involved in the embodiment of the present application is described in detail with reference to Figures 2 and 3.

[0061] Please refer to FIG2 , which is a flowchart of a multi-screen display method provided in an embodiment of the present application. The method is executed by a single-host display system, and the single-host display system may include multiple application running environments.

[0062] The method may include the following steps:

[0063] S201 : For any of the application running environments, synthesize display data based on an authorized display process corresponding to the application running environment to obtain image data to be displayed.

[0064] The display data may be the display data corresponding to image information acquired by an application in the application runtime environment, or the display data corresponding to a page presented to a user by the application in the application runtime environment. For example, when the application in the application runtime environment is a camera, the display data may be the display data corresponding to an image captured by the user; when the application in the application runtime environment is chat software, the display data may be the display data corresponding to the chat software's login interface, the chat box page, the chat list page, or the contact page.

[0065] Optionally, the single-host display system may also include multiple display screens and a direct rendering manager, each of the multiple application running environments corresponds to one of the multiple display screens, each application running environment may include an application set and an authorized display process, and each application set may include one or more applications.

[0066] In one possible implementation, each application runtime environment is a container system to achieve isolation of the application runtime environment.

[0067] Exemplarily, each application runtime environment is a Linux container system (LXC), which implements multiple systems through Namespace and Cgroup.

[0068] In another possible implementation, any application runtime environment may include a display composition process (HWC, Hardware Composer);

[0069] Before synthesizing display data based on the authorized display process corresponding to the application running environment to obtain image data to be displayed, the method may further include:

[0070] The display composition process is authorized and checked, so that when the check passes, it is determined that the display composition process is the authorized display process corresponding to the application running environment, and the authorized display process has the rendering resource calling permission.

[0071] For the specific authorization verification process, please refer to Figure 3 and its related embodiment content.

[0072] S202 : Calling a direct rendering manager to render the image data to be displayed in the application running environment, so as to display the image data on a display screen corresponding to the application running environment.

[0073] Exemplarily, the authorized display process may send the rendered image data to the display screen corresponding to the application runtime environment through the connector interface in the direct rendering manager, such as the first display screen 1 corresponding to the first application runtime environment 1 in FIG1 .

[0074] In another possible implementation, any application execution environment may include multiple image data corresponding to multiple application programs;

[0075] Synthesizing display data based on the authorized display process corresponding to the application running environment to obtain image data to be displayed may include:

[0076] The plurality of image data are synthesized into image data to be displayed using an authorized display process corresponding to the application running environment.

[0077] Specifically, the display data may include multiple image layers (corresponding to the multiple image data mentioned above), and the above multiple image layers need to be integrated into a complete image data through the layer synthesis resources of the direct rendering manager, and the format of the above complete image data needs to be adjusted according to the display requirements of the display screen corresponding to the application running environment (such as color mode and / or resolution of the display screen, etc.), and finally the image data to be displayed can be obtained.

[0078] It can be seen that the embodiments of the present application can isolate the rendering operations of the display data corresponding to each application running environment and display screen, which helps to ensure that each display screen can normally present the target image data, and can also ensure that the rendering process of the display data of each application in each application running environment is carried out normally, thereby improving the data display stability of each display screen and further improving the user experience.

[0079] Further, referring to FIG3 , which is a flowchart of a method for a display composition process to obtain rendering resource calling permissions provided by an embodiment of the present application, wherein any application running environment may include a display composition process;

[0080] The method may include the following steps:

[0081] S301: The display composition process sends an authorization request to the direct rendering manager.

[0082] In another possible implementation, the authorization request may include identification information of the display synthesis process and identity information of the target display screen.

[0083] The target display screen is a display screen where the application set corresponding to the application running environment is expected to present image data. The identity information of the target display screen may include the sequence number information of the target display screen among the plurality of display screens.

[0084] For example, assuming that there are a first display synthesis process 1 (identification information is com.xxx.yyy::111), a second display synthesis process 2 (identification information is com.xxx.yyy::112), and a third display synthesis process 3 (identification information is com.xxx.yyy::113) in the first application running environment, and the sequence number information of the target display screen is 1, if the first application set corresponding to the first application running environment is expected to present image data on the target display screen, an authorization request similar to token = "displayId = 1 &&processName = com.xxx.yyy::111" can be generated, where displayId = 1 &&processName = com.xxx.yyy::111. layId is used to specify the serial number information of the target display screen, displayId=1 represents the display screen with serial number 1, processName is used to specify the identification information of the display synthesis process that is expected to apply for the rendering resource calling permission of the direct rendering manager, processName=com.xxx.yyy::111 represents the display synthesis process with identification information com.xxx.yyy::111. It can be understood that the meaning of this authorization application is "the display synthesis process with identification information com.xxx.yyy::111 is expected to transmit the image data corresponding to the first application set in the first application running environment to the display screen with serial number 1."

[0085] S302: The direct rendering manager determines the permission for displaying the composition process according to the authorization request.

[0086] It should be noted that before the direct rendering manager determines the permissions of the display synthesis process based on the authorization request, it is necessary to establish an association between the display synthesis process in the application runtime environment and the display screen corresponding to the application runtime environment. The display screen corresponding to the application runtime environment will recognize the display synthesis process with which it has an association as a preset display process, and save the identification information of the display synthesis process in the process whitelist. Furthermore, the step of establishing the association can be set by the staff before the single-host display system is put into use, and the user can also customize the preset display process of each display screen. For example, if the first display synthesis process (identification information is com.xxx.yyy::111) in the existing first application runtime environment has an association with the first display screen corresponding to the first application runtime environment, the first display screen will recognize the first display synthesis process as the preset display process and save the identification information com.xxx.yyy::111 in the process whitelist.

[0087] In another possible implementation, the direct rendering manager determines the permission for displaying the composition process according to the authorization request, which may include:

[0088] When the direct rendering manager determines that the display resource corresponding to the target display screen exists and the identification information of the display composition process matches the identification information of the preset display process of the target display screen, the display composition process is determined to be an authorized display process.

[0089] In another possible implementation, the method may further include:

[0090] When the direct rendering manager determines that the display resource corresponding to the target display screen does not exist, or the identification information of the display composition process does not match the identification information of the preset display process of the target display screen, the authorization check is terminated.

[0091] For example, assume that there are a first display screen 1 (the preset display process is the fourth display synthesis process 4, the identification information of the fourth display synthesis process 4 is com.xxx.yyy::114, and the fourth display synthesis process 4 belongs to the fourth application running environment 4), a second display screen 2 (the preset display process is the fifth display synthesis process 5, the identification information of the fifth display synthesis process 5 is com.xxx.yyy::115, and the fifth display synthesis process 5 belongs to the fifth application running environment 5), and a third display screen 3 (the preset display process is the sixth display synthesis process 6, the identification information of the sixth display synthesis process 6 is com.xxx.yyy::115). xxx.yyy::116, the sixth display synthesis process 6 belongs to the sixth application running environment 6), if the authorization request generated by the sixth display synthesis process 6 is token = "displayId = 3 &&processName = com.xxx.yyy::116", first confirm that there is a display screen with serial number 3, and the display synthesis process with identification information com.xxx.yyy::116 is the preset display process of the third display screen 3, so the authorization request of the sixth display synthesis process 6 is considered to be legal, and the rendering resources for the direct rendering manager are issued to the sixth display synthesis process 6. The rendering resource calling permission of the source can further confirm that the sixth display synthesis process 6 is the authorized display process of the sixth application runtime environment 6, and the third display screen 3 is the display screen for the sixth application runtime environment 6 to present the corresponding application set image data; if the authorization request generated by the fifth display synthesis process 5 is token = "displayId = 6 &&processName = com.xxx.yyy::115", first confirm that there is no display screen with serial number 6, then the authorization check for token = "displayId = 6 &&processName = com.xxx.yyy::115" can be terminated; if the authorization request generated by the fifth display synthesis process 5 is token = "displayId = 3 &&processName = com.xxx.yyy::115", first confirm that there is a display screen with serial number 3, but the display synthesis process with identification information com.xxx.yyy::115 is not the preset display process of the third display screen 3, then the authorization check for token = "displayId = 6 &&processName = com.xxx.yyy::115" can be terminated.

[0092] In another possible implementation, each application runtime environment may include an authorization display process. For all authorization display processes corresponding to all application runtime environments, all authorization display processes may include a first authorization display process and at least one second authorization display process, wherein the first authorization display process is the first process to pass the direct renderer authorization check.

[0093] In another possible implementation, the first authorized display process has more rendering resource calling permissions than the second authorized display process.

[0094] For example, multiple application runtime environments in a single-host display system all have their own authorized display processes, but there will be a sequence of time nodes when these authorized display processes obtain rendering resource call permissions from the direct rendering manager. Among them, the first process that passes the direct rendering manager authorization check can be regarded as the first authorized display process, and the other authorized display processes can be regarded as the second authorized display process. The first authorized display process can have DRM-Master permissions, and the second authorized display process can have DRM-Auth permissions. The rendering resource interface of the direct rendering manager corresponding to the DRM-Master permission is more than the rendering resource interface of the direct rendering manager corresponding to the DRM-Auth permission. The specific permission differences can refer to the permission differences between DRM-Master and DRM-Auth in the prior art, which will not be repeated here. Possibly, the direct rendering manager also needs to determine whether the instruction of the authorized display process to call the rendering resource is within the scope of the rendering resource call permission owned by the authorized display process. If it is within the scope of the rendering resource call permission, the authorized display process is allowed to call the corresponding rendering resource; if it is not within the scope of the rendering resource call permission, the resource call request of the authorized display process is rejected.

[0095] For example, assuming that the existing sixth display synthesis process 6, seventh display synthesis process 7, eighth display synthesis process 8 and ninth display synthesis process 9 have the rendering resource calling permission of the direct rendering manager, the order in which the direct rendering manager issues the rendering resource calling permission is: eighth display synthesis process 8 → ninth display synthesis process 9 → sixth display synthesis process 6 → seventh display synthesis process 7, then the eighth display synthesis process 8 has DRM-Master permission, and the sixth display synthesis process 6, seventh display synthesis process 7 and ninth display synthesis process 9 all have DRM-Auth permission.

[0096] It should be noted that in the embodiment of the present application, each application runtime environment will include an authorized display process, and the display data of all applications in each application runtime environment will be rendered by the authorized display process in the application runtime environment. Possibly, each time the vehicle (or application runtime environment) is started, the authorized display process will send authorization permissions to the direct rendering manager, and the direct rendering manager will also verify the authorization request of the authorized display process; or after the direct rendering manager verifies the permissions of each display process, each display process (or authorized display process) will always maintain the same authorization status. For the specific permission verification process, please refer to the corresponding content of Figure 3.

[0097] As can be seen, the method of the embodiment of the present application can determine whether the authorization request of each display process is legitimate based on the existing identity information of the display screen and the preset display process of the display screen, which helps to reduce the communication interaction between the processes, thereby improving the efficiency of the direct rendering manager in verifying the authorization request of the display process. The method of the embodiment of the present application can also issue different rendering resource call permissions to the display processes according to the order in which the display processes obtain rendering resource call permissions, which helps to improve the security of the display data corresponding to each application running environment.

[0098] For example, please refer to Figure 4, which is a flowchart of a permission verification process provided in an embodiment of the present application. The permission verification process can be as follows:

[0099] For display compositing progress, you can have:

[0100] S401: Generate an authorization request based on the identity information of the target display screen and its own identification information.

[0101] S402: Send an authorization request to the direct rendering manager.

[0102] For direct rendering managers, you can have:

[0103] S403: Determine whether the identity information of the target display screen in the authorization request matches the identity information of the existing display screen.

[0104] If the identity information of the target display screen does not match the identity information of the existing display screen, then S404 deems the authorization request for the display synthesis process illegal and rejects the authorization request for the display synthesis process.

[0105] If the identity information of the target display screen matches the identity information of the existing display screen, then S405 determines whether the display synthesis process is a preset display process for the target display screen.

[0106] If the display synthesis process is not the preset display process of the target display screen, then S406 deems the authorization request of the display synthesis process illegal, and rejects the authorization request of the display synthesis process.

[0107] If the display composition process is the preset display process of the target display screen, then S407 determines whether the display composition process is the first display process to which the direct rendering manager issues rendering resource calling authority.

[0108] If the display composition process is the first display process to which the direct rendering manager issues rendering resource calling permissions, then S408 issues DRM-Master permissions to the display composition process.

[0109] If the display composition process is not the first display process to which the direct rendering manager issues the rendering resource calling permission, then S409 issues the DRM-Auth permission to the display composition process.

[0110] The following describes the device involved in the embodiments of the present application with reference to the accompanying drawings.

[0111] 5 , which shows a multi-screen display device 500 provided in an embodiment of the present application. The device 500 may be included in a single-host display system, which may include multiple application runtime environments. The device 500 may include: an application module 501 and a rendering module 502.

[0112] Optionally, the single-host display system may also include multiple display screens and a direct rendering manager, each of the multiple application running environments corresponds to one of the multiple display screens, each application running environment may include an application set and an authorized display process, and each application set may include one or more application programs.

[0113] The application module 501 can be used to synthesize display data for any application running environment based on the authorized display process corresponding to the application running environment to obtain image data to be displayed;

[0114] The rendering module 502 may be configured to call a direct rendering manager to render image data to be displayed in the application runtime environment, so as to display the image data on a display screen corresponding to the application runtime environment.

[0115] In one possible implementation, each application runtime environment is a container system to achieve isolation of the application runtime environment.

[0116] In another possible implementation, any application execution environment may include a display composition process;

[0117] The apparatus may further include: a verification module 503;

[0118] The verification module 503 can be used to perform authorization verification on the display synthesis process, so as to determine that the display synthesis process is an authorized display process corresponding to the application running environment when the verification passes, and the authorized display process has the rendering resource calling permission.

[0119] In another possible implementation, the device may further include:

[0120] The rendering module 502 may also be used to send an authorization request to the direct rendering manager;

[0121] The verification module 503 may also be used to determine the authority to display the synthesis process according to the authorization request.

[0122] In another possible implementation, the authorization request may include identification information of the display synthesis process and identity information of the target display screen.

[0123] In another possible implementation, the device may further include:

[0124] The verification module 503 can also be used to determine that the display synthesis process is an authorized display process when the direct rendering manager determines that the display resources corresponding to the target display screen exist and the identification information of the display synthesis process matches the identification information of the preset display process of the target display screen.

[0125] In another possible implementation, the method may further include:

[0126] The verification module 503 may also be used to terminate the authorization verification when the direct rendering manager determines that the display resource corresponding to the target display screen does not exist, or the identification information of the display synthesis process does not match the identification information of the preset display process of the target display screen.

[0127] In another possible implementation, each application runtime environment may include an authorization display process. For all authorization display processes corresponding to all application runtime environments, all authorization display processes may include a first authorization display process and at least one second authorization display process, wherein the first authorization display process is the first process to pass the direct renderer authorization check.

[0128] In another possible implementation, the first authorized display process has more rendering resource calling permissions than the second authorized display process.

[0129] In another possible implementation, any application execution environment may include multiple image data corresponding to multiple application programs;

[0130] In another possible implementation, the application module 501 may also be configured to synthesize multiple image data into image data to be displayed using an authorized display process corresponding to the application runtime environment.

[0131] Please refer to FIG6 , which is a schematic diagram of another multi-screen display device 600 provided in an embodiment of the present application. The device 600 may include:

[0132] Processor 610, memory 620, and I / O interface 630. The processor 610, memory 620, and I / O interface 630 are communicatively connected, the memory 620 is used to store instructions, and the processor 610 is used to execute the instructions stored in the memory 620 to implement the method steps corresponding to Figures 2 and 3 above.

[0133] The processor 610 is used to execute the instructions stored in the memory 620 to control the I / O interface 630 to receive and send signals and complete the steps in the above method. The memory 620 can be integrated into the processor 610 or set separately from the processor 610.

[0134] Memory 620 may also include a storage system 621, cache 622, and RAM 623. Cache 622 is a primary memory located between RAM 623 and the CPU. It consists of static random access memory (SRAM) chips and has a relatively small capacity but a much higher speed than main memory, approaching the speed of the CPU. RAM 623 is internal memory that directly exchanges data with the CPU. It can be read and written at any time (except when refreshing) and is very fast. It is typically used as a temporary data storage medium for the operating system or other running programs. These three components combine to realize the functions of memory 620.

[0135] As an implementation method, the function of the I / O interface 630 may be implemented by a transceiver circuit or a dedicated transceiver chip. The processor 610 may be implemented by a dedicated processing chip, a processing circuit, a processor, or a general-purpose chip.

[0136] As another implementation, it is possible to implement the apparatus provided in the embodiments of the present application using a general-purpose computer. Specifically, the program code that implements the functions of the processor 610 and the I / O interface 630 is stored in the memory 620, and the general-purpose processor executes the code in the memory 620 to implement the functions of the processor 610 and the I / O interface 630.

[0137] For the concepts, explanations, detailed descriptions and other steps involved in the device and related to the technical solutions provided in the embodiments of the present application, please refer to the description of the method steps performed by the device in the aforementioned method or other embodiments, which will not be repeated here.

[0138] As another implementation of this embodiment, a non-volatile computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed, the method in the above method embodiment is executed.

[0139] As another implementation of this embodiment, a computer program product including instructions is provided. When the instructions are executed, the method in the above method embodiment is performed.

[0140] Those skilled in the art will appreciate that, for ease of explanation, FIG6 shows only one memory and processor. In an actual terminal or server, multiple processors and memories may exist. Memory may also be referred to as a storage medium or storage device, etc., which is not limited in the present embodiment.

[0141] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0142] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAMbus RAM (DR RAM).

[0143] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) is integrated into the processor.

[0144] It should be noted that the memory described herein is intended to include, but not be limited to, these and any other suitable types of memory.

[0145] In addition to the data bus, the bus may also include a power bus, a control bus, a status signal bus, etc. However, for the sake of clarity, various buses are labeled as buses in the figure.

[0146] It should also be understood that the first, second, third, fourth and various numerical numbers involved in this document are only distinctions made for the convenience of description and are not intended to limit the scope of this application.

[0147] It should be understood that the term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0148] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it will not be described in detail here.

[0149] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0150] Those skilled in the art will appreciate that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0152] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0154] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk).

[0155] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement part or all of the steps of any multi-screen display method described in the above method embodiments.

[0156] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any multi-screen display method described in the above method embodiments.

[0157] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A multi-screen display method, characterized in that: The method is performed by a single host display system, the single host display system includes multiple application running environments, and the method includes: For any of the application running environments, display data synthesis is performed based on the authorized display process corresponding to the application running environment (S201) to obtain image data to be displayed; A direct rendering manager is called (S202) to render the image data to be displayed in the application running environment, so as to display the image data in the display screen corresponding to the application running environment.

2. The method according to claim 1, characterized in that Each of the application operating environments is a container system to achieve isolation of the application operating environments.

3. The method according to claim 1 or 2, characterized in that: Any of the application operating environments includes display synthesis process; Before synthesizing display data based on the authorized display process corresponding to the application running environment to obtain image data to be displayed, the method further includes: An authorization check is performed on the display synthesis process to determine that the display synthesis process is an authorized display process corresponding to the application running environment when the check passes, and the authorized display process has rendering resource calling authority.

4. The method according to claim 3, characterized in that The performing authorization verification on the display synthesis process includes: The display composition process sends (S301) an authorization request to the direct rendering manager; The direct rendering manager determines ( S302 ) permissions for the display composition process according to the authorization request.

5. The method according to claim 4, characterized in that The authorization request includes identification information of the display synthesis process and identity information of the target display screen.

6. The method according to claim 5, characterized in that The direct rendering manager determines the permission of the display synthesis process according to the authorization request, including: When the direct rendering manager determines that there are display resources corresponding to the target display screen, and the identification information of the display synthesis process matches the identification information of the preset display process of the target display screen, the display synthesis process is determined to be an authorized display process.

7. The method according to claim 6, characterized in that The method further comprises: When the direct rendering manager determines that there is no display resource corresponding to the target display screen, or the identification information of the display synthesis process does not match the identification information of the preset display process of the target display screen, the authorization check is terminated.

8. The method according to any one of claims 1 to 7, characterized in that Each of the application running environments includes an authorization display process. For all the authorization display processes corresponding to all the application running environments, all the authorization display processes include a first authorization display process and at least one second authorization display process, wherein the first authorization display process is the first process to pass the direct renderer authorization check.

9. The method according to claim 8, characterized in that The first authorized display process has more rendering resource calling permissions than the second authorized display process.

10. The method according to any one of claims 1 to 9, characterized in that Any of the application running environments includes a plurality of image data corresponding to a plurality of application programs; The step of synthesizing display data based on the authorized display process corresponding to the application running environment to obtain image data to be displayed includes: The plurality of image data are synthesized into image data to be displayed using an authorized display process corresponding to the application running environment.

11. A multi-screen display device (500), characterized in that: The device (500) is executed by a single host display system, the single host display system includes multiple application running environments, and the device (500) includes: An application module (501) is used for synthesizing display data for any of the application running environments based on an authorized display process corresponding to the application running environment to obtain image data to be displayed; The rendering module (502) is used to call the direct rendering manager to render the image data to be displayed in the application running environment so as to display it on the display screen corresponding to the application running environment.

12. The multi-screen display device (500) according to claim 11, characterized in that: Also includes: The verification module (503) is used to perform authorization verification on the display synthesis process, so as to determine that the display synthesis process is an authorized display process corresponding to the application running environment when the verification passes, and the authorized display process has the rendering resource calling permission.

13. The multi-screen display device (500) according to claim 11 or 12, characterized in that: Each of the multiple application execution environments corresponds to one display screen of the multiple display screens included in the single-host display system.

14. The multi-screen display device (500) according to any one of claims 11 to 13, characterized in that: Each of the application running environments includes an application set and an authorization display process, and each application set includes one or more application programs.

15. A multi-screen display device (600), characterized in that: include: A processor (610), a memory (620) and an I / O interface (630), wherein the processor (610), the memory (620) and the I / O interface (630) are communicatively connected, wherein the memory (620) is used to store a set of program codes, and the processor (610) is used to call the program codes stored in the memory (620) to execute the method according to any one of claims 1 to 10.

16. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores instructions which, when executed on a computer, implement the following The method according to any one of claims 1 to 10.

17. A vehicle, characterized in that: The vehicle includes the multi-screen display device according to any one of claims 11 to 15 or the non-volatile computer-readable storage medium according to claim 16.

Images