A display interface sharing and management method and system between isolated systems

Abstract

The present application relates to a method and system for sharing and managing display interfaces between isolated systems, and relates to the field of computer technology, including: a display server system and a display client system respectively running in two system domains, the display server system creating a main virtual display interface corresponding to a slave virtual display interface of the display client system, mapping the framebuffer, display configuration information and related display events of the main virtual display interface to the corresponding shared mapping memory, the display client system creating a slave virtual display interface, mapping the slave virtual display interface to the corresponding shared mapping memory, and writing the slave virtual display interface data to the corresponding shared mapping memory, the display server system domain obtaining the slave virtual display interface data from the shared mapping memory, and updating, managing and processing the slave virtual display interface data of the corresponding shared mapping memory when necessary.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and system for sharing and managing display interfaces between isolated systems. Background Art

[0002] Currently, in hardware-isolated multi-systems, display interface sharing can only be achieved through display device hardware sharing. A display device can only be used by one isolated system domain at the same time, and virtual sharing of display devices requires a large amount of data interaction. Sharing through remote desktop also requires the involvement of the network, which increases the difficulty of system development; at the same time, the RTOS system does not fully support remote desktop, multi-screen display, display management, display rendering, etc. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a method and system for sharing and managing display interfaces between isolated systems.

[0004] In a first aspect, the present application provides a display interface sharing and management system between isolated systems, using the following technical solutions:

[0005] A display interface sharing and management system between isolated systems, comprising: a hardware layer, a display server system domain, a display client system domain and a shared mapped memory, wherein the display server system domain and the display client system domain are different system domains that isolate hardware resources by hardware isolation, wherein a display server system runs on the display server system domain, and a display client system runs on the display client system domain, the display server system controls and manages the display devices corresponding thereto in the hardware layer, the display server system and the display client system are connected via a shared mapped memory, wherein

[0006] The display server system running on the display server system domain is used to create a main virtual display interface corresponding to the slave virtual display interface of the display client system, map the framebuffer of the main virtual display interface to the corresponding shared mapping memory, write the display configuration information to the corresponding position in the shared mapping memory, obtain the slave virtual display interface data written by the display client system from the shared mapping memory, display the slave virtual display interface data in the main virtual display interface, and if the user triggers a related display event, parse the related display event and write it to the corresponding shared mapping memory position, and trigger the display event interrupt to notify the display client system to obtain it. The framebuffer is a memory space that caches screen display data.

[0007] The display client system running in the display client system domain is used to create a slave virtual display interface and read the display configuration information written by the display server system in the corresponding shared mapping memory. After the display configuration information matches, the slave virtual display interface is mapped to the corresponding shared mapping memory, and the slave virtual display interface data is written to the corresponding shared mapping memory. After receiving the display event interrupt sent by the display server system domain, the display event information is read and the relevant display event is responded to.

[0008] Shared mapped memory: a memory area shared by the display server system and the display client system. The main virtual display interface, display events, display configuration information of the display server system and the secondary virtual display interface, display events, display configuration information of the display client system are respectively mapped and bound to the same corresponding shared mapped memory.

[0009] In a possible implementation, the display server system includes a display management module, a display device, a system display service, a main virtual display configuration file, and a physical display interface, wherein:

[0010] The display server system loads and runs the system display service according to the display device; the display management module is created by the system display service after the interface environment is loaded. After the display management module is created, a main virtual display interface corresponding to the slave virtual display interface of the corresponding display client system is created according to the main virtual display configuration file;

[0011] The display management module of the display server system maps the framebuffer of the main virtual display interface to the corresponding shared mapping memory according to the main virtual display configuration file, and writes the display configuration information to the corresponding position of the shared mapping memory;

[0012] The display management module of the display server system receives the slave virtual display interface data interrupt and slave virtual display interface data written into the corresponding shared mapped memory by the display client system, updates the slave virtual display interface data of the corresponding shared mapped memory to the main virtual display interface when the corresponding main virtual display interface is used, and manages and processes the main virtual display interface data at the same time, and does not update the slave virtual display interface data of the corresponding shared mapped memory to the main virtual display interface when the corresponding main virtual display interface is not used;

[0013] The display management module of the display server system obtains the relevant display events of the corresponding main virtual display interface through the system display service, parses the relevant display events and writes them to the corresponding shared mapped memory location, and triggers the display event interrupt notification to display the client system domain to read. The relevant display events include mouse clicks, cursor positions, keyboard inputs and touch actions.

[0014] In a possible implementation, the display client system includes a virtual display environment, a display virtualization module, and a virtual display configuration file, wherein:

[0015] The display client system loads and runs the virtual display environment and the display virtual module according to the virtual display configuration file, and creates a virtual display interface;

[0016] After the display virtual module of the display client system successfully creates the slave virtual display interface, it reads the display configuration information written by the corresponding shared mapped memory display server system and matches it with the configuration information in the slave virtual display configuration file. After the display configuration information matches, the display virtual module maps the slave virtual display interface of the display client system to the corresponding shared mapped memory;

[0017] The display virtual module of the display client system writes the corresponding data from the virtual display interface into the corresponding shared mapping memory that has been successfully matched;

[0018] After the display client system receives the display event interrupt sent by the display server system, the display virtual module reads the display event information corresponding to the shared mapped memory and responds to the relevant display event.

[0019] In a second aspect, the present application provides a method for sharing and managing display interfaces between isolated systems, using the following technical solutions:

[0020] A method for sharing and managing display interfaces between isolated systems, comprising:

[0021] Isolating hardware resources by hardware isolation to isolate a display server system domain and a display client system domain, wherein a display server system runs on the display server system domain, and a display client system runs on the display client system domain;

[0022] The display server system reads the display configuration information in the main virtual display configuration file, and initializes the shared mapping memory according to the display configuration information;

[0023] The display server system creates a main virtual display interface corresponding to the display client system, maps the framebuffer of the main virtual display interface to the corresponding shared mapping memory, and writes the display configuration information to the corresponding position in the shared mapping memory, and the display client system creates a slave virtual display interface;

[0024] After the slave virtual display interface is created, the display client system reads the display configuration information written by the display server system in the corresponding shared mapping memory, and after the display configuration information matches, maps the slave virtual display interface to the corresponding shared mapping memory, and writes the slave virtual display interface data into the corresponding shared mapping memory;

[0025] The display server system obtains the slave virtual display interface data written by the display client system from the shared mapped memory, and updates the slave virtual display interface data corresponding to the shared mapped memory to the master virtual display interface when necessary, and manages and processes the master virtual display interface data;

[0026] When the user triggers a related display event, the display server system parses the related display event and writes it into the corresponding shared mapping memory location, and triggers a display event interrupt to notify the display client system to obtain it;

[0027] After receiving the display event interrupt sent by the display server system, the display client system reads the display event information and responds to the relevant display event.

[0028] In a possible implementation, the display server system includes a display management module, a display device, a system display service, a primary virtual display configuration file, and a physical display interface. The display server system creates a primary virtual display interface corresponding to the display client system, maps a framebuffer of the primary virtual display interface to a corresponding shared mapping memory, and writes display configuration information to a corresponding position in the shared mapping memory, including:

[0029] The display server system domain loads and runs the system display service according to the display device;

[0030] The system display service controls the display of the physical display interface. After the physical display interface environment is loaded, the display management module creates a main virtual display interface for use by the corresponding display client system according to the main virtual display configuration file.

[0031] The display management module of the display server system maps the framebuffer of the main virtual display interface to the corresponding shared mapping memory according to the main virtual display configuration file, and writes the display configuration information to the corresponding position of the shared mapping memory.

[0032] In a possible implementation, the display client system includes a virtual display environment, a display virtual module, and a slave virtual display configuration file. After the slave virtual display interface is created, the display client system reads the display configuration information written by the display server system in the corresponding shared mapping memory, and after the display configuration information matches, maps the slave virtual display interface to the corresponding shared mapping memory, and writes the slave virtual display interface data into the corresponding shared mapping memory, including:

[0033] After the display virtual module of the display client system is successfully created from the virtual display interface, the display configuration information written by the corresponding shared mapping memory display server system is read from the virtual display configuration file. After the display configuration information matches, the display virtual module maps the display client system from the virtual display interface to the corresponding shared mapping memory;

[0034] The display client system writes the corresponding data from the virtual display interface into the corresponding successfully matched shared mapping memory.

[0035] In a possible implementation, the display server system includes a display management module, a display device, a system display service, a master virtual display configuration file, and a physical display interface. The display server system obtains slave virtual display interface data written by a display client system from a shared mapped memory, and updates the slave virtual display interface data of the corresponding shared mapped memory when necessary, including:

[0036] The display management module of the display server system receives the slave virtual display interface data written into the corresponding shared mapped memory by the display client system;

[0037] When the corresponding main virtual display interface is used, the slave virtual display interface data of the corresponding shared mapped memory is updated to the main virtual display interface;

[0038] When the corresponding main virtual display interface is not used, the slave virtual display interface data corresponding to the shared mapped memory is not updated to the main virtual display interface, and the main virtual display interface data is managed and processed at the same time.

[0039] In a possible implementation, the display server system includes a display management module, a display device, a system display service, a main virtual display configuration file, and a physical display interface. When a user triggers a related display event, the display server system parses the related display event and writes it into a corresponding shared mapping memory location, and triggers a display event interrupt to notify the display client system to obtain it, including:

[0040] When the user triggers a related display event, the display management module of the display server system obtains the related display event of the corresponding main virtual display interface through the system display service, parses the related display event and writes it to the corresponding shared mapped memory location, and triggers the display event interrupt notification display client system to read. The related display events include mouse clicks, cursor positions, keyboard inputs and touch actions.

[0041] In a possible implementation, the display client system includes a display virtual module. When the display client system receives a display event interrupt sent by the display server system, it reads the display event information and responds to the relevant display event, including:

[0042] After the display client system receives the display event interrupt sent by the display server system, the display virtual module reads the display event information corresponding to the shared mapped memory and responds to the relevant display event.

[0043] In summary, this application includes the following beneficial technical effects:

[0044] The present application improves the utilization rate of display devices, reduces the dependence of the display client system domain on the display environment, saves display hardware resources, and reduces display interface data transmission; at the same time, it provides a method for the RTOS system to use the system display service, so that the Linux system organizes and manages the RTOS system display interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a structural diagram of a display interface sharing and management system between isolation systems in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The present application is further described in detail below in conjunction with the accompanying drawings.

[0047] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but such modifications are protected by patent law as long as they are within the scope of the claims of this application.

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0050] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0051] For ease of understanding, the abbreviations in this application are first explained as follows:

[0052] Xorg supports multi-screen display. It configures display devices through system display services, enables the Linux system to support multi-screen display, and controls multiple display interfaces to flexibly switch and share one display.

[0053] In Linux systems, X Server and Client programs are often installed on the same machine, and in daily use, you can hardly feel the layering. However, X Server and Client can also run on different machines, running programs on one machine and displaying the graphical interface on another machine. At the same time, Linux and other systems support multiple display layers, and render and fuse multiple display layers for display.

[0054] Xvfb can run X Window related applications without a physical graphics card graphical interface and human-computer interaction interface. It simulates a complete X Window system, including display and input devices, but everything is processed in memory, without a display and keyboard.

[0055] Hardware isolation software such as Jailhouse is sent through hardware isolation to isolate different SOC cores, memories, and peripherals into different system domains. Each system domain can run a different operating system independently.

[0056] Secondly, if Figure 1 As shown, an embodiment of the present application provides a display interface sharing and management system between isolated systems, including: a hardware layer, a display server system domain, a display client system domain and a shared mapped memory. The display server system domain and the display client system domain are different system domains that isolate hardware resources by hardware isolation. A display server system runs on the display server system domain, and a display client system runs on the display client system domain. The display server system controls and manages the corresponding display devices in the hardware layer. The display server system runs in the operating system of the display server system domain, and the display client system runs in the operating system of the display client system domain. The two systems are connected to each other through shared mapped memory, so that the display client system domain can run programs that depend on the display environment without relying on hardware, thereby saving hardware resources.

[0057] The specific functions of each part are as follows:

[0058] 1. Display server system

[0059] The display server system runs on the main system domain or the slave system domain isolated by the jailhouse. The display server system runs the Linux and other operating systems. The display server system includes a display management module, a system display service, a main virtual display configuration file, a physical display interface, and a main virtual display interface. The system display service is loaded by the display server system according to the display device of the hardware layer and is used to control the interface display of the main virtual display interface. The display management module is created by the system display service after the interface environment is loaded, and a main virtual display interface is created for the corresponding display client system according to the main virtual display configuration file.

[0060] 1. Display management module

[0061] The functions of the display management module are as follows:

[0062] 1.1 Map the mapped memory of the main virtual display interface to the corresponding shared mapped memory according to the main virtual display configuration file, and write the display configuration information to the corresponding position of the shared mapped memory;

[0063] 1.2 The display management module receives the slave virtual display interface data written by the display client system to the corresponding shared mapped memory, updates the slave virtual display interface data of the corresponding shared mapped memory to the main virtual display interface when the corresponding main virtual display interface is used, and can call the system display service management and process the main virtual display interface data, and display the main virtual display interface content on the display screen (an external display screen such as an HDMI screen, a VGA screen, a DP screen, etc.); when the corresponding main virtual display interface is not used, the slave virtual display interface data of the corresponding shared mapped memory is not updated to the main virtual display interface.

[0064] 1.3 The display management module can call the system display service to set the resolution, refresh rate, interface layout, rotation and other operations of the main virtual display interface, and display the settings of the main virtual display interface on the display screen.

[0065] 1.4. If the display server system has a hardware decoder, and the slave virtual display interface data written by the display client system is encoded, the display management module of the display server system calls the hardware decoder to process the slave virtual display interface data read from the corresponding shared mapped memory.

[0066] 1.5 The display management module captures the relevant display events of the main virtual display interface, parses the relevant display events (display events are mouse clicks, cursor positions, keyboard inputs, touch actions, etc.) and writes them to the corresponding shared mapped memory location, and triggers the display event interrupt to notify the display client system to read.

[0067] 2. System display service

[0068] The system display service is the display server-related components and codes, such as Xorg, DRM, opengl, mesa, and wayland. The system display service can create multiple master virtual display interfaces corresponding to multiple display client systems, and manage the master virtual display interfaces of each display client system through xrandr, etc. Specifically, the system display service can control the master virtual display interface to interact with other display interfaces, that is, the application of the physical display interface of the display server system domain can be dragged to the slave virtual display interface of the display client system domain for display. Furthermore, a display screen can display one to multiple physical display interfaces and one to multiple slave virtual display interfaces, that is, a display device can be used by multiple system domains at the same time.

[0069] 3. Main virtual display configuration file

[0070] The main virtual display configuration file is used to configure the display server system to provide several main virtual display interfaces, which display device supports them, the shared mapping memory information corresponding to the main virtual display interface mapping, the display event mapping information, and the display client system domain to which the main virtual display interface is mapped. At the same time, you can also configure display settings such as resolution, layout, rotation, etc.

[0071] 4. The physical display interface is a display interface that depends on the display device. The display server system can flexibly switch the physical display interface and the main virtual display interface content on the display screen.

[0072] 5. Main virtual display interface

[0073] The display server system may create multiple master virtual display interfaces to map one or more slave virtual display interfaces bound to one or more display client systems.

[0074] 2. Display client system

[0075] The display client system runs in the main system domain or the slave system domain isolated in the jailhouse, and the display client system runs operating systems such as Linux and RTOS. The display client system includes a virtual display environment, a slave virtual display configuration file, a display virtual module, and a slave virtual display interface, wherein the display client system loads and runs the virtual display environment and the display virtual module according to the slave virtual display configuration file, and creates a slave virtual display interface.

[0076] 1. Virtual display environment

[0077] In Linux, it can be provided by Xvfb and does not depend on display hardware devices. In RTOS systems, graphics display libraries, such as lvgl, can be run without relying on hardware and hardware drivers to provide a virtual display environment.

[0078] 2. From the virtual display configuration file

[0079] The slave virtual display configuration file is used to configure the virtual display environment loading, display virtual module parameters and slave virtual display interface parameters. The display client system needs to provide several slave virtual display interfaces, corresponding shared mapping memory information mapped from the virtual display interface, display interface parameter mapping information, display event mapping information, and mapping the corresponding slave virtual display interface to which display server system domain.

[0080] 3. Display virtual modules

[0081] The specific functions of the display virtual module are as follows:

[0082] 3.1 After the virtual display interface is successfully created, the display virtual module reads the display configuration information written by the corresponding shared mapped memory display server system, and matches it with the display configuration information in the virtual display configuration file. After matching, the display virtual module maps the display client system's virtual display interface to the corresponding shared mapped memory;

[0083] 3.2 The display client system writes the corresponding virtual display interface data into the corresponding matched shared mapping memory; if there is a hardware encoder, the display virtual module calls the hardware encoder to encode the virtual display interface data and then write it into the corresponding matched shared mapping memory;

[0084] 3.3 After the display client system receives the display event interrupt sent by the display server system, the display virtual module reads the display event information corresponding to the shared mapped memory and responds to the relevant display event.

[0085] 4. From the virtual display interface

[0086] The display client system can create a mapping from a virtual display interface to bind to a display server system domain, and can also create multiple mappings from virtual display interfaces to bind to multiple display server systems.

[0087] 3. Shared mapped memory

[0088] The shared mapped memory is initialized by the display server system according to the main virtual display configuration file. It is a shared memory area shared by the display server system and the display client system. Each pair of the display client system's slave virtual display interface, display event, display configuration information and the display server system's master virtual display interface, display event, display configuration information are mapped and bound to the same corresponding shared mapped memory. That is, one shared mapped memory is used to store a pair of master virtual display interface and slave virtual display interface information. Multiple pairs of master virtual display interface and slave virtual display interface information require multiple blocks of shared mapped memory.

[0089] The shared mapping memory list structure example is as follows:

[0090] typedef struct share_display_buffer_g

[0091] {

[0092] uint8 status; / / connection status

[0093] uint8 service_num; / / Corresponding to the server system domain number

[0094] uint8 client_num; / / Corresponding to the client system domain number

[0095] struct display_set set; / / Corresponding display configuration information

[0096] struct display_event event; / / Corresponding to related display events

[0097] struct mapped memory fbuffer; / / Corresponding to the master and slave virtual display interface data

[0098] struct share_display_buffer_g next; / / Next pair of main virtual display interface and slave virtual display interface information node

[0099] } share_display_buffer_t;

[0100] The embodiment of the present application also provides a method for sharing and managing display interfaces between isolated systems, which is run on a display interface sharing and management system between isolated systems, as follows:

[0101] 1. The display server system has display devices and running system display services, and provides a main virtual display interface for the display client system to use. At the same time, the display management module maps the main virtual display interface to the shared mapping memory and writes the display configuration information;

[0102] 2. The display client system runs a virtual display interface environment. After the display virtual module of the display client system reads the display configuration information and completes the matching with the corresponding display server system, it maps the slave virtual display interface to the corresponding shared mapping memory, writes the corresponding slave virtual display interface data to the corresponding shared mapping memory, and sends it to the corresponding display server system through an interrupt notification;

[0103] 3. The display management module of the display server system receives the slave virtual display interface data and updates the slave virtual display interface data of the corresponding shared mapped memory when necessary;

[0104] 4. The display server system updates the main virtual display interface according to the updated slave virtual display interface data. The system display service can set the resolution, refresh rate, interface layout, rotation and other operations of the main virtual display interface, and display the settings of the main virtual display interface on the display screen;

[0105] 5. When the user triggers a related display event, the display management module captures the display event of the main virtual display interface, parses it, writes the related display event to the shared mapped memory, and triggers a display event interrupt;

[0106] 6. The display virtual module of the display client system responds to the received related display events, that is, the display events are used in conjunction with the secondary virtual display interface, such as the operating system of the display client system domain responds to the mouse, keyboard, etc. of the secondary virtual display interface.

[0107] Further, to facilitate understanding of the method, a detailed description will be given in conjunction with a specific operating system as follows:

[0108] When there is only one server system running the Kylin desktop operating system, and there are multiple client systems running the Centos operating system (server version, no display environment), the method is as follows:

[0109] 1. The Kylin desktop operating system running on the display server system loads the operating system display service, loads the physical display interface environment through the display device and uses the physical display interface normally;

[0110] 2. The display server system establishes information such as the display client system with which the shared mapped memory is to be shared as needed, and reads the write display configuration information in the main virtual display configuration file;

[0111] 3. The Kylin desktop operating system running on the display server system starts the display management module and initializes the shared mapping memory after the physical display interface environment is loaded;

[0112] 4. The display management module creates a main virtual display interface for binding and use by multiple corresponding display client systems according to the main virtual display configuration file. At the same time, the display management module maps the mapped memory of the main virtual display interface to the corresponding shared mapped memory according to the main virtual display configuration file, and writes the display configuration information to the corresponding position of the shared mapped memory;

[0113] 5. The display client system domain runs the Centos operating system, and the Centos operating system establishes information such as a display server system with a shared mapped memory as needed, and reads and writes display configuration information from the virtual display configuration file;

[0114] 6. The Centos operating system running in the display client system domain does not have a physical display device and a physical display interface environment. Therefore, the virtual display environment and display virtual module are loaded and run through Xvfb and other methods according to the virtual display configuration file, and the corresponding virtual display interface is created (it can be multiple virtual display interfaces);

[0115] 7. After the virtual display interface is successfully created, the display virtual module of the display client system of each Centos operating system running in the display client system domain reads the display configuration information written by the corresponding shared mapped memory display server system, and matches it with the configuration information in the virtual display configuration file. After matching, the display virtual module maps the display client system's virtual display interface to the corresponding shared mapped memory, and modifies the connection status;

[0116] 8. Each Centos operating system running on the display client system writes the corresponding data from the virtual display interface into the corresponding successfully matched shared mapping memory: If both the display server system and the display client system have software or hardware encoders, the display virtual module calls the software or hardware encoder to encode the virtual display interface data and then write it into the corresponding successfully matched shared mapping memory;

[0117] 9. The display management module of the Kylin desktop operating system running on the display server system receives the slave virtual display interface data written into the corresponding shared mapped memory by the display client system, and updates the slave virtual display interface data of the corresponding shared mapped memory to the main virtual display interface when the corresponding main virtual display interface is used; and does not update the slave virtual display interface data of the corresponding shared mapped memory to the main virtual display interface when the corresponding main virtual display interface is not used;

[0118] 10. The Kylin desktop operating system running in the system domain of the display server reads the updated slave virtual display interface data in the shared mapped memory corresponding to the currently used master virtual display interface, and based on the updated slave virtual display interface data, uses the system display service to control the master virtual display interface to be displayed on the display screen through the display device, and uses kwin and other window management, uses opengl and other libraries to call gpu, vpu and other hardware for acceleration and rendering, uses xrandr and other libraries to set display settings such as resolution, refresh rate, brightness, etc., and can flexibly switch the physical display interface and the master virtual display interface content on the display screen;

[0119] 11. At the same time, the display management module of the Kylin desktop operating system running on the display server system obtains the relevant display events of the corresponding main virtual display interface through the system display service, parses the relevant display events of the corresponding main virtual display interface (display events are mouse clicks, cursor positions, keyboard inputs, touch inputs, focus, etc.), writes them into the corresponding shared mapped memory, and triggers the display event interrupt to notify the Centos operating system of the corresponding display client system to read the display event information;

[0120] 12. The display virtual module of the corresponding Centos operating system running on the display client system receives the display event interrupt sent by the display server system. The display virtual module reads the display event information, parses it, and calls the relevant system components of the Centos operating system (such as keyboard processing, mouse processing, etc.) to respond to the relevant display events.

[0121] Furthermore, the corresponding Centos operating system can have software or hardware encoders, and the Kylin desktop operating system can run software or hardware decoders.

[0122] Furthermore, the layer data of the main virtual display interface and the physical display interface can be integrated with each other (multiple display layer data can be integrated together through the system display service) and displayed on the same display screen. The display events of the main virtual display interface are captured by the display management module, so that part of the content of the main virtual display interface and the physical display interface can be displayed on the same display screen at the same time, and respond to the display events of the main virtual display interface and the physical display interface respectively, so as to control the physical display interface and the slave virtual display interface of different system domains through display events on the same display screen.

[0123] Furthermore, the display server system domain can also run desktop operating systems such as ubuntu, Centos, Debian, etc., and the display client system domain can run operating systems such as ubuntu, Debian, Kylin, rtos, etc. Specifically, when the Kylin interface system runs in the display server system domain, the display client system domain runs real-time systems such as Zephyr and runs graphics display libraries such as lvgl (a graphics display library running in the rtos system, which can draw graphics windows without relying on hardware devices and combine with display events), and the real-time systems such as Zephyr running in the display client system domain do not rely on display hardware to load and run graphics display libraries such as lvgl, and provide a virtual display interface environment. The specific implementation method is similar to the above process when there is one display server system domain and it runs the Kylin desktop operating system, and there are multiple display client system domains and they run the Centos operating system (server version, without display environment), and the details will not be repeated.

[0124] The present application provides a simulated display environment in the display client domain without relying on display hardware by simulating the display environment, and can run programs that rely on the display environment, thereby improving the utilization rate of the display device, reducing the dependence of the display client system domain on the display environment, saving display hardware resources, and reducing the use of data transmission by display software related to the display interface; at the same time, a method for the RTOS system to use the Linux system display service is provided, so that the Linux system organizes and manages the RTOS system display interface.

[0125] The display sharing provided by the present application through the display interface environment simulation method between the operating systems of the system domains with different hardware isolation reduces the transmission of the driver simulation and the driver data compared with the existing scheme of driver simulation sharing at the driver layer; at the same time, the present invention greatly reduces unnecessary data communication and data packet parsing through the shared mapping memory mapping method instead of the communication method of the data packet, thereby greatly reducing the development difficulty and error rate.

[0126] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0127] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A display interface sharing and management system between isolated systems, characterized in that: include: Hardware layer, display server system domain, display client system domain and shared mapped memory, the display server system domain and the display client system domain are different system domains that isolate hardware resources by hardware isolation, wherein a display server system runs on the display server system domain, and a display client system runs on the display client system domain, the display server system controls and manages the corresponding display devices in the hardware layer, the display server system and the display client system are connected through shared mapped memory, wherein, The display server system running on the display server system domain is used to create a main virtual display interface corresponding to the slave virtual display interface of the display client system, map the framebuffer of the main virtual display interface to the corresponding shared mapping memory, write the display configuration information to the corresponding position in the shared mapping memory, obtain the slave virtual display interface data written by the display client system from the shared mapping memory, display the slave virtual display interface data in the main virtual display interface, and if the user triggers a related display event, parse the related display event and write it to the corresponding shared mapping memory position, and trigger the display event interrupt to notify the display client system to obtain it. The framebuffer is a memory space used to cache screen display data; The display client system running in the display client system domain is used to create a slave virtual display interface and read the display configuration information written by the display server system in the corresponding shared mapping memory. After the display configuration information matches, the slave virtual display interface is mapped to the corresponding shared mapping memory, and the slave virtual display interface data is written to the corresponding shared mapping memory. After receiving the display event interrupt sent by the display server system domain, the display event information is read and the relevant display event is responded to. Shared mapped memory: a memory area shared by the display server system and the display client system. The main virtual display interface, display events, display configuration information of the display server system and the secondary virtual display interface, display events, display configuration information of the display client system are respectively mapped and bound to the same corresponding shared mapped memory.

2. A display interface sharing and management system between isolation systems according to claim 1, characterized in that: The display server system includes a display management module, a display device, a system display service, a main virtual display configuration file and a physical display interface, wherein: The display server system loads and runs the system display service according to the display device; the display management module is created by the system display service after the interface environment is loaded. After the display management module is created, a main virtual display interface corresponding to the slave virtual display interface of the corresponding display client system is created according to the main virtual display configuration file; The display management module of the display server system maps the framebuffer of the main virtual display interface to the corresponding shared mapping memory according to the main virtual display configuration file, and writes the display configuration information to the corresponding position of the shared mapping memory; The display management module of the display server system receives the slave virtual display interface data interrupt and slave virtual display interface data written into the corresponding shared mapped memory by the display client system, updates the slave virtual display interface data of the corresponding shared mapped memory to the main virtual display interface when the corresponding main virtual display interface is used, and manages and processes the main virtual display interface data at the same time, and does not update the slave virtual display interface data of the corresponding shared mapped memory to the main virtual display interface when the corresponding main virtual display interface is not used; The display management module of the display server system obtains the relevant display events of the corresponding main virtual display interface through the system display service, parses the relevant display events and writes them to the corresponding shared mapped memory location, and triggers the display event interrupt notification to display the client system domain to read. The relevant display events include mouse clicks, cursor positions, keyboard inputs and touch actions.

3. A display interface sharing and management system between isolation systems according to claim 1, characterized in that: The display client system includes a virtual display environment, a display virtualization module, and a virtual display configuration file, wherein: The display client system loads and runs the virtual display environment and the display virtual module according to the virtual display configuration file, and creates a virtual display interface; After the display virtual module of the display client system successfully creates the slave virtual display interface, it reads the display configuration information written by the corresponding shared mapped memory display server system and matches it with the configuration information in the slave virtual display configuration file. After the display configuration information matches, the display virtual module maps the slave virtual display interface of the display client system to the corresponding shared mapped memory; The display virtual module of the display client system writes the corresponding data from the virtual display interface into the corresponding shared mapping memory that has been successfully matched; After the display client system receives the display event interrupt sent by the display server system, the display virtual module reads the display event information corresponding to the shared mapped memory and responds to the relevant display event.

4. A method for sharing and managing display interfaces between isolated systems, characterized in that: include: Isolating hardware resources by hardware isolation to isolate a display server system domain and a display client system domain, wherein a display server system runs on the display server system domain, and a display client system runs on the display client system domain; The display server system reads the display configuration information in the main virtual display configuration file, and initializes the shared mapping memory according to the display configuration information; The display server system creates a primary virtual display interface corresponding to the display client system, maps the framebuffer of the primary virtual display interface to the corresponding shared mapping memory, and writes the display configuration information to the corresponding position in the shared mapping memory, and the display client system creates a secondary virtual display interface, wherein the framebuffer is a memory space for caching screen display data; After the slave virtual display interface is created, the display client system reads the display configuration information written by the display server system in the corresponding shared mapping memory, and after the display configuration information matches, maps the slave virtual display interface to the corresponding shared mapping memory, and writes the slave virtual display interface data into the corresponding shared mapping memory; The display server system obtains the slave virtual display interface data written by the display client system from the shared mapped memory, and updates the slave virtual display interface data corresponding to the shared mapped memory to the master virtual display interface when necessary, and manages and processes the master virtual display interface data; When the user triggers a related display event, the display server system parses the related display event and writes it into the corresponding shared mapping memory location, and triggers a display event interrupt to notify the display client system to obtain it; After receiving the display event interrupt sent by the display server system, the display client system reads the display event information and responds to the relevant display event.

5. A method for sharing and managing display interfaces between isolation systems according to claim 4, characterized in that: The display server system includes a display management module, a display device, a system display service, a primary virtual display configuration file, and a physical display interface. The display server system creates a primary virtual display interface corresponding to the display client system, maps the framebuffer of the primary virtual display interface to the corresponding shared mapping memory, and writes display configuration information to the corresponding position in the shared mapping memory, including: The display server system domain loads and runs the system display service according to the display device; The system display service controls the display of the physical display interface. After the physical display interface environment is loaded, the display management module creates a main virtual display interface for use by the corresponding display client system according to the main virtual display configuration file. The display management module of the display server system maps the framebuffer of the main virtual display interface to the corresponding shared mapping memory according to the main virtual display configuration file, and writes the display configuration information to the corresponding position of the shared mapping memory.

6. A method for sharing and managing display interfaces between isolation systems according to claim 4, characterized in that: The display client system includes a virtual display environment, a display virtual module and a slave virtual display configuration file. After the slave virtual display interface is created, the display client system reads the display configuration information written by the display server system in the corresponding shared mapping memory, and after the display configuration information matches, maps the slave virtual display interface to the corresponding shared mapping memory, and writes the slave virtual display interface data into the corresponding shared mapping memory, including: After the display virtual module of the display client system is successfully created from the virtual display interface, the display configuration information written by the corresponding shared mapping memory display server system is read from the virtual display configuration file. After the display configuration information matches, the display virtual module maps the display client system from the virtual display interface to the corresponding shared mapping memory; The display client system writes the corresponding data from the virtual display interface into the corresponding successfully matched shared mapping memory.

7. A method for sharing and managing display interfaces between isolation systems according to claim 4, characterized in that: The display server system includes a display management module, a display device, a system display service, a main virtual display configuration file, and a physical display interface. The display server system obtains the slave virtual display interface data written by the display client system from the shared mapped memory, and updates the slave virtual display interface data of the corresponding shared mapped memory when necessary, including: The display management module of the display server system receives the slave virtual display interface data written into the corresponding shared mapped memory by the display client system; When the corresponding main virtual display interface is used, the slave virtual display interface data of the corresponding shared mapped memory is updated to the main virtual display interface; When the corresponding main virtual display interface is not used, the slave virtual display interface data corresponding to the shared mapped memory is not updated to the main virtual display interface, and the main virtual display interface data is managed and processed at the same time.

8. A method for sharing and managing display interfaces between isolated systems according to claim 4, characterized in that: The display server system includes a display management module, a display device, a system display service, a main virtual display configuration file, and a physical display interface. When a user triggers a related display event, the display server system parses the related display event and writes it into a corresponding shared mapping memory location, and triggers a display event interrupt to notify the display client system to obtain it, including: When the user triggers a related display event, the display management module of the display server system obtains the related display event of the corresponding main virtual display interface through the system display service, parses the related display event and writes it to the corresponding shared mapped memory location, and triggers the display event interrupt notification display client system to read. The related display events include mouse clicks, cursor positions, keyboard inputs and touch actions.

9. A method for sharing and managing display interfaces between isolated systems according to claim 4, characterized in that: The display client system includes a display virtual module. When the display client system receives a display event interrupt sent by the display server system, it reads the display event information and responds to the relevant display event, including: After the display client system receives the display event interrupt sent by the display server system, the display virtual module reads the display event information corresponding to the shared mapped memory and responds to the relevant display event.

Images