Screen display method and device, electronic device, and computer program

Abstract

In the embodiments of the present application, a screen display method and device, an electronic device, and a computer program are provided. The method is executed by an electronic device and can be applied to fields such as cloud technology and intelligent transportation. The method includes the steps of: generating main display virtual information for simulating a main display when receiving a display information acquisition request, the display information acquisition request being issued by a target instance, the target instance being any one instance among a plurality of instances, and the target display corresponding to the target instance being a main display or an auxiliary display; and feeding back the main display virtual information to the target instance, so as to allow the target instance to determine the target display corresponding to the target instance as a main display, and to display a corresponding screen on the target display.

Description

[Technical Field]

[0001] This application claims priority from a Chinese patent application bearing application number 202210060285.2, filed with the China Patent Office on January 19, 2022, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of computers, and more particularly to a screen display method and device, an electronic device, and a computer program. [Background technology]

[0003] With the spread of computer technology, screen display technologies that support the concurrent display of multiple processes are rapidly developing. However, multi-screen displays in related technologies obviously rely on a main (also called primary) display device. Specifically, such display technologies perform screen rendering in a parallel manner, but the final screen is all displayed on the main display device. This increases the load on the main display device, reducing the number of concurrent executions and potentially resulting in screen display abnormalities or failures. Summary of the Invention [Problem to be solved by the invention]

[0004] In order to solve at least one of the above technical problems, the embodiments of the present application aim to provide a screen display method and device, an electronic device, and a computer program that can reduce the abnormality rate in the process of a target instance sensing a display. [Means for solving the problem]

[0005] In some embodiments, embodiments of the present application provide a screen display method, the method being performed by an electronic device, the method comprising: When a display information acquisition request is received, generate main display virtual information for simulating a main display, the display information acquisition request being issued by a target instance, the target instance being any one of a plurality of instances, and the target display corresponding to the target instance being the main display or the auxiliary display; and The method includes feeding back the main display virtual information to the target instance, causing the target instance to determine the target display corresponding to the target instance as the main display, and displaying the rendered screen (also called the rendered screen) on the target display.

[0006] In some embodiments, embodiments of the present application provide a screen display device, the device comprising: a simulation module configured to generate main display virtual information for simulating a main display when receiving a display information acquisition request, the display information acquisition request being issued by a target instance, the target instance being any one instance among a plurality of instances, and the target display corresponding to the target instance being a main display or an auxiliary display; and The system includes a feedback module that feeds back the main display virtual information to the target instance, causing the target instance to determine the target display corresponding to the target instance as the main display and display the rendered screen on the target display.

[0007] In some embodiments, the embodiments of the present application provide a computer-readable storage medium, the computer-readable storage medium storing at least one instruction or at least one program, and the at least one instruction or at least one program being loaded and executed by a processor to realize the above-mentioned screen display method.

[0008] In some embodiments, embodiments of the present application provide an electronic device, which includes at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the at least one processor executes the instructions stored in the memory to realize the above-mentioned screen display method.

[0009] In some embodiments, embodiments of the present application provide a computer program product, which includes computer programs or instructions that, when executed by a processor, implement the above-described screen display method. [Effects of the Invention]

[0010] In an embodiment of the present application, the main display virtual information is simulated and sent to the target instance, so that the target instance determines the target display corresponding to the target instance as the main display and displays the rendered screen on the target display, thereby realizing isolation between different displays or different instances. The target instance can only see its corresponding target display and cannot sense the existence of other displays, which avoids various anomalies and operational failures that may occur when the target instance senses the existence of other displays, and breaks through the bottleneck of parallel performance. [Brief explanation of the drawings]

[0011] In order to more clearly explain the technical solutions and advantages in the embodiments of the present application or related technologies, the following will briefly introduce the drawings that need to be used in the description of the embodiments or related technologies. It is obvious that the drawings in the following description are only some examples for explaining the embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without any creative work. [Figure 1] 10 is a flowchart of a multi-screen display in the related art. [Figure 2]FIG. 10 is a diagram illustrating a load distribution of screen parallel display in the related art. [Figure 3] FIG. 1 illustrates a possible implementation framework of a screen display method provided in an embodiment of the present application. [Figure 4] 1 is a flowchart of a screen display method provided in an embodiment of the present application. [Figure 5] 1A-1C are diagrams illustrating screen displays provided in examples of the present application. [Figure 6] FIG. 10 is a diagram showing a comparison of the effects of the screen displays provided in the examples of the present application. [Figure 7] FIG. 1 shows stress test data provided in the examples of the present application. [Figure 8] FIG. 2 is a diagram illustrating the logic of the screen display provided in the embodiment of the present application. [Figure 9] FIG. 2 is a block diagram of a screen display device provided in an embodiment of the present application. [Figure 10] FIG. 10 is a diagram showing a display isolation device accessed by a simulation module provided in an embodiment of the present application. [Figure 11] FIG. 1 is a diagram showing the hardware configuration of an apparatus for implementing the method provided in the embodiment of the present application. [Figure 12] FIG. 10 is a diagram showing the hardware configuration of another device for implementing the method provided in the embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0012] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings of the embodiments of the present application. It is obvious that the described embodiments are only a part of the embodiments of the present application, not all of the embodiments. In addition, all other embodiments that a person skilled in the art can derive based on the embodiments of the present application without any creative work also fall within the scope of protection of the present application.

[0013] It should be noted that the terms "first," "second," etc. in the specification, claims, and drawings of this application are used to distinguish between similar objects and are not intended to limit a particular order or context. It should be understood that data used in this manner can be interchanged where appropriate, so that the embodiments of the application described herein may be practiced in orders other than those illustrated or described herein. Furthermore, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusions; for example, a process, method, system, product, or server comprising a series of steps or units is not limited to the explicitly listed steps or units, but may also include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0014] In order to make the objectives, technical solutions and advantages disclosed in the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the drawings and examples. It should be understood that the specific embodiments described herein are merely examples for interpreting the embodiments of the present application, and do not limit the embodiments of the present application.

[0015] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or the quantity (quantity) of the indicated technical features. Therefore, a feature qualified by "first" or "second" may explicitly or implicitly include one or more of the feature. In the description of the embodiments of this application, unless otherwise specified, "plurality" means two or more than two. To facilitate understanding of the technical solutions and resulting technical effects in the embodiments of this application, we first introduce the relevant terminology as follows:

[0016] 1) Intelligent Traffic System (ITS) or Intelligent Transportation System: The effective and comprehensive application of advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operation research, artificial intelligence, etc.) to traffic transportation, service control, and vehicle manufacturing, strengthening the three-way connection between vehicles, roads, and users, thereby forming a comprehensive transportation system that ensures security, improves efficiency, improves the environment, and saves energy.

[0017] 2) Intelligent Vehicle Infrastructure Cooperative Systems (IVICS (registered trademark)): Abbreviated as vehicle infrastructure cooperative systems, IVICS is one development direction for intelligent transportation systems. Vehicle infrastructure cooperative systems employ advanced wireless communications and new generation Internet technologies to achieve omnidirectional dynamic real-time information interaction between vehicles and between vehicles and roads, and perform vehicle-initiated security control and road cooperative management based on the collection and fusion of dynamic traffic information across space and time. This fully realizes effective cooperation between people, vehicles, and roads, ensures traffic security, and improves traffic efficiency, resulting in a safe, efficient, and environmentally friendly road transportation system.

[0018] 3) Cloud gaming: A gaming method based on cloud computing. In the cloud gaming execution mode, all games are executed on the server side, and the rendered game screen is compressed and transmitted to the client via the network. The client game device does not require a high-end processor or graphics card, and only requires basic video decompression capabilities to support the game execution.

[0019] 4) Application Programming Interface (API): A set of predefined functions whose purpose is to provide application programs and developers with the ability to access a set of routines based on software or hardware without needing to access the source code or understanding the details of the internal working mechanisms.

[0020] 5) Graphics card virtualization: This is a scheme that allows a single graphics card to be used by multiple virtual machines (VMs) through corresponding technology. This technology is divided into two methods: hardware and software implementations. Of these, hardware-implemented graphics card virtualization usually requires technical support from the graphics card manufacturer.

[0021] 6) Virtual Machine (VM): A complete computer system that is simulated by software, has the functionality of a complete hardware system, and runs in a completely isolated environment. A VM must run on a virtualization platform and is also called a guest machine.

[0022] 7) Container: Container technology effectively divides the resources of a single operating system into isolated groups, allowing for better balancing of conflicting resource usage needs among the isolated groups. This technology is called a container.

[0023] 8) Graphics Processing Unit (GPU): Also called a display core, visual processor, or display chip, GPU is a microprocessor specialized for image and graphics processing in personal computers, workstations, game consoles, and some mobile devices (e.g., tablet computers, smartphones, etc.). In an embodiment of the present application, the GPU can be a graphics card.

[0024] 9) Peripheral Component Interconnect Express (PCIe): A high-speed serial computer expansion bus standard. PCIe is a high-speed serial point-to-point dual-channel high-bandwidth transmission standard. Connected devices are assigned exclusive (dedicated) channel bandwidth and do not share bus bandwidth. It primarily supports functions such as active power management, error reporting, end-to-end high-reliability transmission, hot swapping, and quality of service assurance.

[0025] 10) Frames Per Second (FPS): FPS is an imaging definition that refers to the number of frames per second, or more generally, the number of frames in an animation or video. FPS is a measure of the amount of information used to store and display dynamic video. The more frames per second, the smoother the action displayed. 30 is usually the minimum to avoid jerky motion. Some computer video formats can only provide 15 frames per second.

[0026] 11) RGBA color space: A color space that represents red, green, blue, and alpha.

[0027] 12) Instance: In object-oriented program design, an instance is called an instance after it is instantiated. A class is static and does not occupy process memory, but an instance has dynamic memory. In a database, an instance represents a collection of several programs and refers to a database program that can support database operations.

[0028] Related art has proposed several schemes for displaying images. Of course, the embodiments of the present application are not limited to specific application scenarios for displaying images, and may be used in scenarios such as cloud gaming, cloud communities, cloud e-commerce, and intelligent driving. Taking cloud gaming as an example, cloud gaming is a gaming method based on cloud computing. In the cloud gaming execution mode, all games are executed on the server side, and the rendered game screen is compressed and then transmitted to the client via a network. The electronic device that executes the client does not require a high-end processor or graphics card, and can support game execution with only basic video decompression capabilities.

[0029] In a cloud gaming scenario, the server side can run one game instance under one operating system, and each game instance can serve one client, and the server side runs the game instance, compresses the displayed screen, and transmits it to the client, which can then decompress it to display the screen. Note that the embodiments of this application are not limited to game instances.

[0030] The embodiments of the present application may further be implemented based on virtual machine technology, where one operating system can run one virtualized application on the server side, and each virtualized application can correspond to one client. The virtualized application is executed on the server side, and the displayed screen of the virtualized application is compressed and transmitted to the client, where the client can decompress it to display the screen.

[0031] In some embodiments, under a cloud gaming scenario, multiple graphics cards may be configured for one server-side operating system to execute multiple game instances. The requirements for the cloud gaming environment are as follows: in a multi-graphics card system, each graphics card must be connected to one display, i.e., the graphics card must establish a one-to-one mapping with the display. The server typically provides a graphics card connected to a virtual display in a virtualization or simulation manner. See FIG. 1, which is a flowchart of a multi-screen display in the related art.

[0032] In step 01, a cloud game is executed.

[0033] In step 02, the game engine of the cloud game enumerates information of the main display device, specifically, the information of the display device may be macroscopic quantity information of the display device.

[0034] In step 03, information on each display device is acquired. Specifically, when the macroscopic quantity information is obtained, each display device in the server environment is acquired. These display devices may be physical display devices (real display devices) or virtual display devices.

[0035] In step 04, the position of the game window is set to the main display.

[0036] In step 05, the rendered screen is displayed in the game window.

[0037] After setting the position of the game window on the main display in step 04, calculation and rendering of the game screen are started, and the game screen is output to the window. Each game instance may correspond to one display (which may be either the main display or the auxiliary display, and the auxiliary display is a non-main display), and the graphics card corresponding to the display performs related calculation and rendering of the game screen, and then displays the rendering result in the game window of the main display.

[0038] As can be seen from the above description of cloud game screen display technology, in a parallel scenario of a multi-graphics card and multi-game example, the cloud game server may cause the following technical problems due to various interaction operations during the game:

[0039] First, there are performance and cost issues. Because multiple game instances all have their windows positioned on the main display and the final game screen is output to the same display, the PCIe of the GPU connected to the main display may become bottlenecked, resulting in display performance issues. Alternatively, the theoretical number of games running concurrently is relatively low, resulting in increased game execution costs. Referring to Figure 2, Figure 2 illustrates the load distribution of screen parallel display in the related art. GPU0, GPU1, GPU2, and GPU1 are graphics cards, of which GPU0 is the graphics card corresponding to the main display. Each graphics card can provide computation and screen rendering services for its corresponding game instance, but the final rendered screen can only be displayed in a window on the main display corresponding to GPU0, resulting in a high load on GPU0. For example, if a game requires a resolution of 1920*1080, a frame rate of 60 FPS, and the size of the RGBA graphics data is 1920*1080*4*60 / (1024*1024)=475 megabytes (MB), and the theoretical transmission speed of PCIe Gen3 x16 is 16,000 megabytes per second (GB / s), the theoretical number of concurrent executions that can be achieved based on the above settings is approximately 30, which is severely limited.

[0040] In addition, there may be compatibility issues, such as some games not launching or window positions being incorrect. Please note that there may be differences in behavior and expression between different game instances.

[0041] In order to solve the above-mentioned problems of performance, cost and compatibility, an embodiment of the present application provides a screen display method, which can realize thorough isolation between graphics cards and between instances, not only realizing parallelism at the screen rendering level, but also realizing separation in screen display, thereby avoiding the display of multiple screens on the same main display, not only breaking through performance bottlenecks and limitations on the number of concurrent executions, but also avoiding the emergence of compatibility problems.

[0042] Embodiments of this application may involve cloud technology. Cloud technology refers to a hosting technology that integrates a series of resources, such as hardware, software, and networks, over a wide area network or a local area network to achieve data computation, storage, processing, and sharing. Cloud technology is a collective term for network technology, information technology, integration technology, management platform technology, application technology, etc., based on the application of cloud computing commercial models, forming a resource pool that can be used as needed, flexibly, and conveniently. Cloud computing technology is expected to provide essential support. Background services in technical network systems require large amounts of computational and storage resources, such as video websites, picture websites, and more portal websites. With the rapid development and application of the Internet industry, in the future, each item will have its own identifier and will need to be transmitted to a background system for logic processing. Furthermore, data of different degrees and levels will be separated and processed, and data from various industries will require powerful system support, which must be realized by cloud computing.

[0043] The embodiments of this application may be applied to various cloud applications based on cloud technology, such as medical clouds. Medical clouds refer to the use of cloud computing, mobile technology, multimedia data, 4G communications, big data, IoT, and other new technologies in conjunction with medical technology to build a medical and health service cloud platform, enabling the sharing of medical resources and the expansion of medical coverage. By applying and combining cloud computing technology, medical clouds can improve the service efficiency of medical institutions and make medical care more convenient for residents. For example, hospital appointment registration and medical insurance accounting are all applications that combine cloud computing with the medical field, and medical clouds also offer advantages such as data security, information sharing, dynamic expansion, and global layout.

[0044] Referring to FIG. 3, FIG. 3 illustrates a possible implementation framework for the screen display method provided in the embodiment of the present application. As shown in FIG. 3, the implementation framework may include at least a terminal device 01 and a screen display server 02. The terminal device 01 may be an electronic device on the Internet, which can provide various Internet-based services for users, and the services are provided by a client in the embodiment of the present application. The terminal device 01 may include various types of user terminals, such as, but not limited to, laptops, tablet computers, desktop computers, set-top boxes, mobile devices, and in-vehicle terminals (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, and portable game consoles). The client in the embodiment of the present application can provide applications in various scenarios, such as, but not limited to, cloud technology, cloud gaming, cloud rendering, artificial intelligence, intelligent transportation, driver assistance, video media, intelligent communities, and instant communication.

[0045] The screen display server 02 interacts with the terminal device 01 to provide services to the user. Specifically, the terminal device 01 can request a screen from the screen display server 02. In this case, a target instance corresponding to the terminal device 01 and running on the screen display server 02 can issue a display information acquisition request. Upon receiving the display information acquisition request, the screen display server 02 generates main display virtual information through simulation. The display information acquisition request is issued by a target instance corresponding to the target display, which may be a main display or an auxiliary display. The main display virtual information is fed back to the target instance, causing the target instance to determine the target display as the main display and display the rendered screen on the target display. The target instance compresses the displayed screen and transmits it to the terminal device, allowing the terminal device to display the screen in a decompressed format. In this way, the client does not need to have screen calculation or screen rendering capabilities, only basic decompression capabilities are required.

[0046] The screen display method in the embodiment of the present application is introduced below: The information in the embodiment of the present application may be authorized by the user or may be information fully authorized by each party.

[0047] FIG. 4 is a flowchart of a screen display method provided in an embodiment of the present application. Although the embodiment of the present application provides the operational steps of the method as shown in the embodiment or flowchart, more or fewer operational steps may be included based on common sense or creativity. The order of the steps listed in the embodiment is only one of multiple ways of performing the steps and does not represent the only execution order. When executed by an actual system, terminal device, or server product, the method may be executed according to the order of the embodiment or the method shown in the drawings, or may be executed in parallel (e.g., in a parallel processor or multi-processing environment). The above-mentioned method may also include the following steps:

[0048] In step S101, main display virtual information for simulating a main display is generated when a display information acquisition request is received, where the display information acquisition request is issued by a target instance, the target instance is any one instance among multiple instances, and the target display corresponding to the target instance is a main display or an auxiliary display.

[0049] For example, when a display information acquisition request is received, main display virtual information is generated by simulation, and the above-mentioned display information acquisition request is issued by a target instance corresponding to a target display, and the above-mentioned target display is a main display or a non-main display.

[0050] In a multi-instance scenario, each instance may be assigned a graphics card, and the corresponding graphics card may perform its associated computation or screen rendering. By assigning the corresponding graphics card resources to each instance, the computation or screen rendering of each instance can be performed independently and in parallel without affecting each other. The premise for each graphics card to perform computation or screen rendering is that it is associated with its corresponding display. In the embodiments of the present application, the display associated with the graphics card is not limited, and it may be a physical display or a virtual display. In the embodiments of the present application, a target instance may be any one instance running on a server. For a target instance, the server can assign it its appropriate target graphics card and target display, which may be a main display or an auxiliary display. For example, suppose a server is configured with four graphics cards, A, B, C, and D, each corresponding to four displays, A1, B1, C1, and D1, of which A1 is the main display and B1, C1, and D1 are auxiliary displays. If graphics card B can be associated with instance B-10, and instance B-10 is the target instance, then the target display is B1, which is obviously the auxiliary display. If graphics card A can be associated with instance A-10, and instance A-10 is the target instance, then the target display is A1, which is obviously the main display.

[0051] A target instance can communicate with its corresponding client, and in response to a client sending a screen request to its corresponding target instance, the target instance corresponding to the client issues a request to obtain display information to the screen display server.

[0052] In some embodiments, the display information acquisition request includes a display quantity acquisition request, and the main display virtual information includes a display quantity or a main display indicator (ID) feature. Generating the main display virtual information for simulating the main display when receiving the display information acquisition request can be realized by the following technical solution: when receiving the display quantity acquisition request, obtain the display quantity or the main display indicator feature from the display isolation device, where the display quantity is the display quantity that needs to be fed back in the case of the first target, and in the case of the first target, a screen corresponding to the target instance is displayed on the main display, and the main display indicator feature is used to indicate that the target display is the main display.

[0053] For example, when the above-mentioned display quantity acquisition request is received, the display isolation device is accessed to obtain the above-mentioned display quantity or the above-mentioned main display indicator characteristic output from the above-mentioned display isolation device, and the above-mentioned display isolation device is used to output the display quantity that needs to be fed back when displaying a screen corresponding to the above-mentioned target instance on the main display, or to output the main display indicator characteristic to indicate that the above-mentioned target display is the main display.

[0054] For example, the display isolation device should have the following key capabilities: when obtaining the total number of displays, it only needs to return one each time, and there is no need to query the display data. Of course, in practice, the number of displays may be greater than 1, which depends on the actual number of graphics cards and displays in the server environment. When obtaining the determination information indicating whether the current display is the main display, it only returns the determination information indicating that the current display is the main display, and the determination information is the indicator characteristic of the main server, so the target instance may mistakenly identify the display corresponding to the indicator characteristic of the main server as the main display.

[0055] In some embodiments, transmitting a screen corresponding to the target instance to the main display includes transmitting a screen corresponding to the target instance to the main display and the main display being the only display, where the target instance mistakenly believes that there is only one display in the server environment and this display is the main display.

[0056] In the embodiment of the present application, there is no limitation on the number of indicators or the main indicator characteristic indicator to be fed back. It is sufficient to use the indicator number or the main indicator characteristic indicator generated by simulation to confuse the target instance. The confusion can be achieved by confusing the target instance so that it mistakenly believes its corresponding target indicator to be the main indicator. For example, if the number of indicators fed back is 1 and the main indicator characteristic indicates that the indicator currently associated with the target instance is the main indicator, the target instance can assume that only one indicator exists in the current server environment. This indicator is the main indicator. Since the target instance knows that the target indicator exists in the current server environment, the target instance can naturally assume that the target indicator is the main indicator, thereby achieving the confusion.

[0057] In some embodiments, the display information acquisition request includes a main display position feature request, and the main display virtual information includes the main display position feature. Generating the main display virtual information for simulating the main display when receiving the above-mentioned display information acquisition request can be realized by the following technical solution: when receiving the main display position feature request, the main display position feature is obtained from the display isolation device, in which the display isolation device is used to determine the main display position feature that needs to be fed back in the case of the second target, and in the case of the second target, a screen corresponding to the target instance is transmitted to the main display.

[0058] For example, when the above-mentioned main display position characteristic request is received, the display isolation device is accessed to obtain the main display position characteristic output from the above-mentioned display isolation device, and the above-mentioned display isolation device is used to determine the above-mentioned main display position characteristic that needs to be fed back when transmitting a screen corresponding to the above-mentioned target instance to the main display.

[0059] Specifically, when a screen corresponding to the target instance is transmitted to the main display, this includes the case where the screen corresponding to the target instance is transmitted to the main display and the main display is the only display, where the target instance mistakenly believes that there is only one display in the server environment and this display is the main display.

[0060] In the embodiment of the present application, the main display position characteristic to be fed back is not limited, and the goal of confusing the target instance can be achieved by using the simulated main display position characteristic. The confusion goal can be achieved by confusing the target instance so that it considers its corresponding target display to be the main display. For example, if the main display position characteristic to be fed back is the position of the main display when there is only one main display and no other displays in the server environment, the target instance can consider that there is only one display in the current server environment, and this display is the main display. If the target instance knows that the target display exists in the current server environment, the target instance can naturally consider the target display to be the main display, thereby achieving the confusion goal.

[0061] In the above two embodiments, the main display mark, the position, and the number of displays are simulated, but of course, other confusion methods may be adopted in the embodiments of the present application, depending on the specific information about the main display that the target instance needs to obtain. In the embodiments of the present application, the purpose of confusion can be achieved by simulating virtual information about the main display, so that the target instance can mistakenly believe that there is only one display in the server environment and that this display is the main display.

[0062] Taking cloud gaming as an example, cloud gaming typically obtains display device information through the following steps:

[0063] First, obtain the display quantity. For example, the target game instance can obtain the display quantity through an application program interface. For example, the application program interface here may be a "computer-only display function (GetSystemMetrics(SM_CMONITORS))" and a "multi-display function (EnumDisplayMonitors / EnumDisplayDevices)." In this case, by hijacking the API, the API call can be intercepted and the display isolation device can be accessed to obtain the display quantity. In fact, only one value needs to be returned each time. This achieves the purpose of confusion, since the target instance cannot obtain the relevant data of the display in a real server environment.

[0064] Next, after obtaining the display number, the target game instance may use the API: EnumDisplaySetting to obtain display information. In this case, by intercepting the API and then communicating with the display isolation device, corresponding information such as whether it is the main display, coordinates, resolution, etc. can be obtained. The isolation device can virtualize the coordinates of each display as (0,0) and each display as the main display. In practice, the display information on the server is not enumerated.

[0065] In the embodiment of the present application, the display isolation device in the cloud gaming scenario needs to have the following key capabilities:

[0066] When cloud gaming gets the total number of displays, it actually only needs to return one each time and does not query the display data. Of course, in actual cases, the number of displays in the server environment may be greater than 1, which depends on the actual number of graphics cards and displays in the server environment.

[0067] When the cloud game obtains whether the current display is the main display, it only informs it that it is the main display, i.e., feeds back the marking characteristics of the main server, causing it to mistakenly recognize its corresponding display as the main display.

[0068] When the cloud game gets the current display coordinate information, it informs it that the coordinate information is (0,0), and the true coordinate information is not (0,0).

[0069] Of course, other related main display characteristics can also be simulated according to needs to achieve the purpose of confusing the target instance.

[0070] By performing the above steps, the target instance can only see the display of the graphics card it runs on, and it considers it to be the main display, and it has no idea that other displays exist in the current server environment, so it will not interact with other displays, thereby achieving complete isolation between displays in a way that confuses the target instance.

[0071] In step S102, the main display virtual information is fed back to the target instance, causing the target instance to determine the target display corresponding to the target instance as the main display, and causing the target display to display the rendered screen.

[0072] For example, by feeding back the above-mentioned main display virtual information to the above-mentioned target instance, the above-mentioned target instance determines the above-mentioned target display as the above-mentioned main display and displays the rendered screen on the above-mentioned target display.

[0073] The target instance sends rendered data only to the main display, which can trigger the main display to display its rendered screen, and the target instance mistakenly identifies its corresponding target display as the main display; in this way, the target instance can display rendered screens only on the target display, and these screens will not be transmitted to and displayed on the true main display, thereby achieving thorough isolation of the displays.

[0074] In a cloud gaming scenario, the server can inform the target instance running therein that there is only one display and that this display is the main display, and by informing it of the display's location, attribute information, etc., confuse the target instance and cause it to mistakenly believe that the target display associated with it is the main display, so that it can send it a rendered screen.

[0075] Referring to Figure 5, which illustrates a screen display in an embodiment of the present application, GPU0, GPU1, GPU2, and GPUN represent graphics cards, each of which has a corresponding display, with the primary display being the main display and the others being auxiliary displays. Whether the primary display or the auxiliary display is the corresponding instance, it may be confused when performing the method in the embodiment of the present application, leading to the mistaken belief that the display corresponding to that instance is the only main display and that there are no other displays. This allows the rendered screen to be displayed on the display corresponding to that instance, rather than being transmitted to the true main display for display as in Figure 2, thereby achieving load balancing and preventing the true main display from being overloaded. Comparing Figures 2 and 5 reveals differences in display effect between the embodiment of the present application and the related technology. The game instance running on each GPU is only related to the virtual display connected to the current GPU, so there is no problem of the PCIe load on the GPU connected to the main display becoming too heavy, and each GPU can be completely independent of the virtual display.

[0076] In an embodiment of the present application, the target instance performs screen rendering on a target graphics card corresponding to the target display, and displays the rendered screen on a display determined based on the main display virtual information. In an embodiment of the present application, since the display determined based on the main display virtual information is the target display associated with the target instance, after the screen is calculated and rendered on the graphics card associated with the target instance, the screen can be directly displayed on the associated target display, without the need to transmit the screen to the true main display for display. This improves display speed, isolates the display screen, and reduces the load on the true main display.

[0077] Specifically, in step S102, the target instance determines its corresponding target display as the main display, which actually means that the target instance determines the target display as the main display based on the main display virtual information. In step S102, the target instance displays the rendered screen on the target display, which actually means that the target display sets a window position corresponding to the screen in the target display and then transmits the screen to the window position of the target display.

[0078] Refer to Figure 6, which shows a comparison of screen display effects. The upper diagram in Figure 6 illustrates the screen display effects of the related art. As can be seen, the screens corresponding to each example are all displayed on the main display, and although the screens can be rendered in parallel by each graphics card, they all ultimately need to be transmitted to and displayed in a window on the main display. This places a significant burden on the PCIe bandwidth connected to the main display. Refer to Figure 7, which shows load test data. Figure 7 shows test data for the PCIe 3.0x8 version. The theoretical PCIe bidirectional bandwidth is 8GB / s, but the actual maximum is 6GB / s. The current usage reaches more than 5GB / s, which fully illustrates the bottleneck phenomenon of bandwidth usage in the related art.

[0079] The bottom diagram in Figure 6 illustrates the screen display effect in an embodiment of the present application. As can be seen, the screen rendered by each instance is displayed on the display device corresponding to that instance, and each screen does not need to be transmitted to and displayed on the main display device. Isolation between displays is realized. Specifically, this is thorough isolation between graphics cards in a multi-graphics card-multi-display scenario, and between different instances corresponding to different clients within different clients or servers, achieved in an embodiment of the present application based on the principle of virtual display implementation.

[0080] Please refer to FIG. 8, which is a diagram showing the logic of the screen display in an embodiment of the present application.

[0081] In step 11, the cloud game is executed.

[0082] In step 12, the display information is enumerated.

[0083] In step 13, the indicator isolation device is accessed.

[0084] In step 14, the display isolation device outputs the main display virtual information.

[0085] In step 15, the target display is determined as the main display based on the main display virtual information, and the window position is set on the main display.

[0086] In step 16, the rendered screen is displayed at the window position.

[0087] Before displaying a screen, a display information acquisition request is received, and after intercepting the request, a display isolation device is accessed and display information is returned based on the access result. The display information can confuse the target instance, causing the target instance to mistakenly recognize the corresponding target display as the main display, set a window position on the target display, and display the rendered screen at the window position of the target display. Furthermore, the displayed screen can be compressed and transmitted to the client, allowing the client to display the screen.

[0088] In the embodiments of the present application, a screen display scheme is provided, which simulates the virtual information of the main display to confuse the target instance, so that the target instance determines the corresponding target display as the main display and displays the rendered screen on the target display, thereby realizing isolation between different displays and different instances. Specifically, this scheme allows the target instance to only see its corresponding target display and cannot sense the existence of other displays, thereby avoiding various problems such as abnormalities and operation failures that may occur when the target instance senses the existence of other displays, and breaking through the bottleneck of parallel performance.

[0089] Taking cloud gaming as an example, embodiments of the present application can achieve thorough isolation of the display device, and when data output from the game reaches the display device, the data only passes through its own PCIe, eliminating the PCIe bandwidth bottleneck problem of a certain graphics card and ensuring high performance and high concurrency. In a cloud gaming environment, games are executed locally, providing better compatibility and supporting a larger number of games. Of course, the embodiments of the present application are not limited to specific application scenarios and can be applied not only to cloud gaming scenarios but also to traditional gaming scenarios with multiple graphics cards and multiple displays. The embodiments of the present application are not limited to specific operating systems and can support not only the Windows operating system but also the Linux operating system or other multi-graphics card and multi-display platforms.

[0090] Please refer to FIG. 9, which is a block diagram of the screen display device in this embodiment, which includes:

[0091] A simulation module 101 is configured to generate main display virtual information for simulating a main display when receiving a display information acquisition request, where the display information acquisition request is issued by a target instance, the target instance is any one instance among a plurality of instances, and the target display corresponding to the target instance is the main display or the auxiliary display; and Feedback module 102: Configured to feed back main display virtual information to the target instance so that the target instance determines its corresponding target display as the main display and displays the rendered screen on the target display.

[0092] In some embodiments, the display information acquisition request includes a display quantity acquisition request, and the main display virtual information includes a display quantity or a main display indicator feature, and the simulation module is configured to perform the following operation: when receiving a display quantity acquisition request, acquire the display quantity or the main display indicator feature from the display isolation device, where the display quantity is the display quantity that needs to be fed back in the case of the first target, and in the case of the first target, a screen corresponding to the target instance is displayed on the main display, and the main display indicator feature is used to indicate that the target display is the main display.

[0093] In some embodiments, the display information acquisition request includes a main display position feature request, and the main display virtual information includes the main display position feature, and the simulation module is configured to perform the following operation: when receiving the main display position feature request, obtain the main display position feature from the display isolation device, in which the display isolation device is used to determine the main display position feature that needs to be fed back in the case of the second target, and in the case of the second target, transmit a screen corresponding to the target instance to the main display.

[0094] 10, which shows a display isolation device accessed by a simulation module. It includes a main display feature simulation module 100 and a display position information simulation module 200. The main display feature simulation module 100 is configured to output a display quantity or a main display indicator feature. The display position information simulation module 200 is configured to output a main display position feature.

[0095] In some embodiments, transmitting a screen corresponding to a target instance to a main display includes transmitting a screen corresponding to a target instance to a main display and the main display being the only display.

[0096] In some embodiments, the target instance is used to perform screen rendering on a target graphics card corresponding to the target display and display the resulting rendered screen on a display determined based on the main display virtual information.

[0097] In some embodiments, the target instance is further configured to perform the following operations: determine the target display as the main display based on the main display virtual information; and set a window position corresponding to the screen on the target display, and transmit the screen to the window position of the target display.

[0098] The apparatus embodiment of the present application is based on the same technical concept as the method embodiment, and therefore will not be described in detail here.

[0099] In an embodiment of the present application, a computer program product or a computer program is further provided, the computer program product or the computer program including computer instructions stored in a computer-readable storage medium, wherein a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to cause the computer device to perform the screen display method.

[0100] An embodiment of the present application further provides a computer-readable storage medium, which can store a plurality of instructions, which can be loaded by a processor to perform the screen display method in the embodiment of the present application.

[0101] In some embodiments, the screen display method includes generating main display virtual information for simulating a main display when receiving a display information acquisition request, where the display information acquisition request is issued by a target instance, the target instance being any one instance among a plurality of instances, and the target display corresponding to the target instance being the main display or the auxiliary display; and feeding back the main display virtual information to the target instance, so that the target instance determines the target display corresponding to the target instance as the main display, and causes the target display to display the rendered screen.

[0102] In some embodiments, the display information acquisition request includes a display quantity acquisition request, and the main display virtual information includes a display quantity or a main display indicator feature, and when the display information acquisition request is received, generating main display virtual information for simulating the main display includes obtaining the display quantity or the main display indicator feature from the display isolation device when the display quantity acquisition request is received, wherein the display quantity is the display quantity that needs to be fed back in the case of the first target, and when the first target is the case where a screen corresponding to the target instance is displayed on the main display, and the main display indicator feature is used to indicate that the target display is the main display.

[0103] In some embodiments, the display information acquisition request includes a main display position characteristic request, and the main display virtual information includes the main display position characteristic; when receiving the display information acquisition request, generating main display virtual information for simulating the main display includes obtaining the main display position characteristic from a display isolation device when receiving the main display position characteristic request, wherein the display isolation device is used to determine the main display position characteristic that needs to be fed back in the case of a second target, and in the case of the second target, transmitting a screen corresponding to the target instance to the main display.

[0104] In some embodiments, transmitting a screen corresponding to a target instance to a main display includes transmitting a screen corresponding to a target instance to a main display and the main display being the only display.

[0105] In some embodiments, before displaying the rendered screen on the target display, the method further includes the target instance rendering the screen on a target graphics card corresponding to the target display.

[0106] In some embodiments, the target instance determining a target display corresponding to the target instance as a main display includes the target instance determining the target display as the main display based on the main display virtual information, and displaying the rendered screen on the target display includes setting a window position corresponding to the screen on the target display and transmitting the screen to the window position of the target display.

[0107] In some embodiments, FIG. 11 illustrates a hardware configuration of a device (electronic device) for implementing a method provided in an embodiment of the present application. The device may participate in or include the configuration of a device or system provided in an embodiment of the present application. As shown in FIG. 11, the device 10 may include one or more processors 102 (denoted as 102a, 102b, ..., 102n in the figure) (the processors 102 may include, but are not limited to, a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for performing communication functions. In addition, the device 10 may further include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. As will be understood by those skilled in the art, the configuration illustrated in FIG. 11 is merely exemplary and does not limit the configuration of the electronic device. For example, device 10 may also include more or fewer components than those shown in FIG. 11, or may have a different arrangement than that shown in FIG.

[0108] It should be noted that one or more processors 102 and / or other data processing circuits may be generally referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination. The data processing circuitry may also be a separate processing module, or may be combined in whole or in part with any one of the other elements in device 10 (or a mobile device). In an embodiment of the present application, the data processing circuitry functions as a processor control (e.g., selection of variable resistance terminal paths connected to an interface).

[0109] The memory 104 is used to store software programs and modules of application software, such as a storage device for program instructions / data corresponding to the methods of the embodiments of the present application. The processor 102 executes the software programs and modules stored in the memory 104 to perform various functions and data processing, thereby realizing the screen display method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some embodiments, the memory 104 may further include memory devices located remotely from the processor 102, and these remote memory devices may be connected to the device 10 via a network. Examples of networks include, but are not limited to, the Internet, a corporate intranet, a local area network, a mobile communication network, and combinations thereof.

[0110] The transmitting device 106 may be used to receive or transmit data over a network. An example of a network may include a wireless network provided by the device 10's communications provider. In one example, the transmitting device 106 includes a network interface controller (NIC), which may be connected to other network devices via a base station to communicate with the Internet. In one example, the transmitting device 106 may be a radio frequency (RF) module, which may be used to communicate with the Internet wirelessly.

[0111] The display may be, for example, a touch-sensitive liquid crystal display (LCD) that allows a user to interact with the user interface of device 10 (or a mobile device).

[0112] FIG. 12 is a block diagram of another electronic device according to an embodiment of the present application. For example, the electronic device 1900 can be provided as a server. As shown in FIG. 12, the electronic device 1900 includes a processing assembly 1922, which includes one or more processors, and a storage resource, a memory 1932, which is used to store instructions executable by the processing assembly 1922, such as an application program. The application program stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. The processing assembly 1922 is also configured to execute the instructions to implement a method.

[0113] The electronic device 1900 further includes a power supply assembly 1926 configured to perform power management for the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 runs an operating system, such as Windows Server 2008 R2, stored in a memory 1932.TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM It can operate based on the following.

[0114] In an exemplary embodiment, a non-volatile computer-readable storage medium is further provided, such as a memory 1932 containing computer program instructions that can be executed by the processing assembly 1922 of the electronic device 1900 to implement a method.

[0115] The order of the embodiments herein is for illustrative purposes only and does not represent the merits or demerits of the embodiments. While specific embodiments of the embodiments herein have been described, other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that shown in the embodiments and still achieve desirable results. Processes depicted in the figures do not necessarily achieve desirable results by performing them in the particular order or sequential order shown. In some implementations, multitasking and parallel processing may also be possible or advantageous.

[0116] The embodiments in the present application are described in a stepwise manner, and the same or similar parts between the embodiments can be referred to, and the main description in each embodiment is about the differences from other embodiments. In particular, the device and server embodiments are almost similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the corresponding description in the method embodiments.

[0117] As can be understood by those skilled in the art, all or part of the steps of the embodiments may be realized by hardware, or may be realized by instructing related hardware by a program. The program may be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk, or an optical disk.

[0118] Although the preferred embodiment of the present application has been described above, the present application is not limited to this embodiment, and any modification to the present application falls within the technical scope of the present application as long as it does not depart from the spirit of the present application.

Claims

1. A method for displaying a screen, executed by an electronic device, comprising: generating main display virtual information for simulating a main display when a display information acquisition request is received, the display information acquisition request being issued by a target instance, the target instance being any one instance among a plurality of instances, and the target display corresponding to the target instance being the main display or the auxiliary display; and The method includes a step of feeding back the main display virtual information to the target instance, causing the target instance to determine a target display corresponding to the target instance as the main display, and displaying a rendered screen on the target display; causing the target instance to display the rendered screen on the target display, setting a window position in the target display corresponding to the screen, and transmitting the screen to the target display at the window position.

2. 2. The method of claim 1, The display information acquisition request includes a display quantity acquisition request, and the main display virtual information includes a display quantity or a main display indicator feature; generating main display virtual information for simulating the main display when the display information acquisition request is received, The method includes a step of reading the display quantity or the main display indicator characteristic from a display isolation device when the display quantity acquisition request is received, wherein the display quantity is the display quantity that needs to be fed back in the case of a first target, and in the case of the first target, a screen corresponding to the target instance is displayed on the main display, and the main display indicator characteristic is used to indicate that the target display is the main display.

3. 2. The method of claim 1, The display information acquisition request includes a main display position characteristic request, and the main display virtual information includes a main display position characteristic; generating main display virtual information for simulating the main display when the display information acquisition request is received, A method comprising the step of obtaining the main display position characteristics from a display isolation device when the main display position characteristics request is received, the display isolation device being used to determine the main display position characteristics that need to be fed back in the case of a second target, where the second target is when a screen corresponding to the target instance is transmitted to the main display.

4. 4. The method according to claim 2 or 3, The method wherein the main display is the only display.

5. 5. The method of claim 4, Before displaying the rendered screen on the target display, the method includes: The method further comprising the step of performing screen rendering on a target graphics card corresponding to the target display.

6. 2. The method of claim 1, causing the target instance to determine a target display corresponding to the target instance as the main display, The method includes the target instance determining the target display as a main display based on the main display virtual information.

7. A device for displaying a screen, a simulation module configured to generate main display virtual information for simulating a main display when receiving a display information acquisition request, the display information acquisition request being issued by a target instance, the target instance being any one instance among a plurality of instances, and the target display corresponding to the target instance being a main display or an auxiliary display; and a feedback module configured to feed back the main display virtual information to the target instance, causing the target instance to determine a target display corresponding to the target instance as the main display and display a rendered screen on the target display; causing the target instance to display the rendered screen on the target display, setting a window position corresponding to the screen on the target display, and transmitting the screen to the window position on the target display.

8. An electronic device, at least one processor; and a memory coupled to at least one of said processors; The storage device stores a computer program, 4. An electronic device, wherein the processor is configured to execute the computer program to implement the method of any one of claims 1 to 3.

9. A program for causing a computer to execute the method according to any one of claims 1 to 3.

Images