Screen projection method, device, electronic device and computer program product

Abstract

The embodiments of the present application provide a screen projection method, device, electronic device, and computer program product, which relate to the field of screen projection technology, wherein a screen projection method includes: in response to satisfying a screen projection trigger condition, sending a first control instruction to a virtual driver, so that the virtual driver responds to the first control instruction, sequentially reads each frame of desktop image data from the target memory allocated for the virtual display, and obtains each frame of desktop image data read by the virtual driver; based on each frame of desktop image data obtained, sends the projection data of the screen to be projected to the projection display terminal for display by the projection display terminal. It can be seen that this solution can realize the normal display of the screen projected by the projection terminal in the projection display terminal.

Description

Technical Field

[0001] The present application relates to the field of screen projection technology, and in particular to a screen projection method, device, electronic device and computer program product. Background Art

[0002] In a work environment, users often use documents such as PPT (PowerPoint, presentation), Word, etc. with read-only permissions.

[0003] In the related art, when a user uses a projection terminal to project a screen to a projection display terminal, it is necessary to use the interface provided by the operating system to capture the screen data. However, the operating system has a graphic protection mechanism for documents with read-only permissions. Therefore, if the screen to be projected contains the page content of a document with read-only permissions, the document portion of the captured screen data will appear black. Ultimately, when the document is displayed on the projection display terminal, the document display area will appear black, and the user will not be able to view the projected screen normally using the projection display terminal.

[0004] Currently, there is an urgent need for a screen projection method to enable the normal display of the screen projected by the projection end on the projection display end. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a screen projection method, device, electronic device, and computer program product to enable the normal display of the screen projected by the projection terminal on the projection display terminal. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides a screen projection method, which is applied to a screen projection end, wherein the screen projection end is a screen projection client in an electronic device or a screen projection device connected to the electronic device, and the screen projection end communicates with a virtual driver in the electronic device; the method includes:

[0007] In response to satisfying a screen projection trigger condition, a first control instruction is sent to the virtual driver, so that the virtual driver responds to the first control instruction and sequentially reads each frame of desktop image data from the target memory allocated for the virtual display; wherein the screen projection trigger condition includes receiving a screen projection instruction for a screen to be projected, and the screen to be projected is a screen in the target desktop and contains page content of a target document; the virtual display is created by the virtual driver, and after the virtual display is created, the operating system of the electronic device obtains the desktop image data of the target desktop and stores each frame of the obtained desktop image data in the target memory as virtual display data of the virtual display;

[0008] Acquiring each frame of desktop image data read by the virtual driver;

[0009] Based on the obtained desktop image data of each frame, the projection data of the screen to be projected is sent to the projection display end for display by the projection display end.

[0010] In a second aspect, an embodiment of the present application provides a screen projection device, which is applied to a screen projection end, wherein the screen projection end is a screen projection client in an electronic device or a screen projection device connected to the electronic device, and the screen projection end communicates with a virtual driver in the electronic device; the device includes:

[0011] A first sending module is configured to send a first control instruction to the virtual driver in response to a screen projection trigger condition being met, so that the virtual driver reads each frame of desktop image data from the target memory allocated for the virtual display in sequence in response to the first control instruction; wherein the screen projection trigger condition includes receiving a screen projection instruction for a screen to be projected, and the screen to be projected is a screen in the target desktop that contains the page content of the target document; the virtual display is created by the virtual driver, and after the virtual display is created, the operating system of the electronic device obtains the desktop image data of the target desktop and stores each frame of the obtained desktop image data in the target memory as virtual display data of the virtual display;

[0012] An acquisition module, configured to acquire each frame of desktop image data read by the virtual driver;

[0013] The second sending module is used to send the projection data of the screen to be projected to the projection display end based on the obtained desktop image data of each frame for display by the projection display end.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0015] Memory for storing computer programs;

[0016] The processor is used to implement any of the above-mentioned screen projection methods when executing the program stored in the memory.

[0017] In a fourth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any of the above-mentioned screen projection methods when executed by a processor.

[0018] Beneficial effects of the embodiments of the present application:

[0019] In the screen projection method provided by the embodiment of the present application, in response to the screen projection trigger condition being met, the screen projection end can send a first control instruction to the virtual driver, so that the virtual driver sequentially reads each frame of desktop image data from the target memory allocated for the virtual display; and the screen projection end can obtain each frame of desktop image data read by the virtual driver to realize the screen projection of the screen to be projected based on each frame of desktop image data. Compared with the related art, when projecting the screen to be projected, the present solution does not need to use the interface provided by the operating system to capture the screen data of the screen, so that the operating system will not trigger the graphics protection mechanism for documents with read-only permissions; specifically, the screen projection end of the present solution can use the virtual driver to obtain each frame of desktop image data stored by the operating system for the virtual display, and since each frame of desktop image data is virtual display data of the virtual display, each frame of desktop image data is normal display data. Therefore, when projecting based on each frame of desktop image data, the projected screen can be displayed normally, and the document display area will not be displayed as a black screen when the screen projection display end displays the screen to be projected. It can be seen that through this solution, the screen projected by the projection end can be displayed normally on the projection display end.

[0020] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0022] Figure 1 A flowchart of a screen projection method based on a system interface provided in an embodiment of the present application;

[0023] Figure 2 A flowchart of a screen projection method provided in an embodiment of the present application;

[0024] FIG3 (a) is a flow chart of another screen projection method provided in an embodiment of the present application;

[0025] FIG3 (b) is a schematic diagram showing a principle of applying a virtual display according to an embodiment of the present application;

[0026] FIG3 (c) is a schematic diagram showing another principle of applying a virtual display according to an embodiment of the present application;

[0027] Figure 4 A schematic structural diagram of a screen projection device provided in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0030] First, some professional terms in the embodiments of this application are introduced:

[0031] Screen projection: The projection end communicates with the projection display end. After entering the projection code on the projection display end, the screen of the projection end can be displayed on the projection display end to realize screen projection.

[0032] Reverse control: After the screen is successfully projected, the projection display end can operate the screen of the projection end.

[0033] Read-only documents: Based on the operating system's permission management technology, administrators can grant other users read and write permissions for documents. For users granted read-only permissions, all documents opened by this user are read-only documents.

[0034] Black screen: For users with read-only permissions, when a read-only document is displayed on the desktop and then the desktop is captured using the standard interface provided by the operating system, the document display area appears black, while other areas are normal.

[0035] Secondly, in order to better understand this solution, the following is an introduction to a screen projection method based on a system interface introduced in an embodiment of the present application in conjunction with the accompanying drawings. Figure 1 As shown:

[0036] Figure 1 The scenario targeted is: a scenario where a screen projection end of a personal computer (PC) device projects its screen to a screen projection display end of another PC device. The operating system may be a Windows system. In this embodiment, both the screen projection end and the screen projection display end may be in the form of software.

[0037] S101, click on the screen projection option;

[0038] It is understandable that users can click the screen projection option on the projection end of the PC device to initiate screen projection.

[0039] S102, calling DXGI to capture screen data;

[0040] It is understandable that the Windows system can call DXGI (DirectX Graphics Infrastructure) to capture (intercept) the screen data on the projection end. The screen data can be an image captured of the document displayed on the screen, and can also be called desktop image data; among them, DirectX is the abbreviation of DirecteXtension, which is a set of cross-platform multimedia programming interfaces.

[0041] S103, DXGI interacts with the graphics card driver in user mode to output the captured screen data to the buffer;

[0042] It is understandable that DXGI can interact with the user-mode graphics driver in the driver, so that the graphics driver outputs the captured screen data to the buffer; wherein the buffer is used to store the captured screen data.

[0043] S104, the graphics card driver interacts with the kernel layer to determine whether the document is a read-only document;

[0044] Among them, the kernel layer is the kernel layer of the Windows system;

[0045] S105, if yes, the kernel layer can enable the graphics protection mechanism and encrypt or shield the screen data;

[0046] It is understandable that if the document is read-only, the kernel layer can enable the graphics protection mechanism and encrypt or shield the screen data. At this time, the document part of the screen data will become a black screen;

[0047] S106, DXGI returns the encrypted screen data;

[0048] It is understandable that DXGI can return the encrypted screen data (black screen data) to the buffer, so that the subsequent projection end will encode the encrypted screen data and output it to the projection display end.

[0049] Then, the operating system has a graphic protection mechanism for documents with read-only permissions, which can set the document part in the screen data captured by the interface to a black screen. When the projection end displays the document subsequently, the document display area will be displayed as a black screen, and the user will not be able to view the screen projected by the projection end normally.

[0050] Based on the problems described above, the embodiments of the present application provide a screen projection method, device, electronic device and computer program product.

[0051] The following is an introduction to the screen projection method provided in the embodiment of the present application.

[0052] Among them, the screen projection method can be applied to various scenarios where screen projection is used, such as local meeting scenarios (i.e., in a room such as an office or conference room, any person projects the screen to be projected on his or her computer to the projection display terminal in the room), remote video conferencing scenarios, remote teaching scenarios, etc., and the embodiments of this application do not make specific limitations on this. In addition, the screen projection method can be applied to the projection terminal of an electronic device, and the projection terminal can be a projection client in the electronic device. In this case, the projection terminal is in software form. Of course, the execution subject of the embodiments of this application can also be the main control module of the projection client. The embodiments of this application do not make specific limitations on this. The projection terminal can also be a projection device connected to the electronic device. In this case, the projection terminal is in hardware form. In addition, the electronic device is equipped with a display. The electronic device can specifically be a laptop computer, a desktop computer, a mobile phone, etc., and the embodiments of this application do not make specific limitations on this.

[0053] In addition, the projection end can communicate with the virtual driver in the electronic device, and the virtual driver can be a driver for driving the virtual display; exemplarily, when the projection end is in the form of a projection client, the installation package of the projection client can include an installation package of the virtual driver (the virtual driver can be a program module in the installation package of the projection client). When the user installs the projection client, the virtual driver can be automatically installed. Of course, the virtual driver can also be downloaded and installed independently; when the projection end is in the form of a projection device, after the projection device is connected to the projection device, the projection device actively triggers the virtual driver to be installed, or the user downloads the driver package of the virtual driver that matches the projection device for installation; and, the virtual driver can be independent of the projection end, or it can be part of the projection end, and the embodiments of this application do not make specific limitations. In addition, it should be noted that, exemplarily, the projection display end can be a hardware device compatible with the projection end, for example: a projection display for communicating with the projection end and displaying the projection content; exemplarily, the projection display end can also be software installed in a device with a display, which communicates with the projection end to obtain the projection data transmitted by the projection end and display it on the device with a display.

[0054] Furthermore, the operating system of the electronic device in the embodiment of the present application is not limited to the Windows system. Any system that supports a virtual display can be applied to the embodiment of the present application, and the embodiment of the present application does not make any specific limitations on this.

[0055] It should be noted that the terms "unit" and "module" in the embodiments of the present application may refer to hardware components, software programs, or software programs and the necessary carriers for the operation of the software programs. As technology evolves, the steps required to execute or the functions undertaken by these "units" or "modules" (for example, the steps described after a certain unit or module is "used for") may be performed using hardware, software, or a combination of hardware and software. As long as the hardware, software, or a combination of hardware and software can execute the steps required by the "unit", it can be considered a "unit" or "module" in the embodiments of the present application.

[0056] Among them, a screen projection method is applied to a screen projection end, wherein the screen projection end is a screen projection client in an electronic device or a screen projection device connected to the electronic device, and the screen projection end communicates with a virtual driver in the electronic device; the method includes:

[0057] In response to satisfying a screen projection trigger condition, a first control instruction is sent to the virtual driver, so that the virtual driver responds to the first control instruction and sequentially reads each frame of desktop image data from the target memory allocated for the virtual display; wherein the screen projection trigger condition includes receiving a screen projection instruction for a screen to be projected, and the screen to be projected is a screen in the target desktop and contains page content of a target document; the virtual display is created by the virtual driver, and after the virtual display is created, the operating system of the electronic device obtains the desktop image data of the target desktop and stores each frame of the obtained desktop image data in the target memory as virtual display data of the virtual display;

[0058] Acquiring each frame of desktop image data read by the virtual driver;

[0059] Based on the obtained desktop image data of each frame, the projection data of the screen to be projected is sent to the projection display end for display by the projection display end.

[0060] In the screen projection method provided by the embodiment of the present application, in response to the screen projection trigger condition being met, the screen projection end can send a first control instruction to the virtual driver, so that the virtual driver sequentially reads each frame of desktop image data from the target memory allocated for the virtual display; and the screen projection end can obtain each frame of desktop image data read by the virtual driver to realize the screen projection of the screen to be projected based on each frame of desktop image data. Compared with the related art, when projecting the screen to be projected, the present solution does not need to use the interface provided by the operating system to capture the screen data of the screen, so that the operating system will not trigger the graphics protection mechanism for documents with read-only permissions; specifically, the screen projection end of the present solution can use the virtual driver to obtain each frame of desktop image data stored by the operating system for the virtual display, and since each frame of desktop image data is virtual display data of the virtual display, each frame of desktop image data is normal display data. Therefore, when projecting based on each frame of desktop image data, the projected screen can be displayed normally, and the document display area will not be displayed as a black screen when the screen projection display end displays the screen to be projected. It can be seen that through this solution, the screen projected by the projection end can be displayed normally on the projection display end.

[0061] The following describes a screen projection method provided by an embodiment of the present application in conjunction with the accompanying drawings.

[0062] like Figure 2 As shown, an embodiment of the present application provides a screen projection method; the method includes:

[0063] S201, in response to a screen projection trigger condition being met, sending a first control instruction to a virtual driver, so that the virtual driver responds to the first control instruction and sequentially reads each frame of desktop image data from a target memory allocated to the virtual display;

[0064] Among them, the screen projection triggering conditions include receiving a screen projection instruction for the screen to be projected, and the screen to be projected is a screen in the target desktop and contains the page content of the target document; the virtual display is created by the virtual driver and after the virtual display is created, the operating system of the electronic device obtains the desktop image data of the target desktop and stores each frame of the obtained desktop image data in the target memory as virtual display data of the virtual display.

[0065] Exemplarily, the screen to be projected can be a screen in the target desktop that contains the complete page content of the target document. In this case, the complete page of the target document needs to be projected; of course, the screen to be projected can also be a screen in the target desktop that contains part of the page content of the target document. In this case, part of the page of the target document needs to be projected. This embodiment of the present application does not make any specific restrictions on this. Moreover, it is reasonable that the screen to be projected is the desktop itself of the target desktop or the target window area within the target desktop. Furthermore, the target document can be a read-only document or a non-read-only document. This embodiment of the present application does not make any specific restrictions on this. In addition, the first control instruction can be a control instruction that instructs the virtual driver to read each frame of desktop image data sequentially from the target memory allocated by the virtual display. The target memory can be the memory allocated by the operating system for display on the virtual display. It should be noted that for any virtual display created, if the virtual display is set to clone mode, the screen content that the virtual display needs to display is the same as the content displayed on the screen of the electronic device (i.e., the display of the electronic device); if the virtual display is set to extended mode, the screen content that the virtual display needs to display may be different from the content displayed on the screen of the electronic device. Furthermore, for the virtual display described in this embodiment, the virtual display may not display the screen, but of course, it may also display the screen, and this application does not limit this; and any virtual display can only be set to one mode: extended mode or clone mode, and cannot respond to multiple modes simultaneously. It can be understood that since the present application needs to rely on a virtual driver to obtain the desktop image data stored by the operating system for the virtual display when performing screen projection, the virtual display has been created before the virtual driver reads each frame of desktop image data in sequence from the target memory allocated for the virtual display; and after the virtual display is created, the operating system of the electronic device obtains the desktop image data of the target desktop and stores each frame of the obtained desktop image data in the target memory as virtual display data of the virtual display; in this way, the virtual driver can read each frame of desktop image data in sequence from the target memory allocated for the virtual display for use in the screen projection process for the screen to be projected.

[0066] Moreover, in one implementation, the target desktop can be the desktop displayed on the display of the electronic device, that is, the screen displayed on the desktop displayed on the display of the electronic device needs to be projected. At this time, the virtual display is: a display set to copy mode. Accordingly, the desktop image data stored in the target memory is the image data of the desktop displayed on the display of the electronic device. In another implementation, the target desktop is the desktop that the virtual display needs to display when the virtual display is set to extended mode. That is, the screen displayed on the desktop that the virtual display needs to display in extended mode needs to be projected. Accordingly, the desktop image data stored in the target memory is the image data of the desktop that the virtual display needs to display in extended mode.

[0067] Specifically, based on the copy mode, the operating system of the electronic device can take a screenshot of the current desktop (the desktop currently displayed on the display of the electronic device) and store each frame of desktop image data obtained by the screenshot in the target memory as virtual display data of the virtual display; it should be emphasized that in the copy mode, the operating system can continuously take screenshots of the current desktop, so as to obtain each frame of desktop image data. When storing the obtained desktop image data in the target memory, each frame of desktop image data can be stored in the target memory in sequence according to the screenshot order. Then, the subsequent virtual driver can respond to the first control instruction and read each frame of desktop image data from the target memory allocated for the virtual display in sequence. In addition, based on the extension mode, the projection end can send a first control instruction to the virtual driver, and the virtual driver can read each frame of desktop image data from the target memory. Each frame of desktop image data is the image data required to be displayed by the extended virtual display. The projection end obtains each frame of desktop image data read by the virtual driver, and based on each frame of desktop image data, sends the projection data of the screen to be projected to the projection display end, so that the projection display end performs an extended display to show different desktop contents. It should be noted that the process of obtaining desktop image data and storing it in the target memory as virtual display data in response to different modes of the virtual display executed by the operating system belongs to the response process of the virtual display according to the setting of the operating system.

[0068] In addition, it can be understood that, for the case where the target desktop is the desktop that the virtual display needs to display when the virtual display is set to extended mode, since the desktop picture that the virtual display in extended mode needs to display is the projected picture, then, before the screen is projected, the virtual display in extended mode has been created, and after it is created, the screen projection process can be triggered; thus, the virtual driver can read each frame of desktop image data directly from the target memory allocated for the virtual display in sequence in response to the first control instruction, without having to analyze whether the virtual display has been created.

[0069] As for the case where the target desktop is the desktop displayed on the display of the electronic device, since the screen to be projected is the screen of the desktop displayed on the display of the electronic device, the projection process can be triggered regardless of whether a virtual display in copy mode has been created in the electronic device; then, upon receiving the projection instruction for the target document, the projection end can send a first control instruction to the communicating virtual driver. If there is no virtual display in the electronic device, the virtual driver can create a virtual display, thereby triggering the operating system to take a screenshot of the target desktop, obtain desktop image data, and store the desktop image data in the target memory; if there is a virtual display in the electronic device, the virtual driver can read each frame of desktop image data in sequence from the target memory allocated for the virtual display, and the embodiments of the present application do not make specific limitations on this.

[0070] In one implementation, the target document is a document with readable permissions for personnel, and the receiving of the projection instruction for the screen to be projected specifically includes: receiving the projection instruction for the screen to be projected when each target person meets the readable conditions; each target person is a person who views the projection data.

[0071] It can be understood that the target document targeted by this implementation method is a document with readable permissions for personnel. At this time, it can be detected whether the person viewing the projection data has the readable conditions for the target document. When the people viewing the projection data all have the readable conditions for the target document, they can receive the projection instruction for the screen to be projected; of course, in one implementation method, the readable condition can be considered as having the permission to read the document, and the embodiment of the present application does not make specific limitations on this.

[0072] For the sake of clarity, the method for determining whether each target person meets the readability condition will be introduced in other embodiments and will not be elaborated on here.

[0073] S202, obtaining desktop image data of each frame read by the virtual driver;

[0074] It can be understood that after the virtual driver reads each frame of desktop image data from the target memory in sequence, the projection end can obtain each frame of desktop image data read by the virtual driver; then, the virtual driver can actively send each frame of desktop image data read to the projection end to enable the projection end to obtain each frame of desktop image data. The embodiment of the present application does not make specific limitations on this.

[0075] In one implementation, the virtual driver is further configured to store the read frame of desktop image data into the target buffer each time a frame of desktop image data is read;

[0076] The step of obtaining the desktop image data of each frame read by the virtual driver includes step A1:

[0077] Step A1: Read desktop image data from the target buffer to obtain each frame of desktop image data read by the virtual driver.

[0078] Among them, the target buffer can be a buffer for caching desktop image data. The target buffer can cache multiple frames of desktop image data at the same time, or it can only cache one frame of desktop image data (each time a frame of desktop image data is read by the projection end, the virtual driver writes another frame of desktop image data to the target buffer). The embodiment of the present application does not make specific limitations on this; and the target buffer can specifically be a swap chain (swapchain), which is not specifically limited in the embodiment of the present application.

[0079] It can be understood that in this implementation, every time the virtual driver reads a frame of desktop image data, it can store the read frame of desktop image data in the target buffer. Correspondingly, the projection end can read the desktop image data from the target buffer to smoothly obtain each frame of desktop image data read by the virtual driver.

[0080] It can be seen that every time the virtual driver reads a frame of desktop image data, it can store the read frame of desktop image data in the target buffer. The projection end can read the desktop image data from the target buffer to obtain each frame of desktop image data read by the virtual driver. Based on the target buffer, the projection end can obtain the desktop image data, which provides a basis for screen projection.

[0081] S203: Based on the obtained desktop image data of each frame, the projection data of the screen to be projected is sent to the projection display terminal for display by the projection display terminal;

[0082] It should be emphasized that if the screen to be projected is the target window area within the target desktop, then each frame of desktop image data can be subjected to image capture processing to obtain image data of the target window area within the target desktop as the data to be projected, as the projection data of the screen to be projected; if the screen to be projected is the desktop itself of the target desktop, then each frame of desktop image data can be used as the data to be projected. Among them, the method of performing image capture processing on each frame of desktop image data may include but is not limited to: based on the coordinate information of the target window area, locating the target window area in each frame of desktop image, and capturing the target window area.

[0083] It can be understood that the projection end can encode and encapsulate the projection data of the screen to be projected to obtain code stream data, and can send the code stream data to the projection display end so that the projection display end can decode the code stream data and display the projection data of the screen to be projected. The projection data of the screen to be projected displayed on the projection display end can also be considered as a display of the target document.

[0084] In the screen projection method provided by the embodiment of the present application, in response to the screen projection trigger condition being met, the screen projection end can send a first control instruction to the virtual driver, so that the virtual driver sequentially reads each frame of desktop image data from the target memory allocated for the virtual display; and the screen projection end can obtain each frame of desktop image data read by the virtual driver to realize the screen projection of the screen to be projected based on each frame of desktop image data. Compared with the related art, when projecting the screen to be projected, the present solution does not need to use the interface provided by the operating system to capture the screen data of the screen, so that the operating system will not trigger the graphics protection mechanism for documents with read-only permissions; specifically, the screen projection end of the present solution can use the virtual driver to obtain each frame of desktop image data stored by the operating system for the virtual display, and since each frame of desktop image data is virtual display data of the virtual display, each frame of desktop image data is normal display data. Therefore, when projecting based on each frame of desktop image data, the projected screen can be displayed normally, and the document display area will not be displayed as a black screen when the screen projection display end displays the screen to be projected. It can be seen that through this solution, the screen projected by the projection end can be displayed normally on the projection display end.

[0085] Optionally, in another embodiment, the virtual display is a display set to a copy mode; before sending the first control instruction to the virtual driver, the method further includes steps B1-B3:

[0086] Step B1, detecting whether a virtual display currently exists in the electronic device;

[0087] Step B2: If it is detected that no virtual display currently exists in the electronic device, sending a second control instruction to the virtual driver, so that the virtual driver creates a virtual display in response to the second control instruction, and instructs the operating system to set the virtual display to a copy mode to complete the creation; and, in response to result information fed back by the virtual driver indicating that the creation is complete, executing the step of sending the first control instruction to the virtual driver;

[0088] Step B3: If it is detected that the virtual display currently exists in the electronic device, a third control instruction is sent to the virtual driver for detecting whether the virtual display is enabled; if the detection result feedback from the virtual driver indicates that it is in an enabled state, the step of sending the first control instruction to the virtual driver is triggered; if the detection result feedback from the virtual driver indicates that it is in a disabled state, the virtual display is set to an enabled state through the virtual driver, and the step of sending the first control instruction to the virtual driver is triggered.

[0089] As described in the above embodiment, for the case where the target desktop is the desktop displayed on the display of the electronic device, the screen projection process can be triggered regardless of whether a virtual display in copy mode has been created in the electronic device, and when a virtual display in copy mode is created, the virtual display can be in an enabled state or a disabled state. Taking this into account, in this embodiment, steps B1-B3 can be executed to analyze whether the virtual display has been created and whether it is in an enabled state, thereby performing different operations based on different analysis results.

[0090] It is understandable that, for step B1, before sending the first control instruction to the virtual driver, the screen projection end can also detect whether there is a virtual display in the electronic device. If it is detected that the virtual display does not currently exist in the electronic device, step B2 can be executed. If it is detected that there is a virtual display in the electronic device, step B3 can be executed. It should be emphasized that the virtual display is not destroyed immediately after the screen projection ends. In order to save resources, it can also be retained for a predetermined period of time before being destroyed. Therefore, the existence of a virtual display in the electronic device is also detected.

[0091] It can be understood that, for step B2, step B2 is for the case where there is no virtual display in the electronic device. Then, before sending the first control instruction to the virtual driver, the projection end can send a second control instruction to the virtual driver, so that the virtual driver responds to the second control instruction and creates a virtual display. After creating the virtual display, the virtual display can be set to copy mode with the help of the operating system. The copy mode has been introduced in the above embodiment and will not be described in detail here. For example, the projection end of the PC device can send a second control instruction to the virtual driver loaded on the PC device, so that the virtual driver responds to the second control instruction and creates a virtual display, and instructs the Windows system to set the virtual display to copy mode to complete the creation. Among them, the second control instruction can be an instruction to instruct the creation of a virtual display; in addition, after the creation is completed, the virtual display can have the allocated target memory. When the screen projection process for the target document is ended, the screen projection end can send a control instruction representing the destruction of the virtual display to the virtual driver, so that the virtual driver destroys the virtual display, and, the next time the screen projection end responds to the satisfaction of the screen projection trigger condition (the next time the screen projection instruction for the document is received), it can re-send the second control instruction to the virtual driver to create a virtual display. Of course, the virtual display may not be destroyed. This embodiment of the present application does not specifically limit this. In addition, after the creation is completed, the virtual driver can feedback the result information used to represent the completion of the creation to the screen projection end. The screen projection end can trigger the step of sending the first control instruction to the virtual driver, thereby providing an implementation basis for screen projection.

[0092] In addition, if the virtual display is set to extended mode, there is no need to execute step B1. The virtual display exists by default in extended mode, and the projection end can directly send a first control instruction to the virtual driver to enable the projection process in extended mode.

[0093] It is understood that, for step B2, step B2 can be for the case where the virtual display is not destroyed, and a state can be set for the virtual display. The state of the virtual display can specifically be an enabled state or a disabled state. In the enabled state, the virtual display can work normally, and in the disabled state, the virtual display cannot work normally. Then, when the presence of a virtual display in the electronic device is detected, a third control instruction for detecting whether the virtual display is enabled can be sent to the virtual driver to determine the state of the virtual display. In addition, the virtual driver can feedback the detection result to the projection end in response to the third control instruction. If the detection result indicates that the virtual display is in the enabled state, indicating that the virtual display can currently work normally, then the step of sending the first control instruction to the virtual driver can be triggered, so that the virtual driver responds to the first control instruction and sequentially reads each frame of desktop image data from the target memory allocated for the virtual display, providing a basis for screen projection. In addition, if the detection result indicates that the virtual display is in the disabled state, indicating that the virtual display cannot currently work normally, at this time, the virtual driver can send a state change instruction to the operating system so that the operating system sets the state of the virtual display to the enabled state and triggers the step of sending the first control instruction to the virtual driver.

[0094] It can be seen that the screen projection end in the embodiment of the present application can detect whether the virtual display currently exists in the electronic device before sending the first control instruction to the virtual driver. If it is detected that the virtual display currently does not exist in the electronic device, a second control instruction is sent to the virtual driver to enable the virtual driver to create a virtual display and instruct the operating system to set the virtual display to copy mode to complete the creation. Then, when there is no virtual display in the electronic device, a virtual display can be created, and the virtual display can be set to copy mode, thereby ensuring that there is a virtual display set to copy mode in the electronic device, providing an implementation basis for screen projection, thereby realizing the normal display of the screen projected by the screen projection end in the screen projection display end. In addition, the step of sending the second control instruction to the virtual driver is triggered only when there is no virtual display currently in the electronic device, and there will be no repeated creation of the virtual display, saving resources. In addition, if it is detected that there is a virtual display in the electronic device, a third control instruction is sent to the virtual driver to detect whether the virtual display is enabled. If the detection result feedback from the virtual driver indicates that it is in an enabled state, the step of sending the first control instruction to the virtual driver can be directly triggered; if the detection result feedback from the virtual driver indicates that it is in a disabled state, the virtual display can be set to an enabled state through the virtual driver, and the step of sending the first control instruction to the virtual driver can be triggered; then, it can be ensured that the virtual display is in an enabled state (a state where it can work normally) before the first control instruction is sent to the virtual driver, thereby ensuring the stability of the screen projection, and preventing the virtual display from being in a disabled state for projection, thereby achieving the normal display of the screen projected by the projection end in the projection display end. In addition, when the virtual display is set to a display in copy mode, the screen displayed on the electronic device can be projected to the projection display end with the help of the virtual display, providing a basis for implementing the screen projection.

[0095] In another embodiment, the detecting whether the virtual display currently exists in the electronic device includes step B11:

[0096] Step B11: Detecting whether the virtual display currently exists in the electronic device based on the display creation information recorded in the configuration file of the virtual driver.

[0097] It can be understood that the display creation information is recorded in the configuration file of the virtual driver. By querying the display creation information recorded in the configuration file, it can be detected whether a virtual display currently exists in the electronic device; for example, if the display creation file records that a virtual display has been created, it can be considered that the existence of a virtual display in the electronic device is detected.

[0098] It can be seen that the embodiment of the present application can accurately detect whether there is a virtual display in the electronic device based on the display creation information recorded in the configuration file of the virtual driver, and provide an implementation basis for the projection of the screen to be projected, so as to realize the normal display of the screen projected by the projection end in the projection display end.

[0099] Optionally, in another embodiment, sending the first control instruction to the virtual driver includes step C1:

[0100] Step C1: Sending a first control instruction carrying a target handle to the virtual driver; wherein the target handle is a handle assigned to the virtual display by the operating system and obtained through a designated interface of the operating system based on a display identifier of the virtual display read from a configuration file of the virtual driver; the handle assigned to the virtual display is address information of a memory address used to cache interaction data when interacting with the operating system regarding the virtual display;

[0101] Accordingly, the virtual driver responds to the first control instruction and sequentially reads each frame of desktop image data from the target memory allocated for the virtual display, including steps D1-D2:

[0102] Step D1, the virtual driver responds to the first control instruction and obtains address information of the target memory from the operating system based on the target handle;

[0103] Step D2, based on the acquired address information of the target memory, read each frame of desktop image data from the target memory in sequence.

[0104] It can be understood that, for step C1, the first control instruction sent to the virtual driver can carry a target handle, and the acquisition of the target handle can be: the projection end reads the display identifier of the virtual display from the configuration file of the virtual driver, and sends the display identifier to the designated interface of the operating system to obtain the handle assigned by the operating system to the virtual display as the target handle, wherein the designated interface can be the CreateFile interface (a multi-functional function interface), and the target handle can be called hDevice. The embodiment of the present application does not specifically limit this. In addition, the handle assigned by the operating system to the virtual display can also be considered as the address information of the memory address used to cache the interactive data when interacting with the operating system about the virtual display. The virtual driver can identify the handle assigned by the virtual display, thereby obtaining the address information, that is, obtaining the address information of the memory address used to cache the interactive data. In addition, in this embodiment, the first control instruction can be a custom I / O signaling, and the I / O signaling can carry the target handle.

[0105] Also, when sending the first control instruction carrying the target handle to the virtual driver, the DeviceIoControl interface can be used to send it to the virtual driver. The DeviceIoControl interface is an API (Application Programming Interface) used to directly send control instructions to the specified device driver. The embodiment of the present application does not make specific limitations on this.

[0106] It can be understood that, for step D1 and step D2, the virtual driver can obtain the address information of the target memory from the operating system based on the target handle in response to the first control instruction; and based on the obtained address information of the target content, read each frame of desktop image data from the target memory in sequence; wherein the first control instruction may include the IOCTL_GET_VIRTUAL_DISPLAY_IMAGE instruction, indicating that the IOCTL (input / output control) function can obtain the image data displayed by the virtual display, that is, the IOCTL_GET_VIRTUAL_DISPLAY_IMAGE instruction can instruct the IOCTL function to obtain the image data displayed by the virtual display; then, based on the address information of the target memory and the IOCTL_GET_VIRTUAL_DISPLAY_IMAGE instruction, the virtual driver can read each frame of desktop image data in sequence from the target memory allocated for the virtual display. Among them, based on the target handle, obtaining the address information of the target memory from the operating system may include: transmitting a specified instruction for requesting the address information of the target memory to the address information indicated by the target handle, so that the operating system can read the specified instruction in the memory address having the address information indicated by the target handle, and in response to the specified instruction, write the address information of the target memory to the memory address having the address information indicated by the target handle, so that the virtual driver can read the address information of the target memory in the memory address having the address information indicated by the target handle.

[0107] It can be seen that the target handle can indicate the address information of the memory address used to cache the interactive data when interacting with the operating system about the virtual display. When the first control instruction carrying the target handle is sent to the virtual driver, the virtual driver can obtain the address information of the target memory and read each frame of desktop image data from the target memory, thereby providing a basis for realizing the normal display of the screen projected by the projection end in the projection display end.

[0108] Optionally, in another embodiment, the method for determining whether each target person meets the readability condition includes method E1 or E2:

[0109] Mode E1: In response to the target document being a document with a readable permission set for a person, outputting a first prompt message, wherein the first prompt message is used to prompt the presence of page content of the document with a readable permission set in the screen to be projected and whether to continue projecting; upon receiving an instruction from the user indicating that the screen can continue projecting based on the first prompt message, determining whether each target person meets the readable condition;

[0110] Mode E2: In response to the target document being a document with a person-readable permission, obtaining the personal information of each target person, and determining whether each target person meets the readability condition based on the personal information of each target person.

[0111] It can be understood that, for method E1, in response to the situation where the target document is a document with readable permission for a person, a first prompt message can be output to prompt that there is page content of the document with readable permission in the screen to be projected and whether to continue projecting. The user can feedback instructions to the projection end based on the first prompt message; then, if an instruction representing the continuation of screen projection is received, it can be determined that each target person meets the readable condition; and, receiving the instruction given by the user based on the first prompt message, which represents the continuation of screen projection, can also be considered as receiving an instruction to project the screen to be projected, that is, the user has indicated that the screen can be projected, thereby avoiding the need to receive other instructions to project the screen to be projected after receiving the instruction to continue projecting, and avoiding the situation where the instruction indicating that the screen can be projected is received multiple times. The embodiment of the present application does not make specific restrictions on this. In one implementation method, if an instruction representing the continuation of screen projection is not received, the screen projection is stopped.

[0112] It is understandable that, for method E2, in response to the target document being a document with personnel readable permissions, the personnel information of each target person is obtained, and based on the personnel information of each target person, it is determined whether each target person meets the readable condition; in method E2, there is no need for user instructions, but rather a determination is made as to whether each target person meets the readable condition. Among them, the source of the personnel information of each target person may be related to the scene. For example, in a video conferencing scene, the target person may also be referred to as a participant, and the personnel information of each participant may be collected by a camera of an electronic device, which is not specifically limited in this embodiment of the present application. In addition, the personnel information of each target person includes at least the identity information of each target person, which is not specifically limited in this embodiment of the present application.

[0113] It can be seen that in response to the target document being a document with personnel readable permissions, a first prompt message can be output. Only when the user receives an instruction based on the first prompt message to continue screen projection, it can be determined whether each target person meets the readable condition, thereby ensuring the security of screen projection; and the personnel information of each target person can also be obtained, and based on the personnel information of each target person, it can be determined whether each target person meets the readable condition, thereby ensuring the security of screen projection.

[0114] In one implementation, determining whether each target person satisfies a readability condition based on the personnel information of each target person includes: step F1, and / or step F2;

[0115] Step F1: Based on the personnel information of each target person, query whether each target person has read permission for the target document, and obtain a query result; in response to the query result indicating that each target person has read permission for the target document, determine that each target person meets the read condition; and / or,

[0116] Step F2: in response to the query result indicating that at least one target person does not have read permission for the target document, outputting second prompt information indicating that a person without read permission is viewing the target document;

[0117] Accordingly, the method further comprises step G1:

[0118] Step G1, upon receiving the user's instruction to continue screen projection based on the second prompt information, determining whether each target person meets the readability condition; wherein, the second prompt information is also used to prompt whether to continue screen projection.

[0119] It is understood that for step F1, a query can be made from a predetermined permission server to determine whether each target person has read permission for the target document. The permission server is used to record the permission information of each document, and the permission information of any document includes the personnel information of the personnel corresponding to the various permissions of the document. The predetermined permission server can be an RMS (Rights Management Services) server, or of course, it can also be other servers, or documents that store permission information. This embodiment of the present application does not specifically limit this. In addition, the permission information of the target document can include the personnel information of the personnel corresponding to the read permission of the target document. Then, by querying the personnel information of the personnel corresponding to the read permission of the target document, it can be determined whether each target person has read permission for the target document to obtain a query result. Exemplarily, the target persons are employee 1, employee 2, and employee 3, and the permission server can contain the personnel information of the personnel corresponding to the read permission of the target document. From the permission server, it can be queried whether employee 1, employee 2, and employee 3 have read permission for the target document.

[0120] It is understood that when the query result indicates that each target person has read permission for the target document, it can be determined that each target person meets the read condition for the target document. For example, the target persons are Employee 1, Employee 2, and Employee 3, and the query result indicates that Employee 1, Employee 2, and Employee 3 all have read permission for Document 1. Therefore, it can be determined that Employee 1, Employee 2, and Employee 3 meet the read condition for Document 1.

[0121] It can be understood that for step F2, when the query result indicates that there is any target person who does not have the read permission for the target document, it can be determined that each target person does not meet the reading conditions for the target document. At this time, a second prompt information can be output to indicate that there is a person who does not have the read permission to view the document. The second prompt information can be displayed on the display of the electronic device, and the user can give instructions based on the second prompt information.

[0122] It is understandable that, for step G1, when receiving the instruction to continue screen projection issued by the user based on the second prompt information, it can be confirmed that each target person meets the readability condition; of course, in one implementation method, the screen projection end may also receive the instruction to not continue screen projection issued by the user based on the second prompt information. At this time, the prompt information of not continuing screen projection can be output to the screen projection display end. The embodiment of the present application does not make specific restrictions on this. In addition, receiving the instruction given by the user based on the second prompt information to continue screen projection can also be regarded as receiving the screen projection instruction for the screen to be projected, that is, the user has indicated that the screen projection can be performed, thereby avoiding the need to receive other screen projection instructions for the screen to be projected again after receiving the instruction to continue screen projection, and avoiding the situation where the instruction indicating that the screen can be projected is received multiple times.

[0123] It can be seen that the embodiment of the present application can query whether each target person has the read permission for the target document based on the personnel information of each target person to obtain the query result. If each target person has the read permission for the target document, it is determined that each target person meets the reading conditions for the target document. Otherwise, a second prompt information is output to indicate that there are people who do not have the read permission to view the target document. When the target document is a read-only document, it can be ensured that each target person has the read permission for the target document, or the screen can be projected only when the user receives an instruction to continue screen projection based on the second prompt information, thereby ensuring the security of the screen projection.

[0124] Optionally, in another embodiment, the obtaining of the personnel information of each target person includes steps H1-H2:

[0125] Step H1, sending a request for personnel information about persons viewing the projection data to the projection display terminal, so that the projection display terminal determines the personnel information of each target person according to a predetermined determination method, and feeds back the determined personnel information of each target person to the projection terminal;

[0126] Step H2, receiving the personnel information of each target person fed back by the projection display terminal;

[0127] The predetermined determination method includes:

[0128] Executing at least one personnel analysis method to obtain an analysis result of the at least one personnel analysis method;

[0129] In the case that only one personnel analysis method is executed, the personnel information indicated by the analysis result of the personnel analysis method is used as the personnel information of each target person; or

[0130] In the case where multiple personnel analysis methods are executed, the personnel information indicated by the analysis results of the multiple personnel analysis methods is combined to obtain the personnel information of each target person;

[0131] The at least one personnel analysis method includes one or more of the following analysis methods:

[0132] Calling a predetermined camera to perform image acquisition to obtain a target image, and performing personnel analysis on the target image to obtain personnel information; wherein the predetermined camera is a camera having a field of view that includes the attendees;

[0133] Based on the identifier of the conference room where the projection display terminal is located, a request for information about the persons participating in the conference in the conference room at the current time is sent to the conference management system to obtain the personnel information of the persons participating in the conference at the current time;

[0134] In response to the projection display terminal currently being connected to the video conferencing client, requesting personnel information of persons participating in the conference from the video conferencing client;

[0135] It is understandable that, for step H1, a request for personnel information of the persons viewing the projection data can be sent to the projection display end. The projection display end can determine a predetermined determination method according to the current scene, and determine the personnel information of each target person according to the predetermined determination method, and feed back the determined personnel information of each target person to the projection end. It should be emphasized that the predetermined determination method can include executing at least one personnel analysis method and obtaining the analysis result of at least one personnel analysis method. In the case where the personnel analysis method executed is one, the personnel information indicated by the analysis result of the personnel analysis method can be directly used as the personnel information of each target person. For example, by calling a camera to perform image acquisition, obtaining a target image, and performing personnel analysis on the target image, the information of employee 1 and the information of employee 2 are obtained, and used as the personnel information of each target person; or, in the case where multiple personnel analysis methods are executed, the personnel information indicated by the analysis results of the multiple personnel analysis methods can be combined to obtain the personnel information of each target person. For example, if personnel analysis method a indicates the personnel information of employees 1 and 2, and personnel analysis method b indicates the personnel information of employees 1, 2, and 3, then the personnel information of each target person is the personnel information of employees 1, 2, and 3. Furthermore, at least one personnel analysis method may include one or more analysis methods, and this is not specifically limited in the present embodiment.

[0136] Exemplarily, the personnel analysis method may include: calling a predetermined camera to perform image acquisition to obtain a target image, where the called camera is a camera whose field of view includes the attendees; and performing personnel analysis on the target image to obtain personnel information; and any scene with a predetermined camera (a camera whose field of view includes the attendees) can be applied to this analysis method, and the embodiments of the present application do not specifically limit this. Exemplarily, in a conference scene, a camera is provided that can capture the facial features of each attendee in the entire conference room 1. Then, the projection display end can call this camera to perform image acquisition to obtain the target image, and perform personnel analysis on the target image to obtain personnel information.

[0137] Exemplarily, the personnel analysis method may include: for a scenario where there is a conference management system, the conference management system may store information about the participants in each conference room, and based on the identifier of the conference room where the projection display terminal is located, an information request about the participants in the conference room at the current time may be sent to the conference management, thereby obtaining the personnel information of the participants in the conference at the current time. In addition, any scenario with a conference management system may be applicable to the predetermined determination method, and the embodiments of the present application do not make specific limitations on this. Exemplarily, in a conference scenario, the projection display terminal may send an information request about the participants in the conference room at 10 o'clock to the conference management system based on the identifier of conference room 1, and obtain the personnel information of staff 1 and staff 2 who participated in the meeting in conference room 1 at 10 o'clock.

[0138] Exemplarily, the personnel analysis method may include: for a scenario where the projection display terminal is connected to a video conferencing client (which can be considered a video conferencing scenario), the projection display terminal can directly request the personnel information of the persons participating in the meeting from the video conferencing client, thereby determining the personnel information of each target person; wherein, the video conferencing client can contain the personnel information of all persons participating in the meeting; exemplarily, in a video conferencing scenario, the projection display terminal can request the personnel information of staff 1 and staff 2 from the connected video conferencing client.

[0139] It can be understood that, for step H2, the projection end can receive the personnel information of each target person returned by the projection display end.

[0140] In addition, in one implementation method, the projection end can also directly access the above-mentioned conference management system, or directly call the camera, or directly access the above-mentioned video conferencing client to obtain the personnel information of each target person. The embodiment of this application does not make specific limitations on this.

[0141] It can be seen that the embodiment of the present application can determine the personnel information of each target person according to a predetermined determination method, and feed back the determined personnel information of each target person to the projection end, thereby receiving the personnel information of each target person fed back by the projection display end, providing an implementation basis for projection, thereby realizing the normal display of the picture projected by the projection end in the projection display end.

[0142] Optionally, in another embodiment, the present application also provides another screen projection method, as shown in FIG3 (a), FIG3 (b) and FIG3 (c):

[0143] Figure 3 (a) is a flowchart of another screen projection method, in which a document can be projected:

[0144] S301, click on the screen projection option;

[0145] It is understandable that users can click the screen projection option on the projection end of the PC device to initiate screen projection.

[0146] S302, determining the personal information of the participants and obtaining from the RMS server whether the participants have read permission;

[0147] Participants are members who participate in the meeting. It is understood that the personnel information of the participants can be determined, and whether the participants have read permissions can be obtained from the RMS server; and if not, step S303 can be executed;

[0148] S303, determining whether to project a read-only document;

[0149] It is understandable that when the participant does not have the read permission, it can be determined whether to project the read-only document. The projection terminal can output a first prompt message, which is used to prompt whether the page content of the document with read permission exists in the screen to be projected and whether to continue projecting; and if so, step S304 can be executed;

[0150] S304, determining the custom I / O signaling;

[0151] It is understandable that, upon receiving the instruction given by the user based on the first prompt information indicating the continuation of screen projection, the projection end may determine the custom I / O signaling, which is similar to the step of determining the first control instruction described above.

[0152] S305, communicating with the virtual driver;

[0153] The virtual driver may also be called a virtual graphics driver, and a custom I / O signaling may be sent to the virtual driver so that the virtual driver sequentially reads each frame of desktop image data from a target memory allocated for the virtual display;

[0154] S306, saving each frame of desktop image data into a buffer;

[0155] It can be understood that the virtual driver saves each frame of desktop image data into a buffer based on the operating system;

[0156] S307, reading the desktop image data in the buffer;

[0157] It is understandable that the projection end can read the desktop image data from the buffer;

[0158] S308, sending the document projection data to the projection display terminal;

[0159] It can be understood that the obtained desktop image data is used as the projection data of the document and sent to the projection display terminal for display by the projection display terminal.

[0160] FIG3 (b) is an application of a virtual display provided by an embodiment of the present application:

[0161] The PC device 310 includes a physical display 320 (i.e., the display of the above-mentioned electronic device) and a virtual display 330, wherein the PC device 310 is also provided with a screen projection terminal, and the virtual display 330 is set to an extended mode. In the extended mode, the picture to be displayed on the virtual display 330 is the picture to be projected; and, before the screen projection is performed, the virtual display 330 in the extended mode has been created, and after it is created, the screen projection process can be triggered; and, the screen projection terminal can project the picture displayed by the virtual display 330 to the screen projection display terminal 340 to realize the picture display, and the physical display 320 is placed with some information that the user wants to see.

[0162] FIG3 (c) shows another application of a virtual display provided by an embodiment of the present application:

[0163] The PC device 310 includes a physical display 320 and a virtual display 330. A screen projection terminal is also provided in the PC device 310. The virtual display 330 is set to a copy mode. In the copy mode, the screen to be projected can be the screen displayed by the physical display 320; and regardless of whether a virtual display 330 in copy mode has been created in the PC device 310, the screen projection process can be triggered; in copy mode, the operating system of the PC device 310 can take a screenshot of the desktop of the physical display 320, and store each frame of desktop image data obtained by the screenshot in the target memory as display data of the virtual display 330, and then the screen projection terminal can project the screen displayed by the virtual display 330 to the screen projection display terminal 340 to realize screen display.

[0164] It can be seen that in the screen projection method provided by the embodiment of the present application, the screen projection end can send a first control instruction to the virtual driver, so that the virtual driver can read each frame of desktop image data from the target memory allocated for the virtual display in sequence; and the screen projection end can obtain each frame of desktop image data read by the virtual driver to realize the projection of the screen to be projected based on each frame of desktop image data. Compared with the related art, when projecting the screen to be projected, the present solution does not need to use the interface provided by the operating system to capture the screen data of the screen, so that the operating system will not trigger the graphic protection mechanism for documents with read-only permissions; specifically, the screen projection end of the present solution can use the virtual driver to obtain each frame of desktop image data stored by the operating system for the virtual display, and since each frame of desktop image data is the virtual display data of the virtual display, then each frame of desktop image data is the data for normal display, so that the screen projection based on each frame of desktop image data can make the screen projection display normally, and the document display area will not be displayed as a black screen when the screen projection display end displays the screen to be projected. It can be seen that through this solution, the screen projected by the projection end can be displayed normally on the projection display end.

[0165] Based on the above method embodiment, the embodiment of the present application further provides a screen projection device, which is applied to a screen projection end, wherein the screen projection end is a screen projection client in an electronic device or a screen projection device connected to the electronic device, and the screen projection end communicates with a virtual driver in the electronic device; the device includes, for example Figure 4 As shown:

[0166] The first sending module 410 is configured to send a first control instruction to the virtual driver in response to a screen projection trigger condition being met, so that the virtual driver reads each frame of desktop image data from the target memory allocated for the virtual display in sequence in response to the first control instruction; wherein the screen projection trigger condition includes receiving a screen projection instruction for a screen to be projected, the screen to be projected being a screen in the target desktop that contains page content of a target document; the virtual display is created by the virtual driver, and after the virtual display is created, the operating system of the electronic device obtains the desktop image data of the target desktop and stores each frame of the obtained desktop image data in the target memory as virtual display data of the virtual display;

[0167] An acquisition module 420 is configured to acquire each frame of desktop image data read by the virtual driver;

[0168] The second sending module 430 is used to send the projection data of the screen to be projected to the projection display end based on the obtained desktop image data of each frame for display by the projection display end.

[0169] Optionally, the virtual display is a display set to a copy mode; the apparatus further comprises:

[0170] a detection module, configured to detect whether the virtual display currently exists in the electronic device before sending the first control instruction to the virtual driver;

[0171] a third sending module, configured to, if it is detected that the virtual display does not currently exist in the electronic device, send a second control instruction to the virtual driver, so that the virtual driver creates the virtual display in response to the second control instruction, and instructs the operating system to set the virtual display to a copy mode to complete the creation; and, in response to result information fed back by the virtual driver indicating that the creation is complete, execute the step of sending the first control instruction to the virtual driver;

[0172] The fourth sending module is configured to, if it is detected that the virtual display currently exists in the electronic device, send a third control instruction to the virtual driver for detecting whether the virtual display is enabled; if the detection result feedback from the virtual driver indicates that the virtual display is in an enabled state, trigger the step of sending the first control instruction to the virtual driver; if the detection result feedback from the virtual driver indicates that the virtual display is in a disabled state, set the virtual display to an enabled state through the virtual driver, and trigger the step of sending the first control instruction to the virtual driver.

[0173] Optionally, the screen to be projected is the desktop of the target desktop itself or the target window area within the target desktop; and / or, the target desktop is the desktop displayed on the display of the electronic device, or, the target desktop is the desktop that the virtual display needs to display when the virtual display is set to extended mode;

[0174] The first sending module is specifically configured to:

[0175] Sending a first control instruction carrying a target handle to the virtual driver; wherein the target handle is a handle allocated to the virtual display by the operating system and obtained through a designated interface of the operating system based on a display identifier of the virtual display read from a configuration file of the virtual driver; the handle allocated by the virtual display is address information of a memory address used to cache interaction data when interacting with the operating system regarding information about the virtual display;

[0176] The virtual driver reads, in response to the first control instruction, frames of desktop image data from the target memory allocated for the virtual display in sequence, including:

[0177] The virtual driver obtains the address information of the target memory from the operating system based on the target handle in response to the first control instruction;

[0178] Based on the acquired address information of the target memory, each frame of desktop image data is read sequentially from the target memory.

[0179] Optionally, the target document is a document with human readable permission, and the receiving of the screen projection instruction for the screen to be projected specifically includes:

[0180] When each target person meets the readability conditions, a screen projection instruction for the screen to be projected is received; each target person is a person who views the projected screen data.

[0181] Optionally, the method for determining whether each target person meets the readability condition includes:

[0182] In response to the target document being a document with a readable permission for a person, outputting a first prompt message, wherein the first prompt message is used to prompt whether the page content of the document with a readable permission exists in the screen to be projected and whether to continue projecting; upon receiving an instruction from the user indicating that the screen projection is to be continued based on the first prompt message, determining whether each target person meets the readable condition;

[0183] or,

[0184] In response to the target document being a document with a person-readable permission, personal information of each target person is acquired, and based on the personal information of each target person, it is determined whether each target person meets a readability condition.

[0185] Optionally, based on the personnel information of each target person, determining whether each target person meets the readability condition includes:

[0186] Based on the personnel information of each target person, query whether each target person has the read permission for the target document to obtain a query result; in response to the query result indicating that each target person has the read permission for the target document, determine that each target person meets the read condition; and / or,

[0187] In response to the query result indicating that at least one target person does not have read permission for the target document, outputting second prompt information indicating that a person without read permission is viewing the target document;

[0188] The method further comprises:

[0189] When receiving the user's instruction to continue screen projection based on the second prompt information, it is determined that each target person meets the readability condition; wherein, the second prompt information is also used to prompt whether to continue screen projection.

[0190] Optionally, obtaining the personnel information of each target person includes:

[0191] Sending a request for personnel information about persons viewing the projected screen data to the projection display terminal, so that the projection display terminal determines the personnel information of each target person according to a predetermined determination method, and feeds back the determined personnel information of each target person to the projection terminal;

[0192] Receive the personnel information of each target person fed back by the projection display terminal;

[0193] The predetermined determination method includes:

[0194] Executing at least one personnel analysis method to obtain an analysis result of the at least one personnel analysis method;

[0195] In the case that only one personnel analysis method is executed, the personnel information indicated by the analysis result of the personnel analysis method is used as the personnel information of each target person; or

[0196] In the case where multiple personnel analysis methods are executed, the personnel information indicated by the analysis results of the multiple personnel analysis methods is combined to obtain the personnel information of each target person;

[0197] The at least one personnel analysis method includes one or more of the following analysis methods:

[0198] Calling a predetermined camera to perform image acquisition to obtain a target image, and performing personnel analysis on the target image to obtain personnel information; wherein the predetermined camera is a camera having a field of view that includes the attendees;

[0199] Based on the identifier of the conference room where the projection display terminal is located, a request for information about the persons participating in the conference in the conference room at the current time is sent to the conference management system to obtain the personnel information of the persons participating in the conference at the current time;

[0200] In response to the projection display terminal currently being connected to the video conferencing client, personnel information of persons participating in the conference is requested from the video conferencing client.

[0201] In the technical solution of this application, the operations involved in obtaining, storing, using, processing, transmitting, providing and disclosing user personal information are all carried out with the user's authorization.

[0202] The present application also provides an electronic device, such as Figure 5 Shown, including:

[0203] Memory 501, used for storing computer programs;

[0204] The processor 502 is used to implement any of the above-mentioned screen projection methods when executing the program stored in the memory 501.

[0205] Furthermore, the electronic device may further include a communication bus and / or a communication interface, and the processor 502, the communication interface, and the memory 501 communicate with each other via the communication bus.

[0206] The communication bus mentioned in the electronic devices mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into address buses, data buses, control buses, etc. For ease of illustration, only a single thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0207] The communication interface is used for communication between the above electronic device and other devices.

[0208] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0209] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0210] When the screen projection end is a screen projector, the electronic device including a processor for executing any of the above screen projection methods is the screen projector. The screen projector is connected to another electronic device and communicates with a virtual driver in the other electronic device.

[0211] In another embodiment provided in the present application, a computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned screen projection methods is implemented.

[0212] In another embodiment provided by the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any screen projection method in the above embodiments.

[0213] In the above embodiments, all or part of the embodiments can be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, tape), optical media (e.g., DVD), or solid-state drive (SSD).

[0214] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0215] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For related parts, refer to the description of the method embodiment.

[0216] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A screen projection method, characterized in that: Applied to a screen projection end, the screen projection end is a screen projection client in an electronic device or a screen projection device connected to the electronic device, and the screen projection end communicates with a virtual driver in the electronic device; The method comprises: In response to satisfying the screen projection trigger condition, a first control instruction is sent to the virtual driver, so that the virtual driver responds to the first control instruction and sequentially reads each frame of desktop image data from the target memory allocated for the virtual display; wherein the screen projection trigger condition includes receiving a screen projection instruction for the screen to be projected, and the screen to be projected is a screen in the target desktop and contains the page content of the target document; the virtual display is created by the virtual driver and after the virtual display is created, the operating system of the electronic device obtains the desktop image data of the target desktop and stores each frame of the obtained desktop image data in the target memory as the virtual virtual display data of a virtual display; the sending of the first control instruction to the virtual driver comprises: sending the first control instruction carrying a target handle to the virtual driver; wherein the target handle is a handle allocated to the virtual display by the operating system and obtained through a designated interface of the operating system based on the display identifier of the virtual display read from the configuration file of the virtual driver; the handle allocated by the virtual display is address information of a memory address used to cache interaction data when interacting with the operating system about information about the virtual display; the target memory is a memory corresponding to the address information obtained based on the target handle; Acquiring each frame of desktop image data read by the virtual driver; Based on the obtained desktop image data of each frame, the projection data of the screen to be projected is sent to the projection display end for display by the projection display end.

2. The method according to claim 1, characterized in that The virtual display is a display set to a copy mode; before sending the first control instruction to the virtual driver, the method further includes: Detecting whether the virtual display currently exists in the electronic device; If it is detected that the virtual display does not currently exist in the electronic device, sending a second control instruction to the virtual driver, so that the virtual driver creates the virtual display in response to the second control instruction and instructs the operating system to set the virtual display to a copy mode to complete the creation; and, in response to result information fed back by the virtual driver indicating that the creation is complete, executing the step of sending the first control instruction to the virtual driver; If it is detected that the virtual display currently exists in the electronic device, a third control instruction for detecting whether the virtual display is enabled is sent to the virtual driver; if the detection result feedback from the virtual driver indicates that it is in an enabled state, the step of sending the first control instruction to the virtual driver is triggered; if the detection result feedback from the virtual driver indicates that it is in a disabled state, the virtual display is set to an enabled state through the virtual driver, and the step of sending the first control instruction to the virtual driver is triggered.

3. The method according to claim 1 or 2, characterized in that The screen to be projected is the desktop of the target desktop itself or the target window area within the target desktop; and / or, the target desktop is the desktop displayed on the display of the electronic device, or, the target desktop is the desktop that the virtual display needs to display when the virtual display is set to extended mode; The virtual driver reads, in response to the first control instruction, frames of desktop image data from the target memory allocated for the virtual display in sequence, including: The virtual driver obtains the address information of the target memory from the operating system based on the target handle in response to the first control instruction; Based on the acquired address information of the target memory, each frame of desktop image data is read sequentially from the target memory.

4. The method according to claim 1 or 2, characterized in that The target document is a document with a human readable permission, and the receiving of the screen projection instruction for the screen to be projected specifically includes: When each target person meets the readability conditions, a screen projection instruction for the screen to be projected is received; each target person is a person who views the projected screen data.

5. The method according to claim 4, characterized in that Methods for determining whether each target person meets the readability conditions include: In response to the target document being a document with a readable permission for a person, outputting a first prompt message, wherein the first prompt message is used to prompt whether the page content of the document with a readable permission exists in the screen to be projected and whether to continue projecting; upon receiving an instruction from the user indicating that the screen projection is to be continued based on the first prompt message, determining whether each target person meets the readable condition; or, In response to the target document being a document with a person-readable permission, personal information of each target person is acquired, and based on the personal information of each target person, it is determined whether each target person meets a readability condition.

6. The method according to claim 5, characterized in that Determining whether each target person satisfies a readability condition based on the personnel information of each target person includes: Based on the personnel information of each target person, query whether each target person has the read permission for the target document to obtain a query result; in response to the query result indicating that each target person has the read permission for the target document, determine that each target person meets the read condition; and / or, In response to the query result indicating that at least one target person does not have read permission for the target document, outputting second prompt information indicating that a person without read permission is viewing the target document; The method further comprises: When receiving the user's instruction to continue screen projection based on the second prompt information, it is determined that each target person meets the readability condition; wherein, the second prompt information is also used to prompt whether to continue screen projection.

7. The method according to claim 5, characterized in that The obtaining of the personnel information of each target person includes: Sending a request for personnel information about persons viewing the projected screen data to the projection display terminal, so that the projection display terminal determines the personnel information of each target person according to a predetermined determination method, and feeds back the determined personnel information of each target person to the projection terminal; Receive the personnel information of each target person fed back by the projection display terminal; The predetermined determination method includes: Executing at least one personnel analysis method to obtain an analysis result of the at least one personnel analysis method; In the case that only one personnel analysis method is executed, the personnel information indicated by the analysis result of the personnel analysis method is used as the personnel information of each target person; or In the case where multiple personnel analysis methods are executed, the personnel information indicated by the analysis results of the multiple personnel analysis methods is combined to obtain the personnel information of each target person; The at least one personnel analysis method includes one or more of the following analysis methods: Calling a predetermined camera to perform image acquisition to obtain a target image, and performing personnel analysis on the target image to obtain personnel information; wherein the predetermined camera is a camera having a field of view that includes the attendees; Based on the identifier of the conference room where the projection display terminal is located, a request for information about the persons participating in the conference in the conference room at the current time is sent to the conference management system to obtain the personnel information of the persons participating in the conference at the current time; In response to the projection display terminal currently being connected to the video conferencing client, personnel information of persons participating in the conference is requested from the video conferencing client.

8. A screen projection device, characterized in that: Applied to a screen projection end, the screen projection end is a screen projection client in an electronic device or a screen projection device connected to the electronic device, and the screen projection end communicates with a virtual driver in the electronic device; The device comprises: The first sending module is used to send a first control instruction to the virtual driver in response to the satisfaction of the screen projection trigger condition, so that the virtual driver responds to the first control instruction and sequentially reads each frame of desktop image data from the target memory allocated for the virtual display; wherein the screen projection trigger condition includes receiving a screen projection instruction for the screen to be projected, and the screen to be projected is a screen in the target desktop and contains the page content of the target document; the virtual display is created by the virtual driver and after the virtual display is created, the operating system of the electronic device obtains the desktop image data of the target desktop and stores each frame of the obtained desktop image data in the target memory for use as The virtual display data of the virtual display; the sending of the first control instruction to the virtual driver comprises: sending the first control instruction carrying a target handle to the virtual driver; wherein the target handle is: a handle allocated to the virtual display by the operating system and obtained through a designated interface of the operating system based on a display identifier of the virtual display read from a configuration file of the virtual driver; the handle allocated by the virtual display is address information of a memory address used to cache interaction data when interacting with the operating system about information about the virtual display; the target memory is: memory corresponding to the address information obtained based on the target handle; An acquisition module, configured to acquire each frame of desktop image data read by the virtual driver; The second sending module is used to send the projection data of the screen to be projected to the projection display end based on the obtained desktop image data of each frame for display by the projection display end.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images