Screen mirroring method, video conference device, and system

Abstract

Disclosed in the present invention are a screen mirroring method, a video conference device, and a system. The method comprises: receiving a video stream to be screen-mirrored sent by a screen mirroring initiating end; decoding said video stream to generate a plurality of images to be screen-mirrored; creating a virtual camera, so that by means of a standard camera interface, a video conference screen mirroring application controls the virtual camera to acquire said images; and screen-mirroring said acquired images by means of the video conference screen mirroring application. By implementing the present invention, a video conference screen mirroring application can be docked with an operating system by means of a set of standard wireless screen mirroring acquisition interfaces, so that images to be screen-mirrored can be acquired and screen-mirrored.

Description

Screen projection method, video conferencing device and system Technical Field

[0001] The present invention relates to the field of screen projection for video conferencing applications, and in particular to a screen projection method, video conferencing equipment, and a system. Background Art

[0002] Given that the video conferencing screen projection applications on the market now collect wireless projection images by allowing the video conferencing screen projection applications to connect to the wireless screen projection protocol to receive video streams, thereby obtaining wireless projection images; the video conferencing screen projection application needs to have the ability to process video streams itself or the projection initiator needs to transmit the video stream to the operating system for the operating system to process before sending it to the video conferencing screen projection application. However, the process also needs to consider how the video streams of different protocols on the video conferencing screen projection application are connected to the operating system. As a result, the video conferencing screen projection application needs to develop different codes to connect to the operating system, which greatly increases the difficulty of developing the video conferencing screen projection application. Therefore, how to reduce the difficulty of developing the video conferencing screen projection application is an urgent problem that needs to be solved.

[0003] Summary of the Invention

[0004] The present invention provides a screen projection method, video conferencing equipment and system, which can realize the video conferencing screen projection application to connect with the operating system through a set of standard wireless screen projection acquisition interfaces, obtain the image to be projected and project it.

[0005] The present invention provides a screen projection method, comprising: receiving a video stream to be projected sent by a screen projection initiator;

[0006] Decoding the video stream to be projected to generate a plurality of images to be projected;

[0007] Create a virtual camera so that the video conferencing screen projection application can control the virtual camera through a standard camera interface to capture each image to be projected;

[0008] The collected images to be projected are projected through the video conference projection application.

[0009] Furthermore, receiving the video stream to be projected sent by the projection initiator includes:

[0010] Establishing protocol interaction with the screen projection initiator according to the screen projection protocol to which the video stream to be projected belongs;

[0011] After establishing protocol interaction, receive the video stream to be projected sent by the screen projection initiator.

[0012] Furthermore, the standard camera interface includes: camera api2.

[0013] Furthermore, when it is detected that multiple video streams to be projected are received from the screen projection initiator, each video stream to be projected is decoded to generate a plurality of images to be projected corresponding to each video stream to be projected;

[0014] Splicing the images to be projected at the same time between the video streams to be projected to generate several spliced ​​images to be projected;

[0015] Create a virtual camera so that the video conferencing screen projection application can control the virtual camera through a standard camera interface to capture the stitched images to be projected;

[0016] The collected spliced ​​images to be projected are projected through the video conference projection application.

[0017] Furthermore, when it is detected that multiple video streams to be projected are received from the screen projection initiator, each video stream to be projected is decoded to generate a plurality of images to be projected corresponding to each video stream to be projected;

[0018] Create a virtual camera corresponding to each video stream to be projected, so that the video conferencing projection application controls each virtual camera to capture several images to be projected of the corresponding video stream to be projected through a standard camera interface;

[0019] The video conference projection application projects the images to be projected captured by each virtual camera onto each independent display area.

[0020] Based on the above method embodiment, the present invention provides a corresponding device embodiment;

[0021] The present invention provides a video conferencing device, which executes any one of the screen projection methods described in the present invention.

[0022] Based on the above method embodiment, the present invention provides a corresponding system embodiment;

[0023] The present invention provides a screen projection system, comprising: a screen projection initiator and a video conferencing device provided by an embodiment of the device item of the present invention;

[0024] The screen projection initiator is used to send the video stream to be projected to the video conferencing device.

[0025] The embodiments of the present invention have the following beneficial effects:

[0026] The present invention provides a screen projection method, which includes: receiving a video stream to be projected sent by a screen projection initiator; decoding the video stream to be projected to generate a number of images to be projected; creating a virtual camera so that a video conference screen projection application controls the virtual camera through a standard camera interface to capture each image to be projected and project the captured images to be projected; by implementing the present invention, the video conference screen projection application can be connected to the operating system through a set of standard wireless screen projection capture interfaces, obtain the images to be projected for projection, and reduce the difficulty of developing the video conference screen projection application. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] FIG1 is a schematic flow chart of a screen projection method provided by one embodiment of the present invention;

[0028] FIG2 is a schematic diagram of a screen projection process flow provided by an embodiment of the present invention;

[0029] FIG3 is a schematic diagram of a screen projection system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0031] As shown in FIG1 , an embodiment provides a screen projection method, including:

[0032] Step S1, receiving the video stream to be projected sent by the screen projection initiator;

[0033] Step S2: decoding the video stream to be projected to generate a plurality of images to be projected;

[0034] Step S3: creating a virtual camera so that the video conferencing screen projection application controls the virtual camera to capture each image to be projected through a standard camera interface;

[0035] Step S4: Project the collected image to be projected through a video conference projection application.

[0036] Regarding step S1, in a preferred embodiment, receiving the video stream to be projected sent by the screen projection initiator includes:

[0037] Establishing protocol interaction with the screen projection initiator according to the screen projection protocol to which the video stream to be projected belongs;

[0038] After establishing protocol interaction, receive the video stream to be projected sent by the screen projection initiator;

[0039] Specifically, the screen projection initiator initiates screen projection, which can refer to initiating wireless screen projection on a mobile phone or computer. The screen projection protocol used can be AirPlay or Miracast. First, the video conferencing device will establish a protocol interaction with the screen projection initiator based on the screen projection protocol of the video stream to be projected by the screen projection initiator. After the protocol interaction is established, it obtains the video stream to be projected from the screen projection initiator. Among them, the video conferencing device can handle various protocols including AirPlay and Miracast.

[0040] For step S2, in an optional embodiment, after the video conferencing device receives the video stream to be projected from the screen projection initiator, the decoder module of the video conferencing device decodes the received video stream to be projected into an image.

[0041] For steps S3 and S4, in an optional embodiment, the standard camera interface includes: camera api2;

[0042] Specifically, the video conferencing device creates a virtual camera that is compatible with the standard APIs exposed by the operating system platform where the video conferencing device is installed, such as the Android Camera API2, which is widely used in the industry. The video conferencing screen projection application uses standard Camera API operations to capture images from the virtual camera. These images are obtained from the output of the image decoder module by the virtual camera and then presented to the user by the video conferencing screen projection application.

[0043] In a preferred embodiment, when it is detected that multiple video streams to be projected are received from the screen projection initiator, each video stream to be projected is decoded to generate a plurality of images to be projected corresponding to each video stream to be projected;

[0044] Splicing the images to be projected at the same time between the video streams to be projected to generate several spliced ​​images to be projected;

[0045] Create a virtual camera so that the video conferencing screen projection application can control the virtual camera through a standard camera interface to capture each spliced ​​image to be projected, and project the captured spliced ​​image to be projected through the video conferencing screen projection application;

[0046] Specifically, the video conferencing equipment can process multiple protocols at the same time, that is, it can establish protocol interaction with multiple screen projection initiators at the same time. After establishing protocol interaction, it can also simultaneously receive video streams to be projected sent by multiple screen projection initiators. The decoder module decodes the video stream to be projected sent by each screen projection initiator, obtains the image corresponding to each screen projection initiator, and then splices the images corresponding to each screen projection initiator together with the same start time, that is, splices multiple pictures into one picture; establishes a virtual camera, and the video conferencing screen projection application controls the virtual camera through the standard camera interface to collect the spliced ​​images to be projected, and displays the collected images to the user.

[0047] In a preferred embodiment, when it is detected that multiple video streams to be projected are received from the screen projection initiator, each video stream to be projected is decoded to generate a plurality of images to be projected corresponding to each video stream to be projected;

[0048] Create a virtual camera corresponding to each video stream to be projected, so that the video conferencing screen projection application controls each virtual camera through a standard camera interface to capture several images to be projected of the corresponding video stream to be projected, and projects the images to be projected captured by each virtual camera onto each independent display area through the video conferencing screen projection application;

[0049] Specifically, consistent with the previous embodiment, the video conferencing device can process multiple protocols at the same time and establish protocol interactions with multiple screen projection initiators. After receiving the video streams to be projected sent by multiple screen projection initiators and decoding them, the difference from the previous implementation is that multiple virtual cameras are established for the images corresponding to each screen projection initiator, that is, each virtual camera corresponds to an image of a screen projection initiator, and then the video conferencing screen projection application will control each of the virtual cameras to capture the image of the corresponding screen projection initiator, and then display the image captured by each virtual camera on multiple independent screens.

[0050] Based on the above method embodiment, the present invention provides a corresponding device embodiment;

[0051] Another embodiment of the present invention provides a video conferencing device, which is used to implement the screen projection method of any embodiment of the present invention.

[0052] As shown schematically in FIG2 , the video conferencing device is equipped with a protocol module, a decoder module, a virtual camera module, and a video conferencing screen projection application (video conferencing screen projection App);

[0053] The screen projection initiator initiates the screen projection, and the protocol module of the video conferencing device establishes a protocol interaction with the screen projection initiator according to the protocol to which the video stream of the screen projection initiator belongs, and receives the video stream sent by the screen projection initiator; the protocol module transmits the video stream to the decoder module, and the decoder module decodes the video stream to be projected and obtains a number of images to be projected; the virtual camera module establishes a virtual camera, and the virtual camera is adapted to the standard API exposed by the operating system platform, such as the camera API2 on the Android system; the video conferencing screen projection application only needs to control the virtual camera through the standard camera API to collect each image to be projected, and the collected images to be projected can be projected, that is to say, the docking process between the video conferencing screen projection application and the operating system to which the video conferencing device belongs is simplified, and there is no need to develop additional code to connect to the interfaces of different protocols, nor to process the video stream, which greatly reduces the development difficulty of the video conferencing screen projection application.

[0054] In the past, the protocol module was in the video conferencing screen projection application. Every video conferencing screen projection application on the video conferencing device must have a protocol module, and it also needs to have the ability to process the video stream, or it needs to transmit the video stream to the operating system for the operating system to process it before sending it to the video conferencing screen projection application; if the video conferencing screen projection application transmits the video stream to the operating system for the operating system to process it before sending it back, it is also necessary to consider how the video streams of different protocols on the video conferencing screen projection application are connected to the operating system; if the video conferencing screen projection application handles it by itself, the video conferencing screen projection application needs to have many functions, be able to establish protocol interaction with different screen projection initiators, and be able to process images of different protocols. Every video conferencing screen projection application development must develop these functions, which will greatly increase the difficulty of development.

[0055] Based on the above device embodiment, the present invention provides a corresponding system embodiment;

[0056] As shown in FIG3 , another embodiment of the present invention provides a screen projection system, comprising: a screen projection initiator and a video conferencing device provided in the device embodiment; the screen projection initiator is configured to send a video stream to be projected to the video conferencing device;

[0057] Specifically, the screen projection initiator is used to initiate screen projection to the video conferencing device, which may refer to initiating wireless screen projection on a mobile phone or computer, and then the video conferencing device establishes a protocol interaction with the mobile phone. The video conferencing device can handle various protocols including airplay and miracast. After establishing the protocol interaction, the video conferencing device obtains the video stream to be projected sent by the screen projection initiator; after the video conferencing device decodes the video stream into several images to be projected, the video conferencing screen projection application in the video conferencing device collects several projection images through the virtual camera module, and then projects the collected projection images.

[0058] By implementing the above-mentioned embodiments of the present invention, the video conferencing equipment can be capable of processing video streams and multiple protocols, and combined with virtual camera technology, the video conferencing screen projection application can be connected to the operating system through a set of standard wireless screen projection acquisition interfaces, so as to obtain the image to be projected for screen projection, thereby reducing the difficulty of developing the video conferencing screen projection application.

[0059] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications are also regarded as protection of the present invention.

Claims

1. A screen projection method, characterized in that: include: Receive the video stream to be projected sent by the screen projection initiator; Decoding the video stream to be projected to generate a number of images to be projected; Create a virtual camera so that the video conference screen projection application can control the virtual camera to capture each image to be projected through a standard camera interface; The collected images to be projected are projected through the video conference projection application.

2. A screen projection method as claimed in claim 1, characterized in that: Receive the video stream to be projected sent by the screen projection initiator, including: Establishing protocol interaction with the screen projection initiator according to the screen projection protocol to which the screen projection video stream belongs; After establishing the protocol interaction, receive the video stream to be projected sent by the screen projection initiator.

3. A screen projection method according to claim 1, characterized in that: The standard camera interface includes: camera api2.

4. A screen projection method according to claim 1, characterized in that: Also includes: When detecting and receiving video streams to be projected sent by multiple screen projection initiators, decoding each video stream to be projected, and generating a plurality of images to be projected corresponding to each video stream to be projected; Splicing the images to be projected at the same time between the video streams to be projected to generate a number of spliced ​​images to be projected; Create a virtual camera so that the video conference screen projection application can control the virtual camera to collect each spliced ​​screen projection image through a standard camera interface; The collected spliced ​​images to be projected are projected through the video conference screen projection application.

5. A screen projection method as claimed in claim 1, characterized in that: Also includes: When detecting and receiving video streams to be projected sent by multiple screen projection initiators, decoding each video stream to be projected, and generating a plurality of images to be projected corresponding to each video stream to be projected; Create a virtual camera corresponding to each video stream to be projected, so that the video conference projection application controls each virtual camera to collect several images to be projected of the corresponding video stream to be projected through a standard camera interface; The images to be projected, which are captured by each of the virtual cameras, are projected on each independent display area through the video conference projection application.

6. A video conferencing device, characterized in that: The video conferencing device is used to execute the screen projection method as described in any one of claims 1 to 5.

7. A screen projection system, characterized in that: include: A screen projection initiator and a video conferencing device as claimed in claim 6; The screen projection initiator is used to send the video stream to be projected to the video conferencing device.

Images