Video conference system and virtual whiteboard operation method

Abstract

A video conferencing system and a method for operating a virtual whiteboard therewith are disclosed. The virtual whiteboard operation method includes the following steps: Executing a video conferencing program and acquiring a video stream from at least one remote participant. Acquiring a first virtual whiteboard image of the remote participant based on their video stream. Comparing the first virtual whiteboard image with a second virtual whiteboard image of a local participant. Updating the second virtual whiteboard image of the local participant based on the difference between the first and second virtual whiteboard images. Streaming the updated second virtual whiteboard image to the remote participant via the video conferencing program, causing the remote participant to display the updated second virtual whiteboard image in a participant display area of ​​the video conferencing program.

Claims

1. A virtual whiteboard operation method, applicable to a video conferencing system including a camera device, the method comprising: Run a video conferencing program and acquire the video stream of at least one remote participant; Based on the video stream of the at least one remote conference participant, a first virtual whiteboard image of the at least one remote conference participant is obtained; the first virtual whiteboard image is compared with a second virtual whiteboard image of a local conference participant recorded in local memory to obtain a difference image; based on the difference image between the first virtual whiteboard image and the second virtual whiteboard image, the second virtual whiteboard image of the local conference participant is updated; and the updated second virtual whiteboard image is streamed to the at least one remote conference participant through the video conferencing program, causing the at least one remote conference participant to display the updated second virtual whiteboard image in a participant display area of ​​the video conferencing program, wherein the step of obtaining the first virtual whiteboard image associated with the at least one remote conference participant based on the video stream of the at least one remote conference participant includes: displaying a video conferencing interface including the video stream of the at least one remote conference participant; Capture the video conferencing interface image of the video conferencing program using the screenshot function; and identify the first virtual whiteboard image associated with the at least one remote conference participant from the video conferencing interface image.

2. The virtual whiteboard operation method as described in claim 1, wherein the step of identifying the first virtual whiteboard image associated with the at least one remote meeting participant from the meeting interface image includes: Detect a preset outline corresponding to a predetermined color in the conference interface image to obtain the first virtual whiteboard image with the preset outline.

3. The virtual whiteboard operation method as described in claim 1, wherein prior to the step of streaming the updated second virtual whiteboard image to the at least one remote conference participant via the video conferencing program, the method further comprises: The updated second virtual whiteboard image is concatenated to the video conferencing program via a Device Media Foundation Translator (DMFT) module.

4. The virtual whiteboard operation method as described in claim 1 further includes: During the execution of the video conferencing program, a real-time image sequence is captured by the camera device, wherein the real-time image sequence includes multiple captured images; a hand key point detection model is used to analyze the finger positions in each of the multiple captured images to obtain a handwritten input handwriting; and the second virtual whiteboard image of the local conference participant is updated based on the handwritten input handwriting.

5. The virtual whiteboard operation method as described in claim 4, wherein the step of analyzing the finger positions in each of the plurality of captured images using the hand key point detection model to obtain the handwriting input includes: The real-time image sequence captured by the camera device is transferred to the hand keypoint detection model running in the application layer through a Device Media Basic Transformation (DMFT) module, so as to determine the finger position according to the output of the hand keypoint detection model; and to draw a handwritten handwriting image according to the finger position in each of the multiple captured images.

6. The virtual whiteboard operation method as described in claim 5, wherein the step of updating the second virtual whiteboard image of the local conference participant based on the handwritten input includes: An updated second virtual whiteboard image is generated by synthesizing the handwritten handwriting image with the second virtual whiteboard image recorded in the local memory.

7. The virtual whiteboard operation method as described in claim 1, further comprising: The video conferencing program displays the first virtual whiteboard image of at least one remote conference participant and the second virtual whiteboard image of the local conference participant through multiple participant display areas.

8. The virtual whiteboard operation method as described in claim 1 further includes: The initial state of the second virtual whiteboard image for the local conference participants is determined based on a background image.

9. The virtual whiteboard operation method described in claim 8, wherein the background image includes a pure white image or a slide image.

10. A video conferencing system, comprising: A camera device; a transceiver; A storage device that stores multiple instructions; A processor, coupled to the camera device, the transceiver, and the storage device, executes the plurality of instructions to: execute a video conferencing program and acquire a video stream from at least one remote conference participant; acquire a first virtual whiteboard image of the at least one remote conference participant based on the video stream of the at least one remote conference participant; compare the first virtual whiteboard image with a second virtual whiteboard image of a local conference participant recorded in local memory, and acquire a difference image; update the second virtual whiteboard image of the local conference participant based on the difference image between the first virtual whiteboard image and the second virtual whiteboard image; and stream the updated second virtual whiteboard image to the at least one remote conference participant via the transceiver through the video conferencing program, causing the at least one remote conference participant to display the updated second virtual whiteboard image in a participant display area of ​​the video conferencing program, wherein the processor further performs: displaying a video conferencing interface including the video stream of the at least one remote conference participant; Capture the video conferencing interface image of the video conferencing program using the screenshot function; and identify the first virtual whiteboard image associated with the at least one remote conference participant from the video conferencing interface image.

Images