Method and system for controlling terminal video input source in real time

Abstract

The invention provides a method and a system for controlling a terminal video input source in real time. The method comprises the following steps that: director software accesses a video conference; the director software receives video streams of each path of camera of the participating terminal; the director software selects which video stream of the conference participating terminal is used as a video input source to be sent to the video conference through the conference control server; and the director software performs pan-tilt control on the camera where the video input source is located in the conference participating terminal through the conference control server. According to the method and the system for controlling the video input source of the terminal in real time, the video input source of the participating terminal can be effectively selected and controlled at a far end.

Description

Technical Field

[0001] The present invention relates to the field of video communication technology, and in particular to a method and system for controlling a terminal video input source in real time. Background Art

[0002] In a video conference, generally, each participating terminal only needs to send one video stream to the video conference, which not only saves network bandwidth resources, but also provides a better experience for participating terminals. Because in most application scenarios, each participating terminal only needs to send one video stream to the video conference to meet the viewing needs of other participating terminals. When a participating terminal sends multiple video streams to the video conference, it will not only distract the attention of the participants, but also cause a great waste of bandwidth resources, affecting the participating terminal's experience. Therefore, when there are multiple cameras at the participating terminal for image acquisition, it is generally necessary to select one of the video streams captured by the camera to send to the video conference.

[0003] In the prior art, when there are multiple cameras at a participating terminal to capture the images of the participating terminal, the video input source can only be selected and controlled at the participating terminal, and cannot be selected and controlled remotely. This requires participants to constantly adjust the cameras according to actual needs during the video conference and cannot focus on the meeting content itself, which affects the participants' meeting experience. Summary of the invention

[0004] The present invention provides a method and system for controlling a terminal video input source in real time, which effectively realizes the selection and control of the video input source of a conference participant terminal at a remote end.

[0005] In a first aspect, the present invention provides a broadcast director software for accessing a video conference;

[0006] The director software receives the video streams from the cameras of the participating terminals;

[0007] The director software selects which video stream of the participating terminal is sent to the video conference as the video input source through the conference control server;

[0008] The director software performs pan / tilt control on the camera where the video input source is located in the participating terminal through the conference control server.

[0009] In a second aspect, the present invention further provides a system for controlling a terminal video input source in real time, characterized in that the system comprises: an access module, a receiving module, a selection module and a control module, wherein:

[0010] The access module is used for the director software to access the video conference;

[0011] The receiving module is used for the director software to receive the video streams of the cameras of the participating terminals;

[0012] The selection module is used for the director software to select which video stream of the participating terminal is sent as a video input source to the video conference through the conference control server;

[0013] The control module is used for the director software to perform pan / tilt control on the camera where the video input source in the conference terminal is located through the conference control server.

[0014] The advantages of the real-time control terminal video input source method and system provided by the present invention are: first, the video cameras of the participating terminals are remotely selected and controlled, so that the participants do not have to worry about technical issues such as adjusting the cameras, and can focus on the meeting content itself, reduce external interference, and improve the participating experience of the participating terminals; second, it is convenient for conference control personnel to uniformly control and manage the cameras of the participating terminals, which can reduce the occurrence of unexpected situations; third, by remotely controlling the cameras of the participating terminals, it is possible to quickly respond to changes in meeting needs and improve the efficiency of the meeting; fourth, remote control can ensure that important content is correctly recorded and archived, which is convenient for subsequent review and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0016] Figure 1 is a flow chart of a method for real-time control of terminal video input source provided by an embodiment of the present invention;

[0017] Figure 2 is a flow chart of receiving a video stream provided by an embodiment of the present invention;

[0018] Figure 3 is a flow chart of selecting a video input source provided by an embodiment of the present invention;

[0019] Figure 4 is a block diagram of a real-time control terminal video input source system provided by an embodiment of the present invention;

[0020] Figure 5 It is an architecture diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. SUMMARY OF THE INVENTION

[0023] As mentioned above, the present invention provides a flexible management method and system for real-time control of terminal video input sources, which can significantly improve the experience of participating terminals.

[0024] Exemplary Methods

[0025] Figure 1 : is a flow chart of a method for real-time control of a terminal video input source provided by an embodiment of the present invention, and the embodiment comprises the following steps:

[0026] S101: The director software joins the video conference. After the video conference starts, the director software needs to join the video conference like ordinary participating terminals, but the difference from ordinary participating terminals is that if there are multiple cameras at the participating terminal to collect the screen of the participating terminal, the director software will receive the video streams of all cameras of each participating terminal in the video conference, while the ordinary participating terminal only receives the video streams of one camera of other participating terminals in the video conference.

[0027] Specifically, first, the conference control personnel logs into the director software after passing the identity authentication. Because the director software has the authority to control and adjust all cameras of other participating terminals, in order to ensure the privacy of the participating terminals, the identity of the controller of the director software needs to be verified. Only after the account and password of the conference control personnel are successfully verified can they successfully log into the director software and control and adjust the cameras of other participating terminals.

[0028] Among them, in order to prevent the conference control personnel from affecting the normal convening of the video conference due to their entry into the conference, the conference control personnel join the video conference in a different way from other participating terminals. The conference control personnel join the video conference in an invisible way through the director software, and other participating terminals cannot sense the conference control personnel's entry into the conference. In this way, the conference control personnel can adjust and control the cameras of the participating terminals according to the actual needs of the meeting without affecting the normal convening of the meeting.

[0029] S102: The director software receives the video streams of the cameras of the participating terminals. Because the conference control personnel need to manage the cameras of all participating terminals in a unified manner through the director software, the conference control personnel need to receive the video streams of all cameras of the participating terminals through the director software.

[0030] like Figure 2 As shown, step S102 specifically includes:

[0031] S202: The director software sends a message to the video conference server requesting all video streams of the participating terminals.

[0032] Specifically, after the video conference starts, the director software will need to request the video streams of each participating terminal through the video conference server. The request stream message sent by the director software to the server usually contains the identifier, resolution, frame rate and other parameters of the required video stream.

[0033] S202: The video conference server requests all video streams of the participating terminals.

[0034] Specifically, when the video conference server receives the stream request message from the director software, it will establish a connection with the corresponding participant terminal and request the video stream required by the director software from the participant terminal.

[0035] S203: The participating terminal encodes the video streams of all its cameras.

[0036] Specifically, when the participating terminal receives the stream request message from the video conferencing server, in order to reduce the size of the video data, reduce bandwidth requirements, make the video stream more efficient through the network transmission and enhance the security of data transmission, the participating terminal will select a suitable encoding format to encode the required video stream and package the encoded video data into a streaming media format.

[0037] S204: The participating terminal sends the encoded video stream to the video conference server.

[0038] S205: The video conference server sends the video stream to the director software.

[0039] Among them, since the video conference server needs to send multiple video streams to the director software, and the director software requests these video streams so that the conference control personnel can select and manage the video input sources that the participating terminals need to send to the video conference, the director software does not need to request video streams with too high resolution. In addition, in order to reduce the occupation of bandwidth resources and improve the transmission efficiency of video streams, the video streams sent by the video conference server to the director software are all low-resolution video streams. However, the video streams sent in the video conference are for all participating terminals to watch. In order to improve the conference experience of participants, higher-resolution video streams are required.

[0040] S103: The director software selects, through the conference control server, which video stream of the participating terminal is sent as a video input source to the video conference.

[0041] Specifically, in order to improve the conference experience and save bandwidth resources, under normal circumstances, the participating terminal only needs to send one video stream to the video conference. In order to enable the participants to focus on the meeting itself, the conference control personnel need to use the director software to select which video stream the participating terminal will send as the video source to the video conference.

[0042] For example, in the process of online live teaching, in order to improve the authenticity of teaching, multiple cameras will be installed at the teacher's location to collect the teacher's lecture images from multiple angles. In order to allow the teacher to concentrate on teaching, the conference control personnel will manage and control the images collected by different cameras through the director software in the background. Figure 5 As shown, it is an architecture diagram provided by an embodiment of the present invention. An ordinary terminal directly joins a video conference through the Internet, while the director software joins the video conference invisibly through the Internet. There are two screens in the video conference screen: the main speaker preview and the audience preview. The conference control personnel can see the main speaker preview, the audience preview and the screens collected by each camera through the director software. When the conference control personnel clicks the main speaker preview, the director software can select one of the camera screens as the main speaker preview screen through the conference control server, and the audience screen is the student screen. Generally, a single camera collects the student screen. When multiple cameras collect the student screen, the conference control personnel can also select which camera of the student terminal collects the video stream as the video input source through the director software. The conference control personnel can perform PTZ control (i.e., pan / tilt control) on the selected camera through the director software, for example, control the rotation of the camera in the horizontal and vertical directions, and can also perform zoom control (ZOOM) on the camera to adjust the magnification of the camera.

[0043] Specifically, Figure 3 As shown, step S103 specifically includes:

[0044] S301: The conference control personnel preview the video images of all cameras of the participating terminals through the director software.

[0045] Continuing with the above embodiment, for example, the teacher terminal has three cameras to capture the teaching screen. Camera 1 captures the full body image of the teacher, camera 2 captures the blackboard image of the teacher, and camera 3 captures a close-up image of the teacher's upper body. The conference control personnel can see the images captured by camera 1, camera 2 and camera 3 through the director software.

[0046] S302: The conference control personnel selects a video stream from a camera as a video input source and sends it to the video conference according to the current needs of the conference. After that, the video conference server sends the video stream selected by the conference control personnel to the video conference.

[0047] Continuing with the above embodiment, for example, when a teacher is explaining on the blackboard, it is necessary to capture the screen of the teacher's writing on the blackboard. The conference controller needs to select camera 1 and send the video stream of camera 1 as the input source to the video conference. At this time, the preview screen of the speaker in the video conference is the screen captured by camera 1.

[0048] In addition, the participating terminals can also select which camera's video stream to send as a video input source to the video conference.

[0049] In the above embodiment, the teacher can also select at the terminal which camera's captured video stream is sent as the input source to the video conference.

[0050] S104: The director software performs pan / tilt control on the camera where the video input source of the conference participant terminal is located through the conference control server.

[0051] The pan / tilt control is the operation of the conference controller to perform horizontal and vertical rotation and zooming of the camera serving as the video input source of the participating terminal according to the actual needs of the conference.

[0052] Specifically, the director software will perform pan / tilt control (PTZ control) on the camera where the video input source is located in the participating terminal through the restAPI interface provided by the conference control server, so that the conference control personnel can make appropriate adjustments to the images captured by the camera in the video conference according to actual conditions.

[0053] In the above embodiment, for example, if the range of the teacher's writing on the blackboard exceeds the range of the camera's captured image, the conference control personnel can adjust the camera 1 up and down or left and right through the restAPI interface provided by the conference control server so that the camera 1 can capture the image of the teacher writing on the blackboard. Alternatively, the conference control personnel can also reduce the image captured by the camera so that the camera 1 can capture the image of the teacher writing on the blackboard.

[0054] In addition, in order to provide an immersive conference experience to participants in the video conference, the audio will be updated synchronously when the video screen sent by the participating terminal to the video conference is switched or the pan / tilt control of the camera where the video input source in the participating terminal is located is performed.

[0055] Exemplary Systems

[0056] Correspondingly, an embodiment of the present invention further provides a system for controlling terminal video input sources in real time. Figure 3 : is a block diagram of a real-time control terminal video input source system provided by an embodiment of the present invention, such as Figure 3 As shown, the system 400 provided in this embodiment includes: a director software 401, a conference control server 402, a video conference server 403 and a conference participant terminal 404, wherein:

[0057] The director software also includes an access module 405, a receiving module 406, a selection module 407 and a control module 408, wherein:

[0058] The access module 405 is used for the director software to access the video conference;

[0059] The receiving module 406 is used for the director software to receive the video streams of the cameras of the participating terminals;

[0060] The selection module 407 is used for the director software to select which video stream of the participating terminal is sent as a video input source to the video conference through the conference control server;

[0061] The control module 408 is used for the director software to perform pan / tilt control on the camera where the video input source in the conference terminal is located through the conference control server.

[0062] The pan / tilt control is that the director software performs horizontal and vertical rotation and zoom operations on the camera serving as the video input source of the participating terminals through the conference controller according to the actual needs of the meeting.

[0063] The access module 405 includes:

[0064] Login module 409, used for the conference control personnel to log in to the director software after passing the identity authentication;

[0065] The invisible joining module 410 is used for the conference controller to join the video conference in an invisible manner through the director software.

[0066] The receiving module 406 includes:

[0067] The first stream request module 411 is used for the director software to send a message requesting all video streams of the participating terminals to the video conference server;

[0068] The video conference server 403 includes:

[0069] The second stream request module 414 is used for the video conference server to request all video streams of the participating terminal;

[0070] The conference participant terminal 404 includes:

[0071] Encoding module 416, used for the participating terminal to encode the video streams of all its cameras;

[0072] A first sending module 417, configured for the participating terminal to send the encoded video stream to the video conference server;

[0073] The video conference server 403 also includes:

[0074] The second sending module 415 is used for the video conferencing server to send the video stream to the director software.

[0075] The selection module 407 includes:

[0076] Preview module 412, used for the conference control personnel to preview the video images of all cameras of the participating terminals through the director software;

[0077] The video source selection module 413 is used by the conference control personnel to select a video stream of a camera as a video input source and send it to the video conference through the director software according to the current needs of the conference, and send the selection instruction to the video conference server.

[0078] The video conference server 403 also includes:

[0079] The video stream selected by the conference control personnel is sent to the submodule in the video conference.

[0080] The video stream sent to the director software is a low-resolution video stream, and the video stream sent to the video conference is a high-resolution video stream.

[0081] In addition to the conference control personnel being able to select which camera's video stream to send as the video input source to the video conference through the director software, the participating terminals can also select which camera's video stream to send as the video input source to the video conference.

[0082] The control module 408 includes:

[0083] The director software uses the restAPI interface provided by the conference control server to perform a submodule of the pan / tilt control of the camera where the video input source in the participating terminal is located.

[0084] In order to provide an immersive conference experience to the participants in the video conference, when the video screen sent by the participating terminal to the video conference is switched or the camera where the video input source in the participating terminal is located is controlled by the pan / tilt function, the audio will also be updated synchronously.

[0085] It should be noted that although the operations of the method for real-time controlling the terminal video input source of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0086] In addition, although some devices, units or modules of the real-time control terminal video input source system are mentioned in the above detailed description, this division is only exemplary and non-mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules described above can be concretized in one module. On the contrary, the features and functions of a module described above can be further divided into being concretized by multiple modules.

[0087] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims. According to the above disclosure of the present invention, the present invention provides: 1. A method for real-time control of a terminal video input source, characterized in that the method comprises: The director software is connected to the video conference; The director software receives the video streams from the cameras of the participating terminals; The director software selects which video stream of the participating terminal is sent to the video conference as the video input source through the conference control server; The director software performs pan / tilt control on the camera where the video input source is located in the participating terminal through the conference control server. 2. The method for real-time control of terminal video input source according to item 1 above is characterized in that the pan / tilt control is a conference controller that performs horizontal, vertical rotation and zoom operations on the camera serving as the video input source of the participating terminal according to the actual needs of the meeting. 3. The method for real-time control of terminal video input source according to item 1 above is characterized in that the step of connecting the director software to the video conference specifically includes: After the conference controller passes the identity authentication, he / she logs into the director software; The conference control personnel join the video conference in invisible mode through the director software. 4. The method for real-time control of terminal video input source according to any one of items 1 to 3 above is characterized in that the step of the director software receiving the video streams of the cameras of the conference terminal specifically comprises: The director software sends a message to the video conference server requesting all video streams of the participating terminals; The video conference server requests all video streams of the participating terminal; The participating terminal encodes the video streams of all its cameras; The participating terminal sends the encoded video stream to the video conference server; The video conference server sends the video stream to the director software. 5. The method for real-time control of terminal video input source according to any one of items 1 to 4 above is characterized in that the step of the director software selecting which video stream of the participating terminal is sent as the video input source to the video conference through the conference control server specifically comprises: The conference control personnel preview the video images of all cameras of the participating terminals through the director software; The conference control personnel selects a video stream of a camera as a video input source and sends it to the video conference according to the current needs of the conference. 6. The method for real-time control of terminal video input source according to item 5 above is characterized in that the steps after the conference control personnel selects a video stream of a camera as a video input source and sends it to the video conference according to the current needs of the conference include: The video conference server sends the video stream selected by the conference control personnel to the video conference. 7. The method for real-time control of terminal video input source according to any one of items 1 to 6 above is characterized in that the video stream sent to the director software is a low-resolution video stream, and the video stream sent to the video conference is a high-resolution video stream. 8. The method for real-time control of terminal video input source according to any one of items 1 to 7 above is characterized in that the participating terminals can also choose which camera's video stream to send as the video input source to the video conference. 9. The method for real-time control of terminal video input source according to any one of items 1 to 8 above is characterized in that the step of the director software controlling the camera where the video input source of the conference terminal is located through the conference control server specifically includes: The director software performs pan / tilt control on the camera where the video input source is located in the participating terminal through the restAPI interface provided by the conference control server. 10. The method for real-time control of terminal video input source according to any one of items 1 to 9 above is characterized in that when the video screen sent by the participating terminal to the video conference is switched or the pan-tilt control of the camera where the video input source in the participating terminal is located is performed, the audio will also be updated synchronously. 11. A real-time control terminal video input source system, characterized in that the system includes: director software, conference control server, video conference server and conference participating terminal, The director software also includes an access module, a receiving module, a selection module and a control module, wherein: The access module is used for the director software to access the video conference; The receiving module is used for the director software to receive the video streams of the cameras of the participating terminals; The selection module is used for the director software to select which video stream of the participating terminal is sent as a video input source to the video conference through the conference control server; The control module is used for the director software to perform pan / tilt control on the camera where the video input source in the conference terminal is located through the conference control server. 12. According to the real-time control terminal video input source system described in item 11 above, it is characterized in that the pan-tilt control is that the director software performs horizontal, vertical rotation and zoom operations on the camera serving as the video input source of the participating terminal through the conference controller according to the actual needs of the meeting. 13. The real-time control terminal video input source system according to item 11 above is characterized in that the access module comprises: The login module is used for conference control personnel to log in to the director software after passing identity authentication; The invisible conference joining module is used for conference control personnel to join the video conference in invisible mode through the director software. 14. The real-time control terminal video input source system according to any one of items 11 to 13 above, characterized in that: The receiving module comprises: The first stream request module is used for the director software to send a message requesting all video streams of the participating terminals to the video conference server; The video conference server comprises: A second stream request module, used for the video conference server to request all video streams of the participating terminal; The conference participating terminal includes: The encoding module is used for the participating terminal to encode the video streams of all its cameras; A first sending module, used for the participating terminal to send the encoded video stream to the video conference server; The video conference server also includes: The second sending module is used for the video conferencing server to send the video stream to the director software. 15. The real-time control terminal video input source system according to any one of items 11 to 14 above, characterized in that the selection module comprises: A preview module is used for the conference control personnel to preview the video images of all cameras of the participating terminals through the director software; The video source selection module is used by the conference control personnel to select a video stream from a camera as a video input source and send it to the video conference through the director software according to the current needs of the conference, and send the selection instruction to the video conference server. 16. The real-time control terminal video input source system according to item 15 above, characterized in that the video conference server further comprises: The video stream selected by the conference control personnel is sent to the submodule in the video conference. 17. The real-time control terminal video input source system according to any one of items 11 to 16 above is characterized in that the video stream sent to the director software is a low-resolution video stream, and the video stream sent to the video conference is a high-resolution video stream. 18. The real-time control terminal video input source system according to any one of items 11 to 17 above is characterized in that the participating terminals can also choose which camera's video stream to send as the video input source to the video conference. 19. The real-time control terminal video input source system according to any one of items 11 to 18 above, characterized in that the control module comprises: The director software uses the restAPI interface provided by the conference control server to perform a submodule of the pan / tilt control of the camera where the video input source in the participating terminal is located. 20. The real-time control terminal video input source system according to any one of items 11 to 19 above is characterized in that when the video screen sent by the participating terminal to the video conference is switched or the pan-tilt control of the camera where the video input source in the participating terminal is located is performed, the audio will also be updated synchronously.

Claims

1. A method for real-time control of terminal video input source, characterized in that: The method comprises: The director software is connected to the video conference; The director software receives the video streams from the cameras of the participating terminals; The director software selects which video stream of the participating terminal is sent to the video conference as the video input source through the conference control server; The director software performs pan / tilt control on the camera where the video input source is located in the participating terminal through the conference control server.

2. The method for real-time control of terminal video input source according to claim 1, characterized in that: The steps of connecting the director software to the video conference specifically include: After the conference control personnel pass the identity authentication, they log into the director software; The conference control personnel join the video conference in invisible mode through the director software.

3. The method for real-time control of terminal video input source according to claim 1 or 2, characterized in that: The step of the director software receiving the video streams of the cameras of each conference terminal specifically includes: The director software sends a message to the video conference server requesting all video streams of the participating terminals; The video conference server requests all video streams of the participating terminal; The participating terminal encodes the video streams of all its cameras; The participating terminal sends the encoded video stream to the video conference server; The video conference server sends the video stream to the director software.

4. The method for real-time control of terminal video input source according to any one of claims 1 to 3, characterized in that: The step of the director software selecting which video stream of the participating terminal is sent as a video input source to the video conference through the conference control server specifically includes: The conference control personnel preview the video images of all cameras of the participating terminals through the director software; The conference control personnel selects a video stream of a camera as a video input source and sends it to the video conference according to the current needs of the conference.

5. The method for real-time control of terminal video input source according to any one of claims 1 to 4, characterized in that: The video stream sent to the director software is a low-resolution video stream, and the video stream sent to the video conference is a high-resolution video stream.

6. A real-time control terminal video input source system, characterized in that: The system includes: director software, conference control server, video conference server and conference terminal. The director software also includes an access module, a receiving module, a selection module and a control module, wherein: The access module is used for the director software to access the video conference; The receiving module is used for the director software to receive the video streams of the cameras of the participating terminals; The selection module is used for the director software to select which video stream of the participating terminal is sent as a video input source to the video conference through the conference control server; The control module is used for the director software to perform pan / tilt control on the camera where the video input source in the conference terminal is located through the conference control server.

7. The real-time control terminal video input source system according to claim 6, characterized in that: The access module comprises: The login module is used for conference control personnel to log in to the director software after passing identity authentication; The invisible conference joining module is used for conference control personnel to join the video conference in invisible mode through the director software.

8. The real-time control terminal video input source system according to claim 6 or 7, characterized in that: The receiving module comprises: The first stream request module is used for the director software to send a message requesting all video streams of the participating terminals to the video conference server; The video conference server comprises: A second stream request module, used for the video conference server to request all video streams of the participating terminal; The conference participating terminal includes: The encoding module is used for the participating terminal to encode the video streams of all its cameras; A first sending module, used for the participating terminal to send the encoded video stream to the video conference server; The video conference server also includes: The second sending module is used for the video conferencing server to send the video stream to the director software.

9. The real-time control terminal video input source system according to any one of claims 6 to 8, characterized in that: The selection module comprises: A preview module is used for the conference control personnel to preview the video images of all cameras of the participating terminals through the director software; The video source selection module is used by the conference control personnel to select a video stream from a camera as a video input source and send it to the video conference through the director software according to the current needs of the conference, and send the selection instruction to the video conference server.

10. The real-time control terminal video input source system according to any one of claims 6 to 9, characterized in that: The video stream sent to the director software is a low-resolution video stream, and the video stream sent to the video conference is a high-resolution video stream.

Images