A device and system integrating a conference terminal and a video wall

Abstract

The present invention is applicable to the field of video conferencing technology, and provides a conference terminal and TV wall integrated device and system. The device includes: a main control system, which is used to obtain remote audio and video data, local audio and video data, and conference-related data; a conference processing system communicatively connected to the main control system, which is used to receive the remote audio and video data, local audio and video data, and conference-related data sent by the main control system, and after corresponding processing, send them to an audio and video playback device according to a first playback rule for playing the conference picture and conference audio; a TV wall processing module communicatively connected to the main control system, which is used to receive the remote video data sent by the main control system, and after corresponding processing, send it to a video playback device according to a second playback rule for playing the TV wall picture. By realizing the functions of a conference terminal and a TV wall in one device, the user cost is effectively saved, and the installation is simple.

Description

Technical Field

[0001] The present invention belongs to the technical field of conference video technology, and in particular relates to a device and system integrating a conference terminal and a video wall. Background Art

[0002] With the continuous development of digital networks and electronic information technology, video conferencing systems have begun to appear and gradually become popular, allowing users to conduct online meetings conveniently, greatly facilitating user use.

[0003] In traditional video conferencing systems, conference terminals and video walls are two different types of equipment, each designed to address different application scenarios. For example, in some large-scale project applications, branch venues typically only require a single conference terminal and supporting peripherals to enable audio and video interaction and communication with the main venue. Meanwhile, the main venue, in addition to the conference terminal for audio and video interaction and communication with branch venues, also requires a video wall device for video monitoring and order supervision of all branch venues.

[0004] This means that users must purchase an additional set of TV wall-related equipment, which not only increases user costs, but also makes the installation and use of the entire system more cumbersome, which is not conducive to user experience. Summary of the Invention

[0005] The present invention provides a conference terminal and TV wall integrated device and system, aiming to solve the problem in the prior art that the TV wall and conference terminals need to be arranged simultaneously, resulting in high costs and complicated installation and use.

[0006] The present invention is implemented as follows: a conference terminal and video wall integrated device, comprising:

[0007] A main control system, which is used to obtain remote audio and video data, local audio and video data, and conference-related data;

[0008] A conference processing system in communication with the main control system, configured to receive remote audio and video data, local audio and video data, and conference-related data sent by the main control system, and send the data to the audio and video playback device after performing corresponding processing in accordance with the first playback rule, so as to play the conference screen and conference audio;

[0009] The TV wall processing system is communicatively connected to the main control system, and is used to receive the remote video data sent by the main control system, and send the data to the video playback device according to the second playback rule after corresponding processing, so as to play the TV wall screen.

[0010] In one embodiment, the device comprises:

[0011] An audio acquisition module communicatively connected to the main control system, which is used to acquire local audio data;

[0012] A video acquisition module communicatively connected to the main control system, which is used to acquire local video data.

[0013] In one embodiment, the device further includes:

[0014] A communication module communicatively connected to the main control system, which is used to establish a network connection with remote conference terminals of each branch venue participating in the meeting, so as to obtain remote audio and video data and meeting-related data sent by the remote conference terminals.

[0015] In one embodiment, the conference processing system includes:

[0016] A first video processing module, which is used to perform corresponding processing on remote video data and local video data;

[0017] An audio processing module, which is used to perform corresponding processing on remote audio data and local audio data;

[0018] A data processing module, which is used to process the meeting-related data.

[0019] In one embodiment, the video wall processing system includes:

[0020] A second video processing module, which is used to perform corresponding processing on remote video data.

[0021] In one embodiment, the audio-visual playback device includes a plurality of conference video displays and a voice playback device. The conference processing system is used to communicatively connect with the conference video displays through a video output interface, and send different video data to different conference video displays for display according to user requirements and the type of video data;

[0022] The conference processing system is also used to communicatively connect with the voice playback device through a voice output interface, and is used to perform voice playback through the voice playback device.

[0023] In one embodiment, the video playback device includes a video wall display. The video wall processing system is used to determine a split screen mode according to user requirements or the quantity of the remote video data, and perform split screen display in the playback interface of the video wall display according to the split screen mode.

[0024] In one embodiment, the device includes:

[0025] A noise reduction module communicatively connected to the main control system, which is used to perform noise reduction processing on environmental noise in local audio data and remote language data;

[0026] An echo cancellation module communicatively connected to the main control system, which is used to cancel the echo in the local audio data and the remote language data.

[0027] In one embodiment, the device includes:

[0028] A subtitle conversion module, which is used to convert the local audio data and the remote audio data into subtitles respectively;

[0029] A translation module, which is used to translate the local audio data and the remote audio data into different languages, and convert them into subtitles in the corresponding languages through the subtitle conversion module for display.

[0030] Audio data

[0031] In a second aspect, a conference terminal and video wall integrated system is provided, including:

[0032] The conference terminal and video wall integrated device as described above, where the conference terminal and video wall integrated device is arranged in the main venue and is used for audio-visual interaction and supervision with each branch venue;

[0033] A plurality of conference terminals respectively arranged in each branch venue, which are used to collect the remote audio-visual data of each branch venue and send it to the conference terminal and video wall integrated device.

[0034] In an embodiment of the present invention, a conference terminal and video wall integrated device and system are provided. The device includes: a main control system, which is used to obtain remote audio-visual data, local audio-visual data, and conference-related data; a conference processing system communicatively connected to the main control system, which is used to receive the remote audio-visual data, local audio-visual data, and conference-related data sent by the main control system, and after corresponding processing, send them to the audio-visual playback device according to the first playback rule to play the conference picture and conference audio; a video wall processing system communicatively connected to the main control system, which is used to receive the remote video data sent by the main control system, and after corresponding processing, send it to the video playback device according to the second playback rule to play the video wall picture. In an embodiment of the present application, by implementing the functions of the conference terminal and the video wall in one device at the same time, the user cost is effectively saved, and the installation is simple. The user can choose to turn on the functions of the conference terminal and the video wall at the same time according to needs, or only turn on one of the functions of the conference terminal or the video wall, which can meet the various needs of users and improve the user experience. Description of the Drawings

[0035] Figure 1 It is a schematic structural diagram of a conference terminal and video wall integrated device provided by an embodiment of the present application;

[0036] Figure 2 is a schematic diagram of an interface display state provided by an embodiment of the present application;

[0037] Figure 3 is a schematic structural diagram of an integrated system of a conference terminal and a video wall provided by an embodiment of the present application. Detailed implementation manners

[0038] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0039] See Figure 1 , an embodiment of the present application provides an integrated device of a conference terminal and a video wall, including:

[0040] A main control system, which is used to obtain remote audio and video data, local audio and video data, and conference-related data;

[0041] A conference processing system communicatively connected to the main control system, which is used to receive the remote audio and video data, local audio and video data, and conference-related data sent by the main control system, and after corresponding processing, send them to an audio and video playback device according to a first playback rule to play the conference video and conference audio;

[0042] A video wall processing system communicatively connected to the main control system, which is used to receive the remote video data sent by the main control system, and after corresponding processing, send it to a video playback device according to a second playback rule to play the video wall video.

[0043] In the embodiment of the present application, the main control system may include controllers such as a single-chip microcomputer, a main control chip, and an MCU.

[0044] In the embodiment of the present application, the remote audio and video data may specifically include audio data and video data of each branch venue, that is, remote audio data and remote video data, and may specifically collect the audio and video data of the conference participants in each venue through the conference terminals set in each branch venue.

[0045] In the embodiment of the present application, the local audio and video data may include audio data and video data of the main venue, that is, local audio data and local video data.

[0046] In the embodiment of the present application, the conference-related data may include document data, screen stream data, etc.

[0047] In the embodiment of the present application, after obtaining remote audio and video data, local audio and video data, and conference-related data, the main control module preprocesses the remote audio and video data, local audio and video data, and conference-related data, and can determine the data to be sent to the conference processing system and the video wall processing system according to the type of data and conference requirements, so as to display the conference screen and the video wall screen simultaneously.

[0048] In the embodiment of the present application, the audio acquisition module communicatively connected to the main control system is used to acquire local audio data; the video acquisition module communicatively connected to the main control system is used to acquire local video data. Specifically, the local audio and video data can be acquired through the audio acquisition module and the video acquisition module set locally and sent to the main control system to generate an audio file and a video file.

[0049] Furthermore, the device may further include an audio source identification module for identifying the main audio source in the current environment according to the image of the surrounding environment acquired by the video acquisition module, and the audio acquisition module is further used to acquire the audio signal corresponding to the audio source identified by the audio source identification module. Since there are many participants and the voices of speeches or discussions in the environment are too loud to correctly collect the audio data of the current speaker, the current speaker can be determined through the image acquired by the video acquisition module, and the audio acquisition of other participants can be turned off or reduced, and the audio acquisition can be performed through the audio acquisition module corresponding to the current speaker.

[0050] Among them, the audio acquisition module can be a wireless microphone, a wired microphone, a gooseneck microphone, etc. The audio acquisition module can include multiple ones, which are respectively arranged around different participants to acquire the audio data of different participants.

[0051] In the embodiment of the present application, the device includes:

[0052] A noise reduction module communicatively connected to the main control system, which is used to perform noise reduction processing on the environmental noise in the local audio data and the remote audio data to remove the interference of the environmental sound on the audio data.

[0053] In the embodiment of the present application, the device may further include: an echo cancellation module communicatively connected to the main control system, which is used to cancel the echo in the local audio data and the remote audio data. To improve the fidelity of the sound.

[0054] In an embodiment of the present application, the video acquisition module may include one or more. For example, when there is one video acquisition module, the rotation function of the video acquisition module can be used to take pictures of the participants and various positions of the meeting venue through omnidirectional rotation. It can also identify the shooting direction according to the audio source. For example, determine the position that needs to be shot currently according to the voices of different participants, and rotate the video acquisition device to obtain the corresponding picture.

[0055] Among them, the video acquisition module can be a camera.

[0056] In an embodiment of the present application, the device further includes a communication module communicatively connected to the main control system, which is used to establish a network connection with the remote conference terminals of each branch venue participating in the meeting, so as to obtain the remote audio and video data and conference-related data sent by the remote conference terminals, so as to realize real-time two-way audio and video interaction.

[0057] Among them, the communication module can be a 4G, 5G, WIFI and other communication modules.

[0058] In an embodiment of the present application, the conference processing system includes: a first video processing module, which is used to perform corresponding processing on the remote video data and local video data; an audio processing module, which is used to perform corresponding processing on the remote audio data and local audio data; a data processing module, which is used to process the conference-related data.

[0059] Specifically, the audio processing module is used to first decode the data through the audio decoding sub-module after receiving the audio data, and then put the decoded audio data of each channel into its own data buffer; the audio synthesis sub-module takes out the packets with the earliest time from each buffer queue, mixes these audio packets, and then passes the mixed data to the voice playback device for playback.

[0060] The first video processing module is used to put each channel of video data into its own data buffer after receiving the remote video data and / or local video data; the video rendering sub-module takes out the packets from each buffer queue in sequence, and the video decoding sub-module decodes the data, and then according to the currently preset split-screen mode, renders the decoded video data of each channel to the video display area of the corresponding conference video display.

[0061] The data processing module is used to put the data packets into the data buffer queue when the relevant data format received is in the video stream format; the data stream rendering sub-module takes out the data packets from the buffer queue in sequence, and the video decoding sub-module decodes the data, and then renders the decoded data stream to the data display area. If the relevant data format received is in the picture format, the data stream rendering sub-module can directly render the picture to the data display area of the corresponding conference video display.

[0062] In the embodiment of the present application, the video wall processing system includes: a second video processing module for performing corresponding processing on the remote video data. Specifically, the second video processing module is used to put each video data into its own data buffer after receiving the remote video data; the video rendering sub-module takes out the data packets from each buffer queue in sequence, and the video decoding sub-module decodes the data, and then according to the currently set multi-screen mode, renders each decoded video data to the video display area of the corresponding video wall screen.

[0063] In the embodiment of the present application, the audio-visual playback device includes a plurality of conference video displays and a voice playback device. The conference processing system is used to communicate with the conference video displays through a video output interface and send different video data to different conference video displays according to the first playback rule for display.

[0064] The conference processing system is further used to communicate with the voice playback device through a voice output interface and perform voice playback through the voice playback device.

[0065] Specifically, the conference processing system can project different video output screens to multiple displays for playback according to different user requirements, conference project requirements, video types and quantities. For example: one video output displays the document or screen stream data picture, and the other video output displays the video pictures of each venue; or both video outputs display the video pictures of different venues. To present a data picture + interactive video picture, or a main venue picture + interactive video picture on multiple conference displays,

[0066] Among them, the interactive video picture can also be presented according to different multi-screen modes, such as single-screen, four-screen, 16-screen, etc.

[0067] Among them, the voice output interface can include various interface forms such as SDI, HDMI, DVI, CVBS, YPbPr, etc.

[0068] Among them, the video output interface can include various interface forms such as HDMI, DVI, VGA, RGB, S terminal, etc.

[0069] In an embodiment of the present application, the video playback device includes a TV wall display, and the TV wall processing system is used to determine the split-screen mode according to user needs or the amount of the remote video data, and perform split-screen display in the playback interface of the TV wall display according to the split-screen mode.

[0070] Specifically, the TV wall subsystem can deliver the TV wall image to multiple monitors for playback in the form of one or more video outputs according to different user needs and project requirements. In addition, the monitor can also present the TV wall video image in different split-screen modes, such as 16-split screen, 25-split screen, 36-split screen, 64-split screen, etc.

[0071] Furthermore, the number of split screens in the video wall processing system is greater than that in the conference system.

[0072] In an embodiment of the present application, the TV wall display may include multiple ones.

[0073] In an embodiment of the present application, the split-screen mode can be switched in real time according to the number of participating user terminals. For example, in some scenarios, users in some branch venues may have network delays or other situations, resulting in gradual joining of the meeting after the meeting starts. At this time, when the split-screen interface of the TV wall cannot meet the needs of presenting videos from all branch venues, the split-screen module can be replaced. When the split-screen mode has reached the maximum, that is, the number of split screens has reached the limit, at this time, another TV wall display can be started to display the TV wall interface.

[0074] In an embodiment of the present application, the remote video data may also include a terminal identification. Each split-screen interface in the TV wall may establish a mapping relationship with the identification of different participating terminals. When the corresponding video data is obtained, it may be displayed through the corresponding split-screen interface.

[0075] In the implementation of this application, the video data displayed on each split-screen interface in the TV wall can be determined according to the order in which different participating user terminals join the meeting. The video data of the user terminal that joins first is displayed on the first split-screen interface, the video data of the user terminal that joins second is displayed on the second split-screen interface, and so on, until all are displayed.

[0076] In an embodiment of the present application, the integrated conference terminal and TV wall device may also include: a subtitle conversion module, which is used to convert local audio data and remote audio data into subtitles, and output them corresponding to the corresponding video screen, so that when the environment is noisy and the user's voice cannot be heard clearly, the current meeting content can be understood by watching the subtitles.

[0077] Further, the subtitles may include subtitles in multiple languages, such as Chinese, English, etc. The device may further include a translation module for translating the local audio data and the remote audio data into different languages, that is, translating the voices output by the users participating in the meeting into different voices, and forming subtitles corresponding to different languages through the subtitle module to be synchronously displayed in different video display interfaces, so as to achieve communication in multiple languages and improve the interaction fluency.

[0078] In an embodiment of the present application, the device further includes a video recording module for recording and storing the remote audio-visual data and the local audio-visual data. When it is necessary to organize the meeting and make other references, it can be directly called for meeting backtracking.

[0079] In an embodiment of the present invention, a meeting terminal and video wall integrated device is provided, including: a main control system for obtaining remote audio-visual data, local audio-visual data, and meeting-related data; a meeting processing system communicatively connected to the main control system for receiving the remote audio-visual data, local audio-visual data, and meeting-related data sent by the main control system, and after corresponding processing, sending them to the audio-visual playback device according to the first playback rule to play the meeting picture and meeting audio; a video wall processing system communicatively connected to the main control system for receiving the remote video data sent by the main control system, and after corresponding processing, sending it to the video playback device according to the second playback rule to play the video wall picture. In an embodiment of the present application, by implementing the functions of the meeting terminal and the video wall in one device, the user cost is effectively saved, and the installation is simple. The user can choose to turn on the functions of the meeting terminal and the video wall at the same time according to needs, or only turn on one of the functions of the meeting terminal or the video wall, which can meet the various needs of the user and improve the user experience.

[0080] In an embodiment of the present application, a meeting terminal and video wall integrated system is further provided. The system includes: the above-mentioned meeting terminal and video wall integrated device, which is set in the main venue and used for audio-visual interaction and supervision with each branch venue; a plurality of meeting terminals respectively set in each branch venue, which are used to collect the remote audio-visual data of each branch venue and send it to the meeting terminal and video wall integrated device for display. Video interaction between the main venue and each branch venue can be realized, and the interactive video picture can be displayed.

[0081] Among them, the meeting terminal may include an audio collection unit and a video collection unit for collecting the audio-visual data of different branch venues. The meeting terminal and video wall integrated device can collect the audio-visual data of the main venue and send it to the meeting terminal, so as to realize two-way audio-visual interaction.

[0082] In the embodiment of the present application, the system further includes a control device, which is used to perform processing such as switching, zooming, and deleting on the played picture, and can start and shut down the device. For example, when a person in a certain branch venue is reporting work, the video picture of that branch venue can be enlarged in the conference display interface or moved to the center position for display so that it can be viewed emphatically.

[0083] Among them, the control device can be a touch device, such as a mobile phone, a tablet computer, etc., and controls the conference terminal and the integrated device through the touch commands input by the user.

[0084] In the present application, by implementing the functions of the conference terminal and the video wall in one system at the same time, the user cost is effectively saved, and the installation is simple. The user can choose to turn on the functions of the conference terminal and the video wall at the same time according to needs, or can only turn on one of the functions of the conference terminal or the video wall, which can meet various needs of the user and improve the user experience.

[0085] The above are only the preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An integrated device of a conference terminal and a video wall, characterized in that, The device comprises: A main control system, which is used to obtain remote audio and video data, local audio and video data, and conference-related data; A conference processing system in communication with the main control system, configured to receive remote audio and video data, local audio and video data, and conference-related data sent by the main control system, and perform corresponding processing and send the data to the audio and video playback device in accordance with the first playback rule to play the conference screen and conference audio; a TV wall processing system in communication with the main control system, configured to receive remote video data sent by the main control system, perform corresponding processing, and then send the data to the video playback device according to the second playback rule to play the TV wall image; An audio acquisition module in communication with the main control system, for acquiring local audio data; A video acquisition module in communication with the main control system, for acquiring local video data; an audio source recognition module, configured to identify the main audio source of the current environment based on the image of the surrounding environment captured by the video capture module, and the audio capture module is further configured to capture an audio signal corresponding to the audio source identified by the audio source recognition module, and to rotate the video capture module based on the audio source identified by the audio source to determine a shooting direction; The audio and video playback device includes multiple conference video displays and voice playback devices. The conference processing system includes: an audio processing module for decoding the audio data through an audio decoding submodule after receiving the audio data, and then placing the decoded audio data of each channel into their respective data buffers; an audio synthesis submodule for taking out the earliest data packets from each buffer queue, mixing these audio data packets, and then passing the mixed data to the voice playback device for playback; a first video processing module for placing each channel of video data into their respective data buffers after receiving remote video data and / or local video data; a video rendering submodule for taking out the oldest data packets from each buffer queue in order of priority; The data packets are taken out sequentially, and the video decoding submodule decodes the data. Then, according to the currently preset split-screen mode, the decoded video data of each channel are rendered to the video display area of the corresponding conference video display; the data processing module is used to put the data packet into the data buffer queue first when the relevant data format received is the video stream format; the data stream rendering submodule takes out the data packet from the buffer queue in sequence, and the video decoding submodule decodes the data, and then renders the decoded data stream to the data display area; if the relevant data format received is the image format, the data stream rendering submodule can directly render the image to the data display area of the corresponding conference video display; The TV wall processing system includes: a second video processing module, which is configured to put each video data into its respective data buffer after receiving remote video data; a video rendering sub-module takes out data packets from each buffer queue in sequence, and a video decoding sub-module decodes the data, and then according to the currently set multi-screen mode, renders the decoded video data of each channel to the corresponding video display area of the TV wall screen. The video playback device includes a TV wall display. The TV wall processing system is configured to determine the multi-screen mode according to user requirements or the quantity of the remote video data, and perform multi-screen display in the playback interface of the TV wall display according to the multi-screen mode, and switch in real time according to the number of participating user terminals. When the multi-screen mode has reached the maximum, another TV wall display is activated to display the TV wall interface; the video data displayed on each multi-screen interface in the TV wall can be determined according to the sequence of different participating user terminals joining the meeting. The video data of the user terminal that joins first is displayed on the first multi-screen interface, and the video data of the second user terminal that joins is displayed on the second multi-screen interface, and so on until all are displayed.

2. The integrated device of a conference terminal and a video wall according to claim 1, characterized in that, The device further includes: A communication module communicatively connected to the main control system, which is configured to establish a network connection with remote conference terminals of each branch venue participating in the meeting to obtain the remote audio and video data and meeting-related data sent by the remote conference terminals.

3. The integrated device of a conference terminal and a video wall according to claim 2, wherein The meeting processing system is configured to communicatively connect with the meeting video display through a video output interface, and send different video data to different meeting video displays for display according to user requirements and the type of video data. The meeting processing system is further configured to communicatively connect with the voice playback device through a voice output interface, and is used for voice playback through the voice playback device.

4. The integrated device of a conference terminal and a video wall according to claim 1, characterized in that The device includes: A noise reduction module communicatively connected to the main control system, which is configured to perform noise reduction processing on environmental noise in the remote audio data and local audio data. An echo cancellation module communicatively connected to the main control system, which is a module configured to cancel echoes in the local audio data and remote audio data.

5. The integrated device of a conference terminal and a video wall according to claim 1, characterized in that, The device includes: A subtitle conversion module, which is configured to convert the local audio data and remote audio data into subtitles respectively. A translation module, which is configured to translate the local audio data and remote audio data into different languages, and convert them into subtitles in the corresponding languages through the subtitle conversion module for display.

6. A conference terminal and video wall integrated system, characterized in that, The system includes: The conference terminal and TV wall integrated device according to any one of claims 1-5. The conference terminal and TV wall integrated device is arranged in the main venue and is used for audio and video interaction and supervision with each branch venue. A plurality of conference terminals are respectively arranged in each branch venue, and are used for collecting remote audio and video data of each branch venue and sending them to the conference terminal and TV wall integrated device.

Images