Multi-meeting-place and multi-screen switching intelligent meeting system thereof

By employing a hierarchical management architecture and multi-screen switching technology, the problem of low communication efficiency among multiple meeting rooms in traditional conferencing systems has been solved. This enables the interoperability of audio and video data across multiple meeting rooms and cross-room interaction, thereby improving meeting efficiency and user experience.

CN119052402APending Publication Date: 2024-11-29JIANGSU LONGCHUANG INFORMATION TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411092794.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Traditional conference systems cannot efficiently schedule meetings in multiple venues simultaneously, and horizontal communication between venues is difficult, resulting in low meeting efficiency.

Method used

It adopts a hierarchical management architecture, including an MCU server, a multi-functional video conferencing terminal, and video acquisition, transmission, processing, and display modules. Combined with domain management, multi-screen switching, and meeting room matching modules, it enables audio and video data interoperability across multiple meeting rooms and cross-room interaction.

Benefits of technology

It enhances the interactivity and flexibility of multi-venue meetings, allowing users to join multiple rooms simultaneously, enabling synchronized and secure transmission of audio and video streams, and improving meeting efficiency and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119052402A_ABST
    Figure CN119052402A_ABST
Patent Text Reader

Abstract

The invention relates to the related technical field of smart conference systems, and discloses a multi-conference-place and multi-screen switching smart conference system, which can accommodate a plurality of sub-conference-places for conferences at the same time through a hierarchical management architecture, isolate messages and callbacks of a plurality of rooms through a multi-room management function, realize cross-room microphone connection and other interactive scenes, and improve the user experience. According to the invention, the user can join a plurality of rooms at the same time and receive audio and video streams from the rooms, so that the interactivity and flexibility between conference rooms are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of smart conference technology, and in particular to a smart conference system with multiple conference venues and multiple screens switching therebetween. Background Art

[0002] With the development of science and technology and the increasing demand of enterprises for efficient conference management, traditional conference methods can no longer meet the needs of modern enterprises, especially in large-scale academic conferences, corporate annual meetings and other occasions, which often need to be held in multiple venues at the same time, and need to achieve real-time synchronous transmission and playback of multimedia content such as slides and videos. In addition, when watching live broadcasts in multiple venues, the audience hopes to be able to switch freely to the images of different venues so as not to miss any exciting content.

[0003] Traditional conference systems, such as the conference system with publication number CN111108491B, are difficult to efficiently arrange meetings in multiple venues. During a traditional meeting, only one person can speak at the same time, and it is difficult for separate communication to be conducted horizontally between sub-venues, resulting in low meeting efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide a multi-venue and multi-screen switching intelligent conference system to solve the problem raised by the above background technology that during the current traditional conference, only one person can speak at the same time, and it is difficult to communicate horizontally between branch venues, resulting in low meeting efficiency.

[0005] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a multi-venue and multi-screen switching intelligent conference system, which adopts a hierarchical management architecture, with MCU servers and multi-functional video conferencing terminals built at the first level, and an additional set of spare MCU servers to ensure stable, reliable and uninterrupted conferences;

[0006] The second-level MCU server is connected to the first-level MCU server to achieve mutual transmission of audio and video data;

[0007] The third-level architecture builds video conferencing terminals in the bottom-level branch venues to ensure that audio and video data of video conferences can be communicated at all levels;

[0008] Each video conferencing terminal is connected to a camera, microphone, display, speaker, and headphones;

[0009] The system includes video acquisition and transmission module, audio acquisition and transmission module, signal processing and synchronization module, multi-screen display and control module, intelligent switching module, venue matching and sub-venue management module;

[0010] The venue matching and sub-venue management modules are used to regulate the connection between sub-venues and between the main venue and some sub-venues. Participants can select one or more connection objects to interconnect to form small meetings, and multiple small meetings can be carried out simultaneously under the control of the main venue.

[0011] Furthermore, the system utilizes multi-room management permissions to isolate messages and callbacks in multiple rooms, and implements cross-room microphone connections and other interactive scenarios, so that users can join multiple rooms at the same time and receive audio and video streams from these rooms.

[0012] Furthermore, the system uses domain management technology to set each audio and video as an independent domain, preventing confidential or sensitive information from being accessed elsewhere on the network. It also manages temporary connections for one-off events and integrates them into the permanent system, ensuring that audio is only transmitted to the intended destination.

[0013] Furthermore, when collecting video information, the video acquisition and transmission module adds an additional auxiliary video stream based on the main video stream. The main video stream uses the video information collected by the camera, and the auxiliary video stream uses the video information recorded by the branch venue display.

[0014] Furthermore, when the audio collection and transmission module is far away from the venue and long-distance transmission is required, an audio optical terminal is used for transmission.

[0015] Furthermore, the signal processing and synchronization module is used to match the video information and the audio information so that the video content and the audio content are displayed synchronously, and introduces a timestamp when encoding the video information and the audio information, and performs time matching of the video information and the audio information through the timestamp.

[0016] Furthermore, the multi-screen display and control module is used to regulate the synchronous display of multiple screens, and uses multi-screen display technology to enable one motherboard to connect to multiple display screens. Each screen can play different content at the same time and be touch-controlled separately, and the operations do not affect each other, which facilitates efficient management under multi-display settings.

[0017] Furthermore, the intelligent switching module is used to regulate the display of video information in multiple venues, and to identify the branch venue where a speaker is speaking based on actions and language, and to amplify and emphasize the display of the video information of the branch venue.

[0018] Furthermore, the venue matching and sub-venue management module is used to regulate the connection between sub-venues and between the main venue and some sub-venues to meet the individual communication needs between sub-venues and between sub-venues and the main venue in a multi-venue hierarchical conference.

[0019] Furthermore, the venue matching and sub-venue management module sets up a basic database, provides user authority and account management functions, the main venue has a fixed public IP address, and the sub-venue is connected to the main venue through the SRT protocol to achieve real-time audio and video interaction between the main and sub-venues.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention discloses a multi-venue and multi-screen switching intelligent conference system, which can accommodate multiple branch venues for simultaneous conferences through a hierarchical management architecture, and isolate messages and callbacks from multiple rooms through a multi-room management function, thereby realizing cross-room microphone connection and other interactive scenarios, so that users can join multiple rooms at the same time and receive audio and video streams from these rooms, thereby improving the interactivity and flexibility between conference rooms. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a schematic diagram of the decomposition management architecture of the present invention;

[0023] Figure 2 It is a schematic diagram of the hardware configuration of the present invention;

[0024] Figure 3 It is a schematic diagram of the system module of the present invention;

[0025] Figure 4 A schematic diagram of a video stream of the present invention;

[0026] Figure 5 It is a schematic diagram of the audio and video playing process of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings.

[0029] Combination Figure 1 - Figure 4 , a smart conference system with multiple venues and multi-screen switching, adopts a hierarchical management architecture. The first level builds MCU servers and multi-functional video conferencing terminals, and is also equipped with an additional set of backup MCU servers to ensure stable, reliable and uninterrupted meetings.

[0030] The second-level MCU server is connected to the first-level MCU server to realize the mutual transmission of audio and video data.

[0031] The third-level architecture constructs video conferencing terminals in the underlying sub-venues to meet the intercommunication requirements of audio and video data of video conferencing at all levels.

[0032] Each video conferencing terminal is connected to a camera, microphone, display, speaker and headphones. The video conferencing terminal encodes and transmits local audio and video data, and decodes the audio and video signals forwarded by the MCU for display and playback.

[0033] The system uses the multi-room management function to isolate messages and callbacks in multiple rooms, realize cross-room microphone connections and other interactive scenarios, and allow users to join multiple rooms at the same time and receive audio and video streams from these rooms, thereby improving the interactivity and flexibility between conference rooms.

[0034] The system uses domain management technology to ensure security between audio systems in each room. By setting each audio and video as an independent domain, confidential or sensitive information can be prevented from being accessed elsewhere on the network. In addition, temporary connections for one-off events can be managed and integrated into permanent systems, ensuring that audio is only transmitted to the intended destination.

[0035] The functional modules of the smart conference system include video acquisition and transmission module, audio acquisition and transmission module, signal processing and synchronization module, multi-screen display and control module, intelligent switching module, venue matching and sub-venue management module.

[0036] Among them, the video acquisition and transmission module is used to collect video information data, and compress the collected signal through the codec to reduce bandwidth occupancy and improve transmission efficiency. The codec adopts one or more of VP8, VP9, ​​H.264 and H.265. After the video data is encoded, it is transmitted to other venues or remote users through the network. It is necessary to ensure that the network connection is stable and the delay is low to ensure the smoothness of video playback. The receiving device needs to have a corresponding decoder to decode the received video data and play it. Modern display systems, such as LCD or LED large screens, can be used to display remote videos and auxiliary information.

[0037] For auxiliary video streams (such as the speaker's computer screen recording), dual-stream technology can be used to achieve this, that is, on the basis of the main video stream, an auxiliary video stream is added to display more information and data sources.

[0038] The audio acquisition and transmission module is similar to the video acquisition and transmission module. It collects the speech content of the conference speaker through the microphone of the video conferencing terminal, and converts the analog audio signal into a digital signal through a preamplifier and an analog-to-digital converter. It adopts Dante digital audio transmission technology and is based on a three-layer IP network. The signal transmission delay is low, the accuracy is high, and the cost is low.

[0039] When the venue is far away and long-distance transmission is required, the use of audio optical transceiver can achieve uncompressed and zero-delay transmission of digital audio signals, which has the advantages of high fidelity and long-distance transmission.

[0040] The signal processing and synchronization module is used to match video information and audio information so that the video content and audio content are displayed synchronously. The specific matching is done by using the timestamps (PTS and DTS) of the audio and video to synchronize. After the media data goes through the demultiplexing process, the audio / video decoding is independent and can be played independently. In the audio stream and video stream, the playback speed is specified by relevant information:

[0041] Video: Frame rate, which indicates the number of frames displayed in one second of video.

[0042] Audio: Sampling rate, which indicates the number of samples played per second.

[0043] From the frame rate and sampling rate, you can know the video / audio playback speed. Both the sound card and the graphics card use one frame of data as the playback unit. If you rely solely on the frame rate and sampling rate for playback, under ideal conditions, it should be synchronized without deviation.

[0044] Take a 44.1KHz AAC audio stream and a 24FPS video stream as an example: an AAC audio frame contains 1024 sampling points per channel, so the playback duration of a frame is: (1024 / 44100)×1000ms=23.22ms;

[0045] The duration of a video frame is: 1000ms / 24=41.67ms. Ideally, the audio and video are completely synchronized. The audio and video playback process is as follows: Figure 5As shown, in actual situations, it is difficult to accurately control the playback time of a frame. The different time consumption of audio and video decoding and rendering may cause a slight difference in the output of each frame. Over time, the asynchrony becomes more and more obvious. At the same time, the audio output is linear, while the video output may be nonlinear, which can also cause deviations. In addition, there is a gap between the audio and video in the media stream itself, which can cause the audio and video to be out of sync. Therefore, we introduce timestamps in audio and video. First, select a reference clock. When encoding, timestamp each audio and video data block based on the reference clock. When playing, adjust the playback based on the audio and video timestamps and the reference clock. This ensures that the audio and video are synchronized during playback.

[0046] The signal processing and synchronization module also processes the audio information, and simulates the sense of position of the sound in space by controlling the volume and delay of the left and right channels. For example, the volume of a certain channel can be increased or decreased to simulate the effect of sound coming from the left or right side, and the delay of the two channels can be adjusted at the same time to make the sound have a certain diffusion effect in space. It can also perform different effects on the signals of the left and right channels respectively, so that the signals of the two channels have slight differences, such as volume, frequency, pitch and delay, thereby simulating the sound position similar to that in the real world. In this way, the signal processing and synchronization of video information and audio information can be realized, and the volume and timing differences of the left and right channels can be simulated to distinguish different venues and identify the speaker's position. This not only improves the quality of multimedia playback, but also enhances the user's immersive experience.

[0047] The multi-screen display and control module is used to control the synchronous display of multiple screens. It uses multi-screen display technology to connect one motherboard to multiple display screens. Each screen can play different content at the same time and can be touch-controlled separately. The operations do not affect each other, which facilitates efficient management under multi-display settings.

[0048] The intelligent switching module is used to control the display of video information in multiple venues. It can identify the branch venue where the speaker is speaking based on the action and language, and enlarge the video information of the venue for emphasis. Specifically, the action recognition technology is used to monitor and analyze the actions of humans or objects through cameras or other devices. These technologies can recognize and understand human action patterns, and cover human action recognition in videos based on deep learning human action recognition algorithms. Combined with the pre-trained CLIP multimodal large model, the action intentions of the participants can be further accurately identified. At the same time, the real-time speech recognition technology is used in combination with the deep learning speech recognition model to identify the speeches of the participants. In this way, it is possible to accurately identify which venue is speaking, and enlarge the venue where the speaker is speaking for emphasis.

[0049] The venue matching and sub-venue management module is used to regulate the connection between each sub-venue and between the main venue and some sub-venues, so as to meet the needs of individual communication between each sub-venue and between the sub-venue and the main venue in multi-venue hierarchical meetings. Due to the particularity of multi-venue hierarchical meetings, there will be individual communication needs between sub-venues and between sub-venues and the main venue. However, in traditional meetings, it is difficult to meet the needs of multiple parties to communicate at the same time. Therefore, through the venue matching and sub-venue management module, each sub-venue can independently select one or more connection objects for interconnection. In this way, multiple separate small meetings are divided under an overall meeting. The members of the small meetings can be adjusted in real time, which is fluid, so that discussions on different matters can be carried out at the same time, saving a lot of time.

[0050] To achieve the above functions, you must first set up the basic database, provide user permissions and account management functions, the main venue has a fixed public IP address, and the branch venue is connected to the main venue through the SRT protocol to achieve real-time audio and video interaction between the main and branch venues and ensure low-latency transmission. Through the venue negotiation system console, participants can self-select one or more connection objects to interconnect to form a small conference. After joining the venue, they will automatically watch multiple screens; when other venues join the meeting, they will automatically join the multiple screens viewed on this end.

[0051] The members of a small meeting can be adjusted in real time and are fluid, allowing discussions on different matters to proceed simultaneously, saving a great deal of time. The connection objects are flexible. Through the venue negotiation system console, participants can self-select one or more connection objects for interconnection, making communication convenient and flexible.

[0052] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-venue and multi-screen switching intelligent conference system, characterized by: A hierarchical management structure is adopted. The first level is equipped with MCU servers and multi-functional video conferencing terminals. An additional set of spare MCU servers is also provided to ensure stable, reliable and uninterrupted meetings. The second-level MCU server is connected to the first-level MCU server to achieve mutual transmission of audio and video data; The third-level architecture builds video conferencing terminals in the bottom-level branch venues to ensure that audio and video data of video conferences can be communicated at all levels; Each video conferencing terminal is connected to a camera, microphone, display, speaker, and headphones; The system includes video acquisition and transmission module, audio acquisition and transmission module, signal processing and synchronization module, multi-screen display and control module, intelligent switching module, venue matching and sub-venue management module; The venue matching and sub-venue management modules are used to regulate the connection between sub-venues and between the main venue and some sub-venues. Participants can select one or more connection objects to interconnect to form small meetings, and multiple small meetings can be carried out simultaneously under the control of the main venue.

2. According to claim 1, a multi-venue and multi-screen switching intelligent conference system is characterized by: The system utilizes multi-room management permissions to isolate messages and callbacks from multiple rooms, implement cross-room microphone calls and other interactive scenarios, and allow users to join multiple rooms at the same time and receive audio and video streams from these rooms.

3. According to claim 1, a multi-venue and multi-screen switching intelligent conference system is characterized in that: The system uses domain management technology to set each audio and video as an independent domain, preventing confidential or sensitive information from being accessed elsewhere on the network. It also manages temporary connections for one-off events and integrates them into the permanent system, ensuring that audio is only transmitted to the intended destination.

4. According to claim 1, a multi-venue and multi-screen switching intelligent conference system is characterized in that: When collecting video information, the video acquisition and transmission module adds an additional auxiliary video stream based on the main video stream. The main video stream uses the video information collected by the camera, and the auxiliary video stream uses the video information recorded by the branch venue display.

5. According to claim 1, a multi-venue and multi-screen switching intelligent conference system is characterized by: The audio collection and transmission module uses an audio optical terminal for transmission when the venue is far away and long-distance transmission is required.

6. The multi-venue and multi-screen switching intelligent conference system according to claim 1 is characterized in that: The signal processing and synchronization module is used to match the video information and the audio information so that the video content and the audio content are displayed synchronously, and introduces a timestamp when encoding the video information and the audio information, and performs time sequence matching of the video information and the audio information through the timestamp.

7. The intelligent conference system with multiple conference rooms and multiple screens according to claim 1 is characterized in that: The multi-screen display and control module is used to regulate the synchronous display of multiple screens, and uses multi-screen different display technology to enable one motherboard to connect to multiple display screens. Each screen can play different content at the same time and be touch-controlled separately, and the operations do not affect each other, which facilitates efficient management under multi-display settings.

8. The multi-venue and multi-screen switching intelligent conference system according to claim 1 is characterized by: The intelligent switching module is used to regulate the display of video information in multiple venues, and to identify the branch venue where a speaker is speaking based on actions and language, and to amplify the video information of the venue and display it emphatically.

9. The multi-venue and multi-screen switching intelligent conference system according to claim 1 is characterized in that: The venue matching and sub-venue management module is used to regulate the connection between sub-venues and between the main venue and some sub-venues to meet the individual communication needs between sub-venues and between sub-venues and the main venue in a multi-venue hierarchical conference.

10. The intelligent conference system with multiple conference rooms and multiple screens according to claim 9, characterized in that: The venue matching and sub-venue management module sets up a basic database, provides user permissions and account management functions, the main venue has a fixed public IP address, and the sub-venue is connected to the main venue through the SRT protocol to achieve real-time audio and video interaction between the main and sub-venues.

Citation Information

Patent Citations

  • Method and device for realizing private conversation in multi-point meeting

    CN101022481A

  • Method and system for establishing video conference

    CN101388997A

  • Video conference creating method, video conference system, storage medium and electronic device

    CN110213526A

  • Video conference system control method, multi-point control unit and storage medium

    CN113824919A

  • Audio and video synchronization method and device in cloud conference and electronic equipment

    CN116261000A