A video conference system and a signal processing method and device thereof

Abstract

The present disclosure provides a video conference system and a signal processing method, device and medium thereof, wherein the system comprises a multipoint conference control unit (MCU), the MCU is connected with at least one primary built-in MCU conference terminal, and the primary built-in MCU conference terminal is connected with at least one secondary built-in MCU conference terminal; the backbone network adopts an E1 private line transmission mode, and the local area network adopts an IP network transmission mode; the primary built-in MCU conference terminal and the secondary built-in MCU conference terminal are provided with digital signal processing units to complete call task access and processing of audio and video, information and signaling. The built-in MCU conference terminal can maximize the call of conference terminals in various network forms, the built-in MCU conference terminal call supports the joining of multiple built-in MCU conference terminals in the conference, a scattering network topology structure is formed, and more conference terminals can be accessed to realize video conference.

Description

Technical Field

[0001] This invention relates to the field of video conferencing technology, and in particular to a video conferencing system and its signal processing method and equipment. Background Technology

[0002] Video conferencing is a real-time, two-way or multi-party video communication conference held through video conferencing terminals and video conferencing multipoint control units (MCUs). Through video conferencing, people can easily conduct remote meetings, training and teaching in multiple locations.

[0003] Given the security, stability, and reliability of E1 leased line transmission, important leased line users such as the military, government and enterprises, and telecom operators typically use E1 leased line network mode to convene high-level multi-party audio and video conferences, ensuring the security and reliability of audio and video data. However, due to the limitations of E1 leased line network resources, most leased line users currently use E1 leased line transmission mode for their backbone network and IP network transmission mode within their local area network. Users must rely on multi-level video conferencing multipoint control units (MCUs) to convene cascaded conferences to achieve network integration and expand the conference scope. For small and medium-sized conference sites, using multi-level MCU cascading to convene conferences increases the complexity of convening conferences while also increasing user equipment procurement, maintenance, and manpower costs.

[0004] The hardware interface, transmission bandwidth, and encoding / decoding resources of the E1 leased line video conferencing terminal cannot support the access of multiple E1 leased line conferencing terminals to realize multi-point conferencing with built-in MCUs. Multi-point conferencing with built-in MCUs in conferencing terminals is limited to IP network networking mode, and the working mode of the built-in MCU in the terminal is singular.

[0005] Most manufacturers use a unified multi-screen approach for their built-in MCU encoding multi-screen, but audio and video loop problems prevent cascading and networking with multi-point control units (MCUs) and built-in MCU conference terminals. Summary of the Invention

[0006] To overcome the problems existing in related technologies, this disclosure provides a video conferencing system and its signal processing method and device to solve the technical problem that audio and video loop problems prevent the system from being cascaded and networked with the multi-point control unit (MCU) and the built-in MCU conferencing terminal.

[0007] This specification provides one or more embodiments of a video conferencing system, including: The multipoint conferencing control unit (MCU) is connected to at least one primary built-in MCU conferencing terminal, and the primary built-in MCU conferencing terminal is connected to at least one secondary built-in MCU conferencing terminal. The backbone network between the multipoint conferencing control unit (MCU) and the primary built-in MCU conferencing terminal, or between the primary built-in MCU conferencing terminal and the secondary built-in MCU conferencing terminal, adopts E1 leased line transmission mode, and the local area network adopts IP network transmission mode. The digital signal processing units of the primary built-in MCU conference terminal and the secondary built-in MCU conference terminal complete the call task access and the processing of audio, video, information and signaling.

[0008] This specification provides one or more embodiments of a video conferencing signal processing method implemented according to the video conferencing system described above. The method is executed in a built-in MCU conferencing terminal and includes the following steps: The built-in MCU conference terminal receives call tasks, determines the call task nodes according to the call order, and sorts them. Multiple independent media processing channels are created based on the call bandwidth of the built-in MCU conference terminal. The role information of each calling terminal is determined based on the received call task. Each calling terminal is assigned and bound to an idle media processing channel. The media processing channel is marked with the corresponding role information, and each media processing channel rejects the bit stream sent from the calling terminal with the same marked role information. Each media processing channel will mix and synthesize the received bitstream and send the resulting data to the call terminal it is bound to.

[0009] This specification provides one or more embodiments of a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a video conferencing signal processing method as described above.

[0010] This disclosure discloses a built-in MCU video conferencing terminal that connects to an E1 leased-line conferencing terminal or an E1 leased-line multipoint conferencing control unit MCU. It also supports the connection of multiple IP video conferencing terminals and built-in MCU conferencing terminals, enabling the built-in MCU conferencing terminal to convene conferencing terminals of various network types to the maximum extent. Furthermore, the built-in MCU conferencing terminal supports multiple built-in MCU conferencing terminals to join the conference, forming a radiating network topology structure, which can connect more conferencing terminals to realize video conferencing. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A schematic diagram of the topology of a video conferencing system provided for one or more embodiments of this specification; Figure 2 A schematic diagram of a specific video conferencing system topology provided for one or more embodiments of this specification; Figure 3 A schematic block diagram of the built-in MCU conference terminal structure provided in one or more embodiments of this specification; Figure 4 A flowchart of a video conferencing signal processing method provided for one or more embodiments of this specification; Figure 5 A flowchart illustrating the execution of step S130 in a video conferencing signal processing method provided in one or more embodiments of this specification; Figure 6 This is a schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this invention.

[0014] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0015] System Implementation Examples According to embodiments of the present invention, a video conferencing system is provided, such as... Figure 1 The diagram shown is a schematic representation of the topology of a video conferencing system provided in this embodiment. A video conferencing system according to an embodiment of the present invention includes: The multipoint conferencing control unit (MCU) 1 is connected to at least one primary built-in MCU conferencing terminal 2, and the primary built-in MCU conferencing terminal 2 is connected to at least one secondary built-in MCU conferencing terminal 3. The backbone network between the multipoint conferencing control unit (MCU) 1 and the primary built-in MCU conferencing terminal 2, or between the primary built-in MCU conferencing terminal 2 and the secondary built-in MCU conferencing terminal 3, adopts E1 leased line transmission mode, and the local area network adopts IP network transmission mode.

[0016] The digital signal processing units of the primary built-in MCU conference terminal 2 and the secondary built-in MCU conference terminal 3 complete the call task access and the processing of audio, video, information and signaling.

[0017] In this embodiment, the built-in MCU video conferencing terminal connects to an E1 leased-line conferencing terminal or an E1 leased-line multipoint conferencing control unit MCU. It also supports the connection of multiple IP video conferencing terminals and the built-in MCU conferencing terminal, enabling the built-in MCU conferencing terminal to convene conferencing terminals of various network types to the maximum extent. Furthermore, the built-in MCU conferencing terminal supports multiple built-in MCU conferencing terminals to join the conference, forming a radiating network topology structure, which can connect more conferencing terminals to realize video conferencing.

[0018] In one specific embodiment, such as Figure 2 As shown, the multipoint conference control unit MCU 1 is connected to one primary built-in MCU conference terminal 2 and two conference terminals 4 via an E1 leased line. The primary built-in MCU conference terminal 2 is connected to two secondary built-in MCU conference terminals 3 and two conference terminals 4 via an IP leased line. The two secondary built-in MCU conference terminals 3 are connected to three conference terminals 4 via IP leased lines respectively, realizing the mixed networking of built-in MCU video conferencing terminals and multipoint conference control unit MCU.

[0019] In this preferred embodiment, refer to Figure 3 This is a schematic block diagram of the built-in MCU conference terminal structure provided in this embodiment, wherein each of the first-level built-in MCU conference terminals 2 and the second-level built-in MCU conference terminals 3 includes a processing node management module 10 and a call processing module 20; wherein, The processing channel management module 10 is used to create multiple independent media processing channels based on the call bandwidth of the built-in MCU conference terminal. The media processing channels receive the bit stream sent by the calling terminal and perform audio mixing and multi-screen binding synthesis.

[0020] The call processing module 20 determines the role information of each calling terminal based on the received call task, assigns an idle media processing channel to each calling terminal, and marks the media processing channel according to the corresponding role information. Each media processing channel rejects bitstreams sent from calling terminals with the same marked role information. Each media processing channel mixes and synthesizes the received bitstream and sends the resulting mixture to its bound calling terminal. This setting ensures that each calling terminal is bound to a unique media processing channel, and that this channel is used for mixing and synthesizing multi-view tasks. It also ensures that audio and video signals from the calling terminal corresponding to the role information are discarded, preventing the processed audio and video signals from the calling terminal from being sent back to it, thus avoiding audio and video loop problems with the multipoint conferencing control unit MCU1.

[0021] The processing channel management module 10 also monitors the usage of each media processing channel. After the media processing channel finishes processing the bitstream and sends the processing result back to the call terminal bound to it, the processing channel management module 10 unbinds the call terminal corresponding to the media processing channel. This achieves optimized and reasonable use of the call bandwidth of the built-in MCU conference terminal, reducing system energy consumption. Each media processing channel may include an audio codec node, an audio media processing node, a video codec node, and a video media processing node. The audio codec node and the video codec node implement the encoding and decoding of audio and video in the bitstream. The audio media processing node performs audio-video mixing operations on the audio and video decoded by the audio codec node. The video media processing node performs binding and compositing operations on the multiple screens decoded by the video codec node.

[0022] Optionally, based on the above embodiments, in order to achieve a more optimized utilization of the allocated bandwidth by the processing channel management module 10, the processing channel management module 10 allocates multiple audio codec nodes, audio media processing nodes, video codec nodes, and video media processing nodes according to the call bandwidth of the built-in MCU conference terminal. During the call task processing, the processing channel management module 10 first obtains idle audio codec nodes and video codec nodes, determines the role information of the calling terminal according to the call task, and marks the obtained audio codec nodes and video codec nodes with the role information to bind the calling terminal. The audio codec nodes and video codec nodes receive the bitstream sent from the calling terminal whose marked role information is different from theirs, and reject the bitstream sent from the calling terminal whose marked role information is the same. The codec node decodes each audio segment in the bitstream, and the video codec node decodes each video segment in the bitstream. The channel management module 10 then acquires idle audio and video media processing nodes, marks them using role information, binds the audio media processing nodes to the audio codec node and receives the decoded audio information, and binds the video codec node to the video media processing node and receives the decoded video information. The audio media processing node mixes the decoded audio information and feeds it back to the audio codec node for encoding; the video media processing node synthesizes the decoded video information and feeds it back to the video codec node for encoding; the audio and video codec nodes feed the encoded audio and multiple video segments back to the bound calling terminal; finally, the channel management module 10 unbinds the calling terminal from each node, releasing bandwidth.

[0023] In this embodiment, the call task may include, but is not limited to, the call terminal's identity information (information used to distinguish the participants) and role information (information used to distinguish the terminal device attributes or to differentiate different call task issuing entities, such as multipoint conference control unit (MCU), built-in MCU conference terminal, conference terminal name or ID, etc.); in this embodiment, the call terminal is the one that initiates the video call.

[0024] In this embodiment, the call task can be a call initiated by a conference terminal connected to the built-in MCU conference terminal, or a call initiated by other conference terminals through the built-in MCU conference terminal / multipoint conference control unit MCU.

[0025] In this embodiment, the call processing module 20 in the built-in MCU conference terminal configures a media processing channel for each call terminal to process audio and video information. Then, the media processing channel is set to reject the bitstream from the corresponding call terminal itself, ensuring that the audio encoded signal transmitted to the conference room does not contain the bitstream collected by the conference room, thus solving the problem of the transmission loop of the built-in MCU conference terminal and the MCU multi-point conference video signal.

[0026] In this preferred embodiment, each of the first-level built-in MCU conference terminals 2 and the second-level built-in MCU conference terminals 3 includes a task sorting module 30: used to sort the call tasks in real time according to the preset call task priority of the call terminal, and the corresponding built-in MCU conference terminal receives and processes the call tasks according to the task sorting; wherein the terminal priority corresponding to the call task priority is: multipoint conference control unit MCU 1 > first-level built-in MCU conference terminal 2 > second-level built-in MCU conference terminal 3 / conference terminal 4, and each built-in MCU conference terminal can sort the multiple screens according to the task priority; In this embodiment, the task sorting module 30 is designed to ensure that only one multipoint control unit conference exists in the built-in MCU cascaded conference, thus resolving the issue of other built-in MCU conference terminals accessing the multipoint conference control unit MCU when multiple built-in MCU conference terminals are networked. Firstly, it prevents the built-in MCU terminal conference from accessing multiple multipoint conference control unit MCUs, which would cause conference control message disorder. Secondly, it can place the content of the multipoint conference control unit MCU screen at a key position in the synthesis of multiple screens by the built-in MCU conference terminals according to priority.

[0027] Method Implementation Examples According to embodiments of the present invention, a video conferencing signal processing method based on the above-described video conferencing system is provided, such as... Figure 4 The diagram shown is a flowchart of the video conferencing signal processing method provided in this embodiment. According to the video conferencing signal processing method of this embodiment, the processing method is executed in a built-in MCU conferencing terminal and includes the following steps.

[0028] In step S110, the built-in MCU conference terminal receives the call task, determines the call task node according to the call order, and sorts it. Optionally, the call task may be a call initiated by a conference terminal connected to the built-in MCU conference terminal or a call initiated by another conference terminal through the built-in MCU conference terminal.

[0029] In this embodiment, the calling task may include, but is not limited to, the calling terminal's identity information (information used to distinguish participating members) and role information (information used to distinguish terminal device attributes, such as multipoint conference control unit MCU, built-in MCU conference terminal, conference terminal name or ID, etc.); in this embodiment, the calling terminal is the one that initiates the video call.

[0030] In step S120, multiple independent media processing channels are created based on the call bandwidth of the built-in MCU conference terminal; Optionally, the media processing channel receives the bitstream sent by the calling terminal and performs audio mixing and multi-screen binding synthesis.

[0031] In this embodiment, each media processing channel may include an audio codec node, an audio media processing node, a video codec node, and a video media processing node. The audio codec node and the video codec node implement the encoding and decoding of audio and video in the bitstream. The audio media processing node performs audio-video mixing operations on the audio and video decoded by the audio codec node. The video media processing node performs binding and compositing operations on the multiple screens decoded by the video codec node.

[0032] In step S130, the role information of each calling terminal is determined according to the received call task, an idle media processing channel is assigned and bound to each calling terminal, and the media processing channel is marked by the corresponding role information. Each media processing channel rejects the bit stream sent from the calling terminal with the same marked role information. In step S140, each media processing channel sends the processed result of mixing and multi-screen binding of the received bitstream to the call terminal bound to it. In step S150, it is determined whether there are any call tasks to be processed. If so, the call tasks are acquired and step S130 is executed.

[0033] This embodiment configures a media processing channel for each calling terminal to process audio and video information, and sets the media processing channel to reject the bitstream from the corresponding calling terminal itself. This ensures that the audio encoded signal transmitted to the meeting room does not contain the bitstream collected by the meeting room, thus solving the problem of the transmission loop of the built-in MCU conference terminal and the MCU multi-point conference video signal transmission loop. This solves the problem that the audio and video loop problem prevents the MCU and the built-in MCU terminal from being cascaded and networked.

[0034] In some embodiments, such as Figure 5 The diagram shows the execution flowchart of step S130 provided in the embodiment. In step S130, the role information of each calling terminal is determined according to the received call task, an idle media processing channel is allocated and bound to each calling terminal, and the media processing channel is marked according to the corresponding role information. Each media processing channel rejects the bitstream sent from the calling terminal with the same marked role information. Specifically, the steps include: Step S1301: Receive the call task and determine the role information of the call terminal according to the call task; Step S1302: Obtain an idle media processing channel, bind the calling terminal to the media processing channel, and mark the audio codec node, audio media processing node, video codec node and video media processing node of the media processing channel through the role information of the calling terminal; Step S1303: The audio codec node and the video codec node receive bitstreams sent from calling terminals whose role information is different from their own, and reject bitstreams sent from calling terminals whose role information is the same as their own.

[0035] In this embodiment, the above steps are performed to ensure that each calling terminal is bound to a unique media processing channel, and that the channel is used for mixing and multi-screen binding and synthesis tasks. It also enables the rejection of audio and video signals from the calling terminal itself corresponding to the role information through role information, and prevents the audio and video signals of the calling terminal itself from being processed and sent back to the corresponding calling terminal, thereby avoiding the problem of audio and video loops in the multipoint conference control unit MCU1.

[0036] Optionally, based on the above steps, in step S140, the process of each media processing channel sending the mixing result of the received bitstream to its bound calling terminal may include the following steps: Step S1410: The audio decoding node receives and decodes each audio information in the bitstream; Step S1411: The audio media processing node performs mixing processing on each audio information and sends the mixed signal to the audio encoding node; Step S1412: After the audio encoding node performs audio encoding on the mixing signal, it obtains the mixing and sends the mixing to the calling terminal corresponding to its marked role information.

[0037] Optionally, based on the above steps, in step S140, the process of each media processing channel sending the processed result of multi-screen binding and synthesis of the received bitstream to the call terminal bound to it may include the following steps: Step S1420: The video decoding node receives and decodes the information of each frame in the bitstream; Step S1421: The video media processing node performs multi-screen binding and compositing processing on the information of each screen, and sends the composite screen signal to the video encoding node. Step S1422: After the video encoding node encodes the synthesized image signal, it obtains the synthesized image and sends the synthesized image to the calling terminal corresponding to its marked role information.

[0038] In this preferred embodiment, in order to optimize the use of the built-in MCU conference terminal's call bandwidth and reduce system energy consumption, step S140 further includes the following step: Step S1401: After the media processing channel completes the processing of the bitstream and sends the processing result back to the call terminal bound to it, the call terminal corresponding to the media processing channel is unbound from it.

[0039] In another embodiment, the generation and allocation of media processing channels can also be achieved through the following steps: Step S210: The built-in MCU conference terminal creates multiple audio codec nodes, audio media processing nodes, video codec nodes, and video media processing nodes according to the call bandwidth. Step S220: Determine the role information of each calling terminal based on the received call task; Step S230: Based on the role information of the calling terminal, obtain the idle audio codec node, audio media processing node, video codec node and video media processing node respectively, and mark the obtained audio codec node, audio media processing node, video codec node and video media processing node through the role information to realize the binding between nodes and build a media processing channel for processing the corresponding call task.

[0040] The audio codec node, audio media processing node, video codec node, and video media processing node selected through the above steps will unbind from other bound nodes after processing their corresponding information, awaiting the generation of the next binding. After the bitstream processing is completed through the above steps, the built-in MCU conference terminal sends the processing result to the corresponding calling terminal according to the role information. Compared with the above embodiment, this embodiment can allocate audio codec nodes, audio media processing nodes, video codec nodes, or video media processing nodes that are idle at any time according to the bitstream processing needs of the built-in MCU conference terminal, realizing the allocation of bandwidth in smaller units and optimizing its reasonable use, further reducing system energy consumption.

[0041] In some embodiments, the primary built-in MCU conference terminal 2 and the secondary built-in MCU conference terminal 3 sort the call tasks in real time according to task priority, and receive and process the call tasks according to the task priority. The terminal priority corresponding to the call task priority is: Multipoint Conferencing Control Unit MCU 1 > Primary Built-in MCU Conference Terminal 2 > Secondary Built-in MCU Conference Terminal 3 / Conference Terminal 4. Each built-in MCU conference terminal can sort the multiple screens according to the task priority. This setting is to ensure that there is only one multipoint control unit conference in the built-in MCU cascade conference, and to solve the problem of other built-in MCU conference terminals accessing the multipoint control unit (MCU) when multiple built-in MCU conference terminals are networked. Firstly, it prevents the built-in MCU terminal conference from accessing multiple multipoint conference control units (MCUs) and causing conference control message disorder. Secondly, it can place the multipoint conference control unit (MCU) screen content in the key position of the built-in MCU conference terminal to synthesize multiple screens according to priority.

[0042] In this embodiment, the built-in MCU conference terminal sorts call tasks in real time according to task priority, and sequentially acquires and processes call tasks according to the task priority, which may include the following steps: Step S310: The built-in MCU conference terminal receives the call task and determines the role information of the calling terminal according to the call task; based on the role information, it determines whether the terminal device sending the call task is a multipoint conference control unit MCU, a first-level built-in MCU conference terminal, a second-level built-in MCU conference terminal, or a conference terminal. Step S320: Determine the terminal device based on the role information to determine the priority level of the call task, add the corresponding call task below the call tasks of the same level in the task queue, and adjust the order of other call tasks.

[0043] In this preferred embodiment, to ensure that only one multipoint conference control unit (MCU) conference exists in the cascaded conference of the built-in MCU conference terminals (i.e., to determine the calling and called party relationships of each conference terminal, the master-slave relationship of the calling and called parties, and to decide who initiates the establishment of the logical channel), each built-in MCU conference terminal in this embodiment will promptly respond to the multipoint conference control unit (MCU) call and accept the call task. Specifically, this includes the following steps: Step S410: The first-level built-in MCU conference terminal receives the call task and determines the role information of the calling terminal according to the call task; it determines whether the terminal device that sends the call task is a multipoint conference control unit (MCU) according to the role information. If it is, proceed to step S420; otherwise, proceed to step S320. Step S420: The first-level built-in MCU conference terminal receives and processes the call task initiated by the multipoint conference control unit MCU; Step S430: Mark the multipoint conference control unit (MCU) attribute of the first-level built-in MCU conference terminal and synchronize the MCU attribute of the multipoint conference control unit to the participating second-level built-in MCU conference terminals; Step S440: The secondary built-in MCU conference terminal receives the MCU attributes of the multipoint conference control unit, updates its attributes to the MCU attributes of the multipoint conference control unit, and synchronizes the MCU attributes of the multipoint conference control unit to other participating conference terminals or built-in MCU conference terminals outside the sending terminal.

[0044] The embodiments of the present invention are method embodiments corresponding to the system embodiments described above. The specific operations of each step can be understood by referring to the description of the processing of each module in the system, and will not be repeated here.

[0045] like Figure 6 As shown, the present invention also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the video conferencing signal processing method of the above embodiments, or when the computer program is executed by a processor, it implements the video conferencing signal processing method of the above embodiments. When the computer program is executed by the processor, it implements the following method steps: In step S110, the built-in MCU conference terminal receives the call task, determines the call task node according to the call order, and sorts it. In step S120, multiple independent media processing channels are created based on the call bandwidth of the built-in MCU conference terminal; In step S130, the role information of each calling terminal is determined according to the received call task, an idle media processing channel is assigned and bound to each calling terminal, and the media processing channel is marked by the corresponding role information. Each media processing channel rejects the bit stream sent from the calling terminal with the same marked role information. In step S140, each media processing channel sends the processed result of mixing and multi-screen binding of the received bitstream to the call terminal bound to it. In step S150, it is determined whether there are any call tasks to be processed. If so, the call tasks are acquired and step S130 is executed.

[0046] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0047] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video conferencing system, characterized in that, include: A multipoint conference control unit (MCU) (1) is connected to at least one first-level built-in MCU conference terminal (2), and the first-level built-in MCU conference terminal (2) is connected to at least one second-level built-in MCU conference terminal (3); the backbone network of the multipoint conference control unit (MCU) (1) and the first-level built-in MCU conference terminal (2), or the first-level built-in MCU conference terminal (2) and the second-level built-in MCU conference terminal (3) adopts E1 leased line transmission mode, and the local area network adopts IP network transmission mode; The digital signal processing units of the first-level built-in MCU conference terminal (2) and the second-level built-in MCU conference terminal (3) complete the call task access and the processing of audio, video, information and signaling; The primary built-in MCU conference terminal (2) and the secondary built-in MCU conference terminal (3) include a processing node management module (10) and a call processing module (20): The processing node management module (10) is used to create multiple independent media processing channels according to the call bandwidth of the built-in MCU conference terminal. The call processing module (20) is used to determine the role information of each call terminal according to the received call task, allocate and bind an idle media processing channel to each call terminal, mark the media processing channel according to the corresponding role information, and each media processing channel rejects the code stream sent from the call terminal with the same marked role information; Each media processing channel will send the processed result of mixing and multi-screen binding of the received bitstream to the call terminal bound to it.

2. The video conferencing system as described in claim 1, characterized in that, The processing channel management module (10) also monitors the usage of each media processing channel. After the media processing channel finishes processing the bitstream and sends the processing result back to the call terminal bound to it, the processing channel management module (10) unbinds the call terminal corresponding to the media processing channel from it.

3. A video conferencing system as described in claim 1, characterized in that, The first-level built-in MCU conference terminal (2) and the second-level built-in MCU conference terminal (3) include a task sorting module (30). The task sorting module (30) is used to sort the call tasks in real time according to the preset call task priority of the call terminal. The corresponding first-level built-in MCU conference terminal (2) or the second-level built-in MCU conference terminal (3) receives and processes the call tasks according to the task sorting. The priority of the conference terminal corresponding to the call task priority is: Multipoint Conference Control Unit (MCU) (1) > Level 1 Built-in MCU Conference Terminal (2) > Level 2 Built-in MCU Conference Terminal (3) / Conference Terminal (4).

4. A video conferencing signal processing method implemented in a video conferencing system according to any one of claims 1-3, characterized in that, The method is executed in a built-in MCU conference terminal and includes the following steps: The built-in MCU conference terminal receives call tasks, determines the call task nodes according to the call order, and sorts them. Multiple independent media processing channels are created based on the call bandwidth of the built-in MCU conference terminal. The role information of each calling terminal is determined based on the received call task. Each calling terminal is assigned and bound to an idle media processing channel. The media processing channel is marked with the corresponding role information, and each media processing channel rejects the bit stream sent from the calling terminal with the same marked role information. Each media processing channel will mix and synthesize the received bitstream and send the resulting data to the call terminal it is bound to.

5. A video conferencing signal processing method as described in claim 4, characterized in that, The process of generating and allocating media processing channels includes the following steps: The built-in MCU conference terminal creates multiple audio codec nodes, audio media processing nodes, video codec nodes, and video media processing nodes according to the call bandwidth. The role information of each calling terminal is determined based on the received call task; Based on the role information of the calling terminal, idle audio codec nodes, audio media processing nodes, video codec nodes, and video media processing nodes are obtained respectively. The obtained audio codec nodes, audio media processing nodes, video codec nodes, and video media processing nodes are marked by role information to realize the binding between each node and build a media processing channel for processing the corresponding call task.

6. The video conferencing signal processing method as described in claim 4, characterized in that, The The built-in MCU conference terminal prioritizes call tasks in real time and receives and processes them according to this priority. The priority of the conference terminal corresponding to the call task priority is: Multipoint Conference Control Unit (MCU) (1) > Level 1 Built-in MCU Conference Terminal (2) > Level 2 Built-in MCU Conference Terminal (3) / Conference Terminal (4).

7. A video conferencing signal processing method as described in claim 6, characterized in that, The built-in MCU conference terminal sorts call tasks in real time according to task priority, and then acquires and processes call tasks sequentially according to the task priority, which may include the following steps: The built-in MCU conference terminal receives call tasks and determines the role information of the calling terminal based on the call task; based on the role information, it determines whether the terminal device sending the call task is a multipoint conference control unit (MCU), a first-level built-in MCU conference terminal, a second-level built-in MCU conference terminal, or a conference terminal. The terminal device is determined based on the role information to determine the priority level of the call task, and the corresponding call task is added below the call tasks of the same level in the task queue, and the order of other call tasks is adjusted.

8. A video conferencing signal processing method as described in claim 6, characterized in that, The built-in MCU conference terminal will promptly respond to calls from the multipoint conference control unit (MCU) and accept call tasks during the conference, specifically including the following steps: The first-level built-in MCU conference terminal receives the call task and determines the role information of the calling terminal based on the call task; it determines whether the terminal device that sends the call task is a multipoint conference control unit (MCU) based on the role information. If it is, it executes step S420; otherwise, it executes step S320. The first-level built-in MCU conference terminal receives and processes call tasks initiated by the multipoint conference control unit MCU; The attributes of the first-level built-in MCU conference terminal are marked with the attributes of the multipoint conference control unit (MCU), and the attributes of the multipoint conference control unit (MCU) are synchronized to the participating second-level built-in MCU conference terminals; The secondary built-in MCU conference terminal receives the attributes of the multipoint conference control unit (MCU), updates its attributes to the attributes of the multipoint conference control unit (MCU), and synchronizes the attributes of the multipoint conference control unit (MCU) to other participating conference terminals or built-in MCU conference terminals outside the sending terminal.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a video conferencing signal processing method as described in any one of claims 4 to 8.

Images