Control method and device of audio-video conference discussion system and storage medium

Abstract

The application discloses a control method and device of an audio and video conference discussion system and a storage medium, relates to the technical field of computer hardware, and comprises the following steps: in response to a user operation input, collecting user operation position data through a man-machine interaction peripheral device, and generating user operation event signaling in combination with a first slave end identifier; extracting the first slave end identifier and the user operation position data in the user operation event signaling through an ARM processor, and determining a slave end display picture according to the first slave end identifier; analyzing a user operation intention based on the user operation position data and the slave end display picture through the ARM processor, generating a display screen refresh image, and issuing the display screen refresh image to a slave end FPGA of the first slave end; and driving the display screen to display the display screen refresh image through the slave end FPGA of the first slave end, so as to solve the technical problem that the overall reliability of a multi-slave end system is low due to the complexity of the slave end software and hardware of an existing audio and video conference system, and improve the overall reliability of the multi-slave end system.

Description

Technical Field

[0001] This application relates to the field of computer hardware technology, and in particular to a control method, device and storage medium for an audio and video conferencing discussion system. Background Technology

[0002] In current audio and video conferencing systems with touchscreens, mainstream solutions generally adopt a distributed architecture of "one host plus multiple units." In this architecture, each unit uses an Advanced Reduced Instruction Set Computing Machine (ARM) application processor as its core computing unit and runs a complete Linux or Android operating system, forming a fully functional terminal device. Each unit independently handles complex local processing tasks, including touch operation response, graphical user interface rendering and display, and audio acquisition and playback. All units connect to the host via wired or wireless networks. The host is also based on an ARM processor and Linux system. Units and the host communicate and synchronize their status through a custom proprietary signaling protocol to achieve basic conferencing and discussion functions.

[0003] However, this architecture design, which pushes highly complex hardware and software systems down to each unit, results in high complexity of unit hardware and software, inherently carries a high risk of failure. Furthermore, a single system typically contains more than ten or even dozens of units, with multiple units' ARM chips communicating with the main ARM chip in real time, further reducing the overall reliability of the system and failing to meet the requirements of stable system operation in conference scenarios. Summary of the Invention

[0004] The main purpose of this application is to provide a control method, device and storage medium for an audio and video conferencing discussion system, which aims to solve the technical problem of low overall reliability of existing audio and video conferencing systems due to the complexity of the hardware and software on the slave end.

[0005] To achieve the above objectives, this application proposes a control method for an audio-visual conferencing discussion system. The audio-visual conferencing discussion system includes a master terminal, a network switch, and at least one slave terminal. The master terminal includes a master field-programmable gate array (FPGA) and an advanced reduced instruction set machine (RISC machine, ARM) processor. Each slave terminal includes a slave FPGA and a human-computer interaction peripheral device. The human-computer interaction peripheral device includes at least a display screen and a keyboard. The control method for the audio-visual conferencing discussion system includes:

[0006] In response to user operation input on the human-computer interaction peripheral device of the first slave end, user operation position data is collected through the human-computer interaction peripheral device of the first slave end, and combined with the first slave end identifier, user operation event signaling is generated through the slave end FPGA of the first slave end. The first slave end can be any of the slave ends, and the user operation input is a touch operation or a button operation.

[0007] The user operation event signaling is forwarded to the ARM processor of the main terminal through the network switch and the main FPGA;

[0008] The ARM processor extracts the first slave identifier and the user operation location data from the user operation event signaling, and determines the slave display screen when the user operation input occurs based on the first slave identifier;

[0009] Based on the user operation location data and the slave display screen, the ARM processor parses the user operation intention, generates a screen refresh image, and sends the screen refresh image to the slave FPGA of the first slave through the master FPGA and the network switch.

[0010] The first slave terminal's display screen is driven by the slave FPGA of the first slave terminal to display the refreshed image of the display screen;

[0011] In one embodiment, prior to the step of collecting user operation location data through the first slave-end human-computer interaction peripheral device in response to user operation input on the first slave-end human-computer interaction peripheral device, the method further includes:

[0012] The ARM processor generates image frame data for the display screen of the first slave end, and encapsulates the image frame data and the first slave end identifier into a network data packet;

[0013] The network data packets are sent to the slave FPGA of the first slave end through the master FPGA and the network switch;

[0014] The image frame data in the network data packet is parsed by the slave FPGA of the first slave end, and the image frame data is written into the external memory of the first slave end;

[0015] The first slave FPGA reads image frame data from the external memory according to a preset refresh cycle, and drives the display screen of the first slave to display the image frame data.

[0016] In one embodiment, the human-computer interaction peripheral device further includes other peripheral devices besides the display screen and keyboard. After the step of parsing the user's operation intent through the ARM processor based on the user operation location data and the slave display screen, the device further includes:

[0017] Based on the user's operation intention, the ARM processor generates peripheral control instructions, which include the target peripheral type, control parameters, and a first target slave identifier.

[0018] Based on the first target slave identifier, the peripheral control command is sent to the slave FPGA of the first target slave through the master FPGA and the network switch;

[0019] Based on the target peripheral type and the control parameters, the target peripheral device of the first target slave end is driven by the slave FPGA of the first target slave end to execute the corresponding operation of the control parameters, and the other peripheral devices include the target peripheral device.

[0020] In one embodiment, when the target peripheral device is a microphone, the step of driving the target peripheral device of the first target slave end to execute the corresponding operation of the control parameters through the slave FPGA of the first target slave end includes:

[0021] The first target slave terminal's microphone is driven by the slave FPGA of the first target slave terminal to perform an audio acquisition operation, so as to acquire audio signals through the microphone of the first target slave terminal;

[0022] After the step of driving the microphone of the first target slave end to perform audio acquisition operation through the slave FPGA of the first target slave end, the method further includes:

[0023] The first target slave identifier and the audio signal are packaged and processed by the slave FPGA of the first target slave to generate a corresponding audio data packet, and the audio data packet is transmitted to the master FPGA through the network switch;

[0024] The received audio data packets are unpacked by the main FPGA to obtain the unpacked audio stream.

[0025] The main FPGA controls the main audio output port to play the unpacked audio stream.

[0026] In one embodiment, when the human-computer interaction peripheral device further includes a microphone and an audio player, the control method of the audio-visual conferencing discussion system further includes:

[0027] Based on system configuration information, the ARM processor selects a target audio source from multiple candidate audio sources, and determines that the candidate audio source includes the microphone audio stream uploaded by the slave end or the input audio stream from the master end.

[0028] The ARM processor encapsulates the second target slave identifier and the target audio source into an audio routing data packet, and sends the audio routing data packet to the master FPGA.

[0029] The audio routing data packet is sent to the slave FPGA of the second target slave end through the master FPGA and the network switch;

[0030] The second target slave FPGA receives and unpacks the audio routing data packet to obtain the target audio stream corresponding to the target audio source after unpacking.

[0031] The target audio stream is played by the audio player of the second target slave terminal controlled by the slave FPGA of the second target slave terminal.

[0032] In one embodiment, the control method for the audio and video conferencing discussion system further includes:

[0033] When the ARM processor receives a high-priority operation signaling sent by a preset high-priority device for a third target slave, the ARM processor performs authorization verification on the high-priority operation signaling. The preset high-priority device includes a master web management terminal, an external centralized control system, or a slave with chairman authority.

[0034] After the high-priority operation signaling is verified, the ARM processor parses the high-priority operation signaling, generates a high-priority control instruction, and places the high-priority control instruction into a high-priority transmission queue. The high-priority control instruction includes a third target identifier, a third peripheral type, and peripheral control parameters.

[0035] The high-priority transmission queue is preemptively scheduled by the master FPGA, and the high-priority control command is transmitted to the slave FPGA of the third target slave through the master FPGA and the network switch based on the third target identifier.

[0036] Based on the third peripheral type and the peripheral control parameters, the state of the third target peripheral register corresponding to the human-machine interaction peripheral device of the third peripheral type in the slave FPGA of the third target slave end is modified by the slave FPGA of the third target slave end, so as to drive the human-machine interaction peripheral device of the third peripheral type to execute the corresponding operation of the peripheral control parameters.

[0037] In one embodiment, the step of determining the slave display screen when the user operation input occurs based on the first slave identifier includes:

[0038] Based on the first slave identifier and the user interface state model maintained by the ARM processor in the master memory that is synchronized with the actual display content of each slave, the ARM processor retrieves the first slave user interface state model corresponding to the first slave from the master memory.

[0039] The ARM processor extracts the timestamp of the user operation input from the user operation event signaling, and based on the timestamp, extracts the slave display screen from the first slave user interface state model.

[0040] In one embodiment, the step of generating a refresh image for the display screen by parsing the user's operation intent through the ARM processor based on the user's operation location data and the slave display screen includes:

[0041] The ARM processor performs coordinate matching between the user operation location data and the location area information of multiple interface elements in the slave display screen to determine the target interface element to be triggered. The slave display screen includes the location area information of multiple interface elements and the operation functions associated with each interface element.

[0042] Based on the target operation function associated with the target interface element, the system function logic corresponding to the target operation function is executed by the ARM processor to render and generate a screen refresh image that reflects the execution result of the system function logic.

[0043] Furthermore, to achieve the above objectives, this application also proposes a control device for an audio-visual conferencing discussion system, the control device comprising:

[0044] The acquisition module is used to respond to user operation input on the human-computer interaction peripheral device of the first slave end, acquire user operation position data through the human-computer interaction peripheral device of the first slave end, and generate user operation event signaling through the slave FPGA of the first slave end in combination with the first slave end identifier. The first slave end can be any of the slave ends, and the user operation input is a touch operation or a key operation.

[0045] The signaling transmission module is used to forward the user operation event signaling to the ARM processor of the master terminal through the network switch and the master FPGA;

[0046] The display screen determination module is used to extract the first slave identifier and the user operation position data from the user operation event signaling through the ARM processor, and determine the slave display screen when the user operation input occurs based on the first slave identifier;

[0047] The refresh image generation module is used to generate a refresh image of the display screen based on the user operation location data and the display screen of the slave end, by parsing the user operation intention through the ARM processor, and by sending the refresh image of the display screen to the slave FPGA of the first slave end through the master FPGA and the network switch.

[0048] The display update module is used to drive the display screen of the first slave end to display the refreshed image through the slave FPGA of the first slave end.

[0049] In addition, to achieve the above objectives, this application also proposes a control device for an audio-visual conferencing discussion system, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for the audio-visual conferencing discussion system as described above.

[0050] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the control method of the audio and video conferencing discussion system described above.

[0051] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the control method for the audio and video conferencing discussion system described above.

[0052] The one or more technical solutions proposed in this application have at least the following technical effects: The control method for an audio-visual conferencing discussion system is applied to an audio-visual conferencing discussion system, which includes a master end, a network switch, and at least one slave end. The master end includes a master FPGA and an ARM processor. Each slave end includes a slave FPGA and a human-machine interface peripheral device, which includes at least a display screen and a keyboard. In response to user input on the human-machine interface peripheral device of the first slave end, user operation position data is collected through the first slave end's human-machine interface peripheral device. Combined with the first slave end's identifier, user operation event signaling is generated through the slave end's slave FPGA. The first slave end can be any slave end, and the user operation input can be a touch operation or a key operation. The slave end only uses the human-machine interface peripheral device and the FPGA to complete user operation data acquisition and signaling generation, significantly reducing the complexity of the slave end and significantly improving the reliability of a single slave end. The user operation event signaling is forwarded to the ARM processor of the master end through the network switch and the master end FPGA. The system extracts the first slave identifier and user operation location data from the user operation event signaling using an ARM processor, and determines the slave display screen when the user operation input occurs based on the first slave identifier. Based on the user operation location data and the slave display screen, the ARM processor parses the user's operation intent, generates a screen refresh image, and sends the screen refresh image to the slave FPGA of the first slave through the master FPGA and network switch. The master FPGA acts as a communication forwarding unit, and the master ARM centrally parses all signaling. Finally, the slave FPGA of the first slave drives the first slave's screen to display the screen refresh image. The slave FPGA receives the screen refresh image sent from the master and drives the screen to display. This eliminates the need for complex logic such as operation intent parsing and display content generation, solving the technical problem of low overall system reliability in existing audio / video conferencing systems due to complex slave hardware and software. By moving the intelligent interaction logic, previously handled independently by each slave device, to the master device for centralized processing, the slave devices retain only hardware-level data acquisition and display driving functions. This significantly simplifies the slave device's hardware and software design, greatly improves the overall reliability of multi-slave systems, and eliminates the system vulnerability caused by excessive complexity of slave device hardware and software in existing technologies. While maintaining the user interaction experience, it achieves the high availability, strong consistency, and long-term stability required by professional conference systems. The control method of the audio-visual conferencing discussion system provided in this application centralizes intelligent decision-making at the master device. The slave devices collect user operation position data through human-computer interaction peripherals and generate user operation event signaling via the slave FPGA. This signaling is efficiently forwarded to the ARM processor via a network switch and the master FPGA. The ARM processor accurately parses the user's intent based on the slave device identifier and user operation position data, generates a new display refresh image, and sends it to the slave FPGA. The slave FPGA receives and drives the display of this refresh image.As a result, the hardware and software complexity of the slave end is greatly reduced, and the risk of single point of failure is significantly reduced. At the same time, the master end manages all slave ends in a unified manner, and the master end FPGA shares the underlying communication, avoiding the ARM processor being impacted by multiple concurrent requests. While maintaining a smooth interactive experience, it achieves highly reliable and easy-to-maintain professional-grade audio and video conferencing control, effectively solving the technical problem of low overall reliability caused by the complexity of slave end hardware and software in existing audio and video conferencing systems. Attached Figure Description

[0053] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating an embodiment of the control method for the audio / video conferencing discussion system of this application.

[0056] Figure 2 A schematic diagram of the control system architecture for an audio-visual conferencing discussion system provided in this application;

[0057] Figure 3 A schematic diagram illustrating a conventional display process provided in this application;

[0058] Figure 4 A flowchart illustrating the linkage between user operation input and peripheral device control provided in this application;

[0059] Figure 5 A schematic diagram illustrating a process for transmitting audio from a slave end, as provided in this application;

[0060] Figure 6 A schematic diagram illustrating the process of receiving audio from the slave end, as provided in this application;

[0061] Figure 7 This is a schematic diagram of the module structure of the control device of the audio and video conferencing discussion system according to an embodiment of this application;

[0062] Figure 8 This is a schematic diagram of the hardware operating environment involved in the control method of the audio and video conferencing discussion system in the embodiments of this application. Detailed Implementation

[0063] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0064] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0065] The main solution of this application embodiment is: applied to an audio and video conferencing discussion system, the audio and video conferencing discussion system includes a master end, a network switch and at least one slave end, the master end includes a master end FPGA and an ARM processor, each slave end includes a slave end FPGA and a human-computer interaction peripheral device, the human-computer interaction peripheral device includes at least a display screen and a keyboard, the control method of the audio and video conferencing discussion system includes: responding to user operation input on the human-computer interaction peripheral device of the first slave end, collecting user operation position data through the human-computer interaction peripheral device of the first slave end, and combining it with the first slave end identifier, generating user operation event signaling through the slave end FPGA of the first slave end, the first slave end being any slave end, the user operation The input is either touch or button operation; the user operation event signaling is forwarded to the ARM processor of the master terminal through the network switch and the master FPGA; the ARM processor extracts the first slave terminal identifier and user operation position data from the user operation event signaling, and determines the slave terminal display screen when the user operation input occurs based on the first slave terminal identifier; based on the user operation position data and the slave terminal display screen, the ARM processor parses the user operation intention, generates a display screen refresh image, and sends the display screen refresh image to the slave terminal FPGA of the first slave terminal through the master FPGA and the network switch; the slave terminal FPGA of the first slave terminal drives the display screen of the first slave terminal to display the display screen refresh image.

[0066] In this embodiment, for ease of description, the control system of the audio and video conferencing discussion system will be used as the execution subject in the following description.

[0067] Current touchscreen-equipped audio and video conferencing systems generally employ a distributed architecture of "one host plus multiple units." In this architecture, each unit uses an ARM application processor as its core computing unit and runs a complete Linux or Android operating system, forming a fully functional terminal device. Each unit independently handles complex local processing tasks, including touch operation response, graphical user interface rendering and display, and audio acquisition and playback. All units connect to the host via wired or wireless networks. The host is also based on an ARM processor and Linux system. Units and the host communicate and synchronize their status through a custom private signaling protocol to achieve basic conferencing functions. However, this architecture, which pushes highly complex hardware and software systems down to each unit, results in high unit hardware and software complexity, inherently posing a high risk of failure. Furthermore, a single system typically contains ten or even dozens of units, with multiple units' ARM chips communicating simultaneously with the host's ARM chip in real time, further reducing the overall system reliability and failing to meet the stable operation requirements of conferencing scenarios.

[0068] This application provides a solution that centralizes intelligent decision-making at the master end. The slave end, through a human-computer interaction peripheral device, collects user operation location data and generates user operation event signaling via a slave FPGA. This signaling is efficiently forwarded to an ARM processor via a network switch and the master FPGA. The ARM processor accurately parses the user's intent based on the slave identifier and user operation location data, generates a new display refresh image, and sends it to the slave FPGA. The slave FPGA receives and drives the display of this refresh image. This significantly reduces the hardware and software complexity of the slave end and greatly diminishes the risk of single-point failure. Simultaneously, the master end centrally manages all slave ends, and the master FPGA handles the underlying communication, preventing the ARM processor from being overwhelmed by multiple concurrent requests. While maintaining a smooth interactive experience, it achieves highly reliable and easily maintainable professional-grade audio and video conferencing control, effectively solving the technical problem of low overall reliability in existing audio and video conferencing systems due to the complexity of the slave end's hardware and software.

[0069] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a control device for an audio / video conferencing discussion system capable of performing the above functions. The following description uses the control system of an audio / video conferencing discussion system as an example to illustrate this embodiment and the subsequent embodiments.

[0070] Based on this, embodiments of this application provide a control method for an audio and video conferencing discussion system, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the control method for the audio / video conferencing discussion system of this application.

[0071] In this embodiment, the control method for the audio-visual conferencing discussion system is applied to the audio-visual conferencing discussion system, which includes a master end, a network switch, and at least one slave end. The master end includes a master field-programmable gate array (FPGA) and a reduced instruction set machine (ARM) processor. Each slave end includes a slave FPGA and human-computer interaction peripheral devices, which at least include a display screen and a keyboard. The control method for the audio-visual conferencing discussion system includes steps 101-105:

[0072] Step 101: In response to user operation input on the human-machine interaction peripheral device of the first slave end, user operation position data is collected through the human-machine interaction peripheral device of the first slave end, and combined with the first slave end identifier, user operation event signaling is generated through the slave FPGA of the first slave end. The first slave end can be any slave end, and the user operation input is a touch operation or a button operation.

[0073] For details, please refer to Figure 2 The audio-visual conferencing system consists of two parts: slave (unit side) and master (host side). The system includes a master, a network switch, and at least one slave. Each slave includes a slave FPGA, human-computer interaction peripherals, and a slave physical layer transceiver (PHY) chip. The human-computer interaction peripherals may include a display screen, keyboard, microphone, speaker, and indicator lights. The display screen includes a liquid crystal display (LCD) module and a touch module. The master includes a master physical layer chip, a master FPGA, and an ARM processor.

[0074] Each slave device uses a slave FPGA as its core control unit. One side of the slave FPGA connects to an external network (network switch) via a slave physical layer chip, enabling bidirectional data interaction with the master device and providing a channel for subsequent audio / video and command transmission. The slave physical layer chip converts the FPGA's digital signals into network-transmittable electrical / optical signals, acting as a "bridge" between the slave FPGA and the external network. Relying on a general network (rather than a custom link), the slave physical layer chip offers greater compatibility, and the slave FPGA directly controls the PHY chip, resulting in significantly lower communication latency compared to the software-level control of an ARM+ operating system. The other side of the slave FPGA directly interfaces with all human-machine interface peripherals, including displays, microphones, speakers, keyboards, and indicator lights, constructing a slave hardware control network. Each slave FPGA undertakes key data processing and control tasks, specifically including audio signal transmission and reception, control and transmission of display content, perception and transmission of touch operations and key commands, and control of slave indicator lights and other peripherals. The slave FPGA serves as the "functional execution center" of the slave device. The slave-side FPGA has dedicated external memory (i.e., external memory), which can be Random Access Memory (RAM). This external memory can cache the content to be displayed on the screen, ensuring the timeliness and stability of display refresh. The slave-side physical layer chip is also connected to Power over Ethernet (PoE). PoE is a technology that transmits both data and power through standard Ethernet cables (such as Cat5e and Cat6 cables), eliminating the need for additional power cables and enabling a single Ethernet cable to handle both data and power.

[0075] The master unit adopts a dual-core collaborative architecture of FPGA+ARM, with the master FPGA and ARM processor complementing each other to jointly support system operation. Specifically, the master FPGA acts as a communication bridge between the master and slave units, enabling stable data interaction with the slave FPGA via standard network protocols or custom non-standard protocols. Its core responsibilities include bidirectional audio signal transmission, transmission of display content, transmission of touch and key event signals, and forwarding of indicator light and other peripheral control commands, ensuring efficient data and command flow between the master and slave units. The master FPGA is the core communication and co-processing unit of the master unit. One end of the master FPGA connects to the ARM processor via an internal bus, and the other end connects to the master physical layer chip, thereby connecting to the network switch. By implementing low-level communication functions through hardware logic, the master FPGA effectively offloads the communication burden from the main control processor (i.e., the ARM processor), allowing the ARM processor to focus on conference logic control and user intent parsing, thus improving the overall system's real-time performance, throughput, and reliability. The master FPGA is also connected to the master audio output port, master audio input port, and master video input port to enable system interfacing with external audio and video devices. The master audio output port can be used to output the mixed audio signal (such as the speaker's voice or system prompts) processed by the ARM processor to the sound reinforcement system or recording equipment; the master audio input port can be used to connect external audio sources (such as chairman's microphone, wireless microphone, or audio from remote conferencing terminals) for the ARM processor to mix or distribute; the master video input port can be used to receive external video signals (such as camera or PC screens), which can be overlaid with UI information by the ARM processor and uniformly distributed to the displays of each slave end, or used for local monitoring. The master audio output port, master audio input port, and master video input port are all time-controlled and data buffered by the master FPGA to ensure low-latency and high-synchronization transmission of audio and video signals. The master ARM assumes the responsibility of "central control and data processing core" of the system, specifically including centralized management of all slave ends, scheduling of the entire system's operating logic, and encoding, decoding, processing, and distribution of audio and video signals; it is the core carrier of system decision-making and data processing. The master physical layer chip (PHY chip) is the underlying hardware interface for network communication on the master side. One end of the master physical layer chip connects to the master FPGA, and the other end can be connected to the network switch via a network cable. The master physical layer chip can convert the digital Ethernet frame signals output by the master FPGA into differential analog electrical signals conforming to the Institute of Electrical and Electronics Engineers (IEEE) 802.3 standard, and transmit them to the switch via the network medium. At the same time, it also converts the differential signals received from the network switch back into digital signals and sends them to the master FPGA for processing.The master physical layer chip works in conjunction with the master FPGA to complete the physical transmission functions below the data link layer, providing the hardware foundation for reliable and high-speed communication between the master and each slave.

[0076] The audio and video conferencing discussion system simplifies design by using FPGAs on the slave end and centralizes decision-making by using ARMs on the master end, reducing complexity on the slave end while ensuring the reliability and collaboration of multi-slave systems. Through efficient collaboration between the master and slave FPGAs, the system constructs a low-latency, high-throughput control channel, supporting the stringent requirements of real-time performance, stability, and consistency in professional conferencing scenarios.

[0077] It should be noted that the first slave end refers to any slave unit where user input occurs, and it has a unique identifier in the system (e.g., ID=05). In this embodiment, the human-computer interaction peripheral device can be either a display screen or a keyboard, serving as the hardware interface for the user to input commands into the system. User input can be touch operation or button operation. Touch operation includes clicking, swiping, etc., on the display screen. Optionally, in response to a user's touch operation on the first slave end's display screen, touch position data is collected through the first slave end's display screen; this touch position data can be screen coordinate values ​​output by the touch chip. Button operation involves the user pressing a physical button, outputting button position data, which can be a button code or a button status signal. Optionally, in response to a user's button operation on the first slave end's keyboard, button position data is collected through the first slave end's keyboard; this button position data can be mapped to virtual coordinates or predefined function codes. The first slave end identifier is a unique code for each slave end, used by the master end to distinguish different slave ends. The slave end FPGA is the core control chip of the slave end, without an operating system, directly connected to and driving all human-computer interaction peripheral devices through hardware logic, possessing real-time response capabilities. The user operation event signaling is a structured data packet generated by the slave FPGA, which may contain: [Packet Header] | [First Slave Identifier] | [Event Type (Touch Operation / Button Operation)] | [User Operation Location Data] | [Timestamp] | [Check Field].

[0078] In some embodiments, when a user performs a touch operation or a key operation on any of the slave (i.e., the first slave) human-computer interaction peripheral devices (such as a display screen or a keypad), the slave human-computer interaction peripheral device responds in real time and collects the user's operation position data. If it is a touch operation, the touch module of the display screen captures the touch position data of the operation point; if it is a key operation, the keyboard outputs the key position data of the corresponding key. Subsequently, the slave FPGA receives the user's operation position data (touch position data or key position data) and automatically associates it with the "first slave identifier" (a unique code for each slave) pre-stored in its internal register. According to the system's preset signaling format (including event type, slave identifier, user operation position data, verification field, timestamp, etc.), the above information is encapsulated into standardized user operation event signaling and temporarily stored in the FPGA's transmission buffer, waiting to be transmitted to the master end via the Ethernet interface. Through FPGA hardware-level input acquisition and structured signaling generation, the user operation input (touch operation or key operation) in the physical world is transformed into structured, transmissible, and identifiable digital event signaling, with an identity identifier, providing accurate input for centralized parsing by the master end. By directly acquiring and packaging data from the slave FPGA, the complexity of running an operating system and application layer logic on the slave side, which is required in traditional solutions, is avoided. This significantly reduces the hardware and software burden on the slave side and provides a precise, efficient, and reliable data source for subsequent centralized intelligent decision-making on the master side.

[0079] Step 102: The user operation event signaling is forwarded to the ARM processor of the main terminal through the network switch and the main FPGA.

[0080] Specifically, the network switch is a standard Layer 2 Ethernet switching device that can connect the master end to all slave ends. It can perform targeted frame forwarding based on the Media Access Control (MAC) address table, ensuring that signaling is only delivered to the master end and not broadcast to other slave ends. The master end FPGA is a field-programmable gate array that acts as a hardware communication coprocessor between the ARM processor and the network, responsible for protocol parsing, data filtering, priority scheduling, and bus bridging. for The main terminal's high-level control core can handle complex tasks such as meeting logic, UI management, and intent parsing.

[0081] In some embodiments, the slave FPGA sends user operation event signaling to the slave physical layer chip, which converts it into Ethernet differential electrical signals and transmits them to the network switch via a network cable. The network switch parses the master address in the signaling and forwards it to the master physical layer chip, which then restores it to digital signaling and transmits it to the master FPGA. The master FPGA verifies the signaling checksum field, strips the frame header, and forwards the user operation event signaling to the master ARM processor. This hardware link achieves low-latency, high-reliability signaling transmission, constructing a low-latency, high-deterministic data path that aggregates distributed user input to the main control center. Simultaneously, the master FPGA offloads the communication burden, preventing the ARM processor from being impacted by underlying network traffic, thus ensuring overall system stability and real-time response capabilities.

[0082] Optionally, each slave end also includes a slave physical layer chip, and the master end also includes a master physical layer chip. The steps of forwarding user operation event signaling to the master end's ARM processor through the network switch and the master end FPGA include:

[0083] The user operation event signaling is encapsulated into Ethernet frames according to a preset link layer protocol by the slave FPGA of the first slave end, and the Ethernet frames are sent to the slave physical layer chip of the first slave end.

[0084] The first slave physical layer chip converts Ethernet frames into differential signals that conform to the physical layer electrical specifications and sends the differential signals to the network switch.

[0085] The network switch parses the master address in the differential signal and forwards the differential signal to the master physical layer chip based on the master address.

[0086] The differential signal is restored into a digital Ethernet frame by the main physical layer chip, and the digital Ethernet frame is sent to the main FPGA.

[0087] The main FPGA performs data link layer protocol parsing on the digital Ethernet frames, extracts user operation event signaling, and sends the user operation event signaling to the ARM processor.

[0088] For example, when the slave FPGA of the first slave end generates a user operation event signaling, it first adds a frame header (containing the master MAC address and slave MAC address), a frame trailer (containing a CRC check field, etc.), and a length identifier to the user operation event signaling according to a preset Ethernet data link layer protocol (such as the IEEE 802.3 standard), encapsulating it into a standardized Ethernet frame to ensure the standardization of data transmission. Subsequently, this Ethernet frame is sent to the slave physical layer chip of the first slave end. The slave physical layer chip converts this digital format Ethernet frame into a differential electrical signal conforming to physical layer electrical specifications, such as a 1000BASE-T standard differential signal, and transmits it to the network switch via a network cable, achieving physical layer adaptation of the signal. After receiving the differential signal, the network switch can perform signal reconstruction and frame parsing, extract the master address information (master MAC address) from the frame header, determine the target transmission port based on the master address information through an internal port mapping table, and forward the differential signal to the master physical layer chip, avoiding data transmission conflicts between multiple slave ends. After receiving the differential signal, the master physical layer chip converts it into a digital Ethernet frame and transmits it to the master FPGA. The master FPGA performs data link layer protocol parsing on the Ethernet frame. It first verifies the Cyclic Redundancy Check (CRC) field to ensure data integrity, then removes the frame header, frame trailer, and other transmission overhead fields to accurately extract the user operation event signaling. Finally, it sends the signaling to the ARM processor through a high-speed interface (such as the AXI bus) inside the master. Low-latency signaling transmission is achieved through a fully hardware link, avoiding the latency accumulation caused by software forwarding and ensuring real-time operation response. Simultaneously, standardized protocol encapsulation and verification mechanisms ensure the accuracy of signaling transmission, reducing the risk of data loss or errors. The network switch's directional forwarding function adapts to multi-slave concurrent communication scenarios, solving the congestion problem in traditional multi-device communication. The master FPGA and physical layer chip share the communication processing load, allowing the ARM processor to focus on logic decision-making, further optimizing system resource allocation. Combined with the lack of software design on the slave side, the overall reliability of the multi-slave system is improved from the transmission link level.

[0089] Step 103: Extract the first slave identifier and user operation position data from the user operation event signaling through the ARM processor, and determine the slave display screen when the user operation input occurs based on the first slave identifier.

[0090] Specifically, the screen displayed on the slave device is the actual screen content and corresponding screen metadata (which may include the location area information of interface elements, the status of each interface element, and the operation functions associated with each interface element) that are presented on the first slave device screen when the user operation occurs. The status of the interface element can be enabled / disabled, and the operation function associated with the interface element can be "click this area = initiate a vote".

[0091] In some embodiments, after receiving a user operation event signaling, the master ARM processor parses the signaling and extracts the first slave identifier and user operation location data (such as touch coordinates or key codes). Subsequently, based on the first slave identifier, the ARM processor searches its local memory for the corresponding slave display screen. This slave display screen is structured data that the ARM processor synchronously updates each time it sends a refresh image to the first slave, accurately recording the location, state, and associated functions of each interface element on the first slave's current screen. Through the logical steps of signaling parsing, identifier matching, and screen confirmation, the slave display screen at the time of user input is accurately located, providing crucial information for subsequent analysis of user operation intentions and ensuring the accuracy of multi-slave system operation analysis. Simultaneously, relying on centralized management from the master, slave functions are further simplified, and system reliability is improved.

[0092] Optionally, the step of determining the slave display screen when the user operation input occurs based on the first slave identifier includes:

[0093] Based on the first slave identifier and the user interface state model maintained by the ARM processor in the master memory that is synchronized with the actual display content of each slave, the first slave user interface state model corresponding to the first slave is retrieved from the master memory by the ARM processor.

[0094] The timestamp of the user operation input in the user operation event signaling is extracted by the ARM processor, and the slave display screen is extracted from the first slave user interface state model based on the timestamp.

[0095] Specifically, the user interface state model maintained by the ARM processor in the master-side memory is a structured UI state snapshot dynamically maintained in the master-side RAM for each slave device. It includes the position, size, status (enabled / disabled), and bound operation functions of all interface elements (buttons, menus, etc.) displayed on the slave device's screen in real time. The user interface state model is updated synchronously each time a new image frame is sent to the slave device. The master-side memory is the storage medium used by the master to store the interface state models of each slave device, supporting high-speed read / write and real-time updates. The timestamp is the time of operation recorded in the user operation event signaling, which can be used to match the slave device's display screen at the moment of the operation. The first slave device's user interface state model is an interface state model maintained exclusively by the master for the first slave device, completely recording the display screen data of that first slave device at different points in time.

[0096] For example, the master ARM processor maintains a user interface state model in its master memory that is strictly synchronized with the real-time display content of each slave device. The user interface state model records the position, state, and associated function of each interface element (such as buttons and menu items) in the current interface in the form of structured data, and is updated synchronously each time a new display refresh image is sent to the corresponding slave device. When a user operation event signal is received, the ARM processor retrieves the first slave user interface state model corresponding to the first slave device from the master memory according to the first slave device identifier in the user operation event signal. At the same time, it extracts the timestamp stamped by the slave FPGA at the time of the operation from the user operation event signal. Using the timestamp as an index, it locates the display data of the corresponding time node in the first slave user interface state model, and accurately extracts the slave device display screen when the user operation input occurs. By centrally maintaining the interface state model on the master end, the target screen can be quickly obtained without additional interaction with the slave end, improving operation response efficiency. Relying on timestamps to achieve accurate screen backtracking can avoid matching deviations caused by transmission delays or screen updates, providing accurate scenario basis for subsequent analysis of operation intentions. At the same time, the slave end does not need to store historical display data, but only needs to execute the display instructions issued by the master end, further simplifying the hardware and software design of the slave end, reducing the risk of failure, and helping to improve the overall reliability of the multi-slave system.

[0097] Step 104: Based on the user's operation location data and the display screen on the slave end, the ARM processor parses the user's operation intent, generates a refresh image for the display screen, and sends the refresh image to the slave FPGA of the first slave end through the master FPGA and network switch.

[0098] Specifically, user operation intent refers to the function that the user expects to achieve through their actions, such as requesting to speak, adjusting volume, or switching pages. The display refresh image is standardized image data generated by the master ARM processor based on the operation intent and used to update the slave display screen; it can be in RGB or YUV format.

[0099] In some embodiments, after obtaining user operation location data and the corresponding slave display screen, the master ARM processor first matches the user operation location data with the metadata of the determined slave display screen. Touch operations are matched with the coordinate range of functional elements, and button operations are matched with a preset function mapping table to accurately parse the user's operation intent (such as requesting to speak or adjusting volume). Subsequently, the ARM processor calls the interface template according to the user's operation intent to generate a standardized display refresh image that conforms to the resolution and pixel format of the slave display screen. After adding a first slave identifier to the display refresh image, it is transmitted to the master FPGA. The master FPGA encapsulates the display refresh image into an Ethernet frame, which is converted into a differential signal by the master physical layer chip and sent to the network switch. The network switch forwards the signal to the first slave physical layer chip, and finally, the slave physical layer chip restores the data and transmits it to the first slave FPGA. By centrally parsing intents and generating images on the master end, the system transforms the user's physical operations on the slave end into business instructions that the system can understand. Based on these instructions, a new interface feedback image (i.e., a screen refresh image) is generated, and then the screen refresh image is efficiently sent to the corresponding slave end to complete the display update. This achieves a smooth, accurate, and consistent human-computer interaction experience while maintaining minimal hardware on the slave end.

[0100] Optionally, the step of generating a refreshed display image by parsing the user's operation intent through the ARM processor based on user operation location data and the display screen on the slave device includes:

[0101] The ARM processor performs coordinate matching between the user's operation location data and the location area information of multiple interface elements in the slave display screen to determine the target interface element to be triggered. The slave display screen contains the location area information of multiple interface elements and the operation functions associated with each interface element.

[0102] Based on the target operation function associated with the target interface element, the system function logic corresponding to the target operation function is executed by the ARM processor to render and generate a refreshed image of the display screen that reflects the execution result of the system function logic.

[0103] Specifically, interface elements are functional components displayed on the slave device's screen, such as the "Request to Speak" button, volume control slider, "Mute" icon, and page switching options. These interface elements are the core objects of user interaction. The location area information represents the coordinate range of each interface element in the slave device's display coordinate system (e.g., 100-200 pixels on the X-axis, 50-100 pixels on the Y-axis), used to match the user's operation position. The associated operation function of each interface element is a business logic identifier bound to it, such as "request_speak", "vote_yes", and "mute_audio", serving as the entry point for function execution. The target interface element is the specific interface element matched to the user's operation position data, i.e., the functional component actually triggered by the user. The target operation function is the pre-defined system function associated with the target interface element, such as the "Request to Speak" button corresponding to the "Initiate Speak Request" function, and the volume slider corresponding to the "Adjust Output Volume" function. The system function logic is the specific business process for implementing the target operation function.

[0104] For example, after the master ARM processor obtains the user's operation location data and the slave display screen, it first retrieves the location area information of multiple interface elements contained in the display screen and the operation functions associated with each interface element. The ARM processor compares the user's operation location data with the location area information of each interface element one by one. If the operation location falls within the coordinate range of a certain interface element, then the interface element is determined to be the triggered target interface element. Subsequently, the ARM processor calls and executes the system function logic corresponding to the target operation function associated with the target interface element. Based on the execution result of the function logic, combined with the resolution and pixel format of the slave display screen, a standardized display refresh image containing operation feedback is rendered and generated. Through precise coordinate matching, the operation intent is accurately interpreted, enabling high-precision intent recognition and dynamic interface generation to be centrally completed by the master.

[0105] Step 105: The display screen of the first slave end is driven by the slave FPGA of the first slave end to display the refreshed image.

[0106] Specifically, the refreshed image on the display screen is transmitted to the slave FPGA of the first slave end via the master FPGA and network switch, and written into the external memory of the first slave end; the slave FPGA of the first slave end reads the new data (i.e. the refreshed image on the display screen) according to the normal display cycle, completes the screen content update of the display screen, and realizes real-time linkage between touch and display.

[0107] In some embodiments, after receiving the refreshed display image from the master, the slave FPGA of the first slave end can write the refreshed display image into the external memory of the first slave end, such as RAM, through an internal DMA channel. Subsequently, the display timing controller integrated inside the slave FPGA of the first slave end automatically generates timing signals such as horizontal synchronization (HSYNC), vertical synchronization (VSYNC), and pixel clock (PCLK) according to the physical parameters of the display (such as resolution, refresh rate, and interface type). It reads the refreshed display image from the frame buffer in the scanning order and outputs the refreshed display image to the display in real time through parallel / serial interfaces such as RGB or Low-Voltage Differential Signaling (LVDS). The entire driving process is completed by the pure hardware logic of the slave FPGA of the first slave end, without the need for operating system or software intervention, and has low latency, high determinism, and strong real-time performance. By driving the display screen in pure hardware through the slave FPGA of the first slave end, only image reception and display output are performed, without the need to run an operating system or graphics rendering software. This significantly reduces the hardware and software complexity of the slave end of the system, avoids display abnormalities caused by software failures, and effectively improves the stability and reliability of the multi-slave audio and video conferencing system in long-term operation.

[0108] This application provides a control method for an audio-visual conferencing system. The system includes a master terminal, a network switch, and at least one slave terminal. The master terminal includes a master field-programmable gate array (FPGA) and an ARM processor. Each slave terminal includes a slave FPGA and a human-machine interface (HMI) peripheral device, which includes at least a display screen and a keyboard. In response to user input on the HMI peripheral device of the first slave terminal, the user's operation position data is collected through the HMI peripheral device and combined with the first slave terminal identifier. The user operation event signaling is then generated through the slave FPGA of the first slave terminal. The first slave terminal can be any slave terminal, and the user operation input can be a touch operation or a key operation. The slave terminal completes user operation data acquisition and signaling generation solely through the HMI peripheral device and the FPGA, significantly reducing the complexity of the slave terminal and significantly improving its reliability. The user operation event signaling is forwarded to the ARM processor of the master terminal via the network switch and the master FPGA. The system extracts the first slave identifier and user operation location data from the user operation event signaling using an ARM processor, and determines the slave display screen when the user operation input occurs based on the first slave identifier. Based on the user operation location data and the slave display screen, the ARM processor parses the user's operation intent, generates a screen refresh image, and sends the screen refresh image to the slave FPGA of the first slave through the master FPGA and network switch. The master FPGA acts as a communication forwarding unit, and the master ARM centrally parses all signaling. Finally, the slave FPGA of the first slave drives the first slave's screen to display the screen refresh image. The slave FPGA receives the screen refresh image sent from the master and drives the screen to display. This eliminates the need for complex logic such as operation intent parsing and display content generation, solving the technical problem of low overall system reliability in existing audio / video conferencing systems due to complex slave hardware and software. By moving the intelligent interaction logic, which is independently handled by each slave device, to the master device for centralized processing, the slave devices only retain hardware-level data acquisition and display driving functions. This greatly simplifies the hardware and software design of the slave devices, significantly improves the overall reliability of multi-slave systems, eliminates the system vulnerability caused by the excessive complexity of slave device hardware and software in existing technologies, and achieves the high availability, strong consistency and long-term stability required by professional conference systems while maintaining the user interaction experience (smooth touch response and timely screen updates).

[0109] In some embodiments, reference Figure 3 Prior to the step of collecting user operation location data through the first slave human-computer interaction peripheral device in response to user operation input on the first slave device, the method further includes:

[0110] Step 301: Generate image frame data of the display screen of the first slave end through the ARM processor, and encapsulate the image frame data and the first slave end identifier into a network data packet;

[0111] Step 302: Send network data packets to the slave FPGA of the first slave end through the master FPGA and network switch;

[0112] Step 303: The image frame data in the network data packet is parsed by the slave FPGA of the first slave end, and the image frame data is written into the external memory of the first slave end;

[0113] Step 304: The first slave FPGA reads image frame data from the external memory according to a preset refresh cycle, and drives the first slave display screen to display the image frame data.

[0114] Specifically, image frame data is the complete screen data generated by the master-end ARM processor for display on the slave-end display screen, such as YUV format, containing interface layout, functional elements, conference information, etc.; network data packets are transmission units encapsulated by the master end with the image frame data and the first slave-end identifier according to the Ethernet protocol, including frame header, data area, and checksum field. External memory is a high-speed storage medium, such as RAM, for the slave end, used to cache image frame data and ensure the continuity of display refresh. The preset refresh cycle is a fixed time interval for the slave-end FPGA to read image frame data, such as approximately 16.7ms for a 60Hz refresh rate, to match the hardware refresh requirements of the display screen.

[0115] As an example, the master ARM processor can generate standardized image frame data corresponding to the first slave display screen based on the needs of the meeting scenario, such as the initial interface layout, meeting participant information, and function button configuration. This image frame data, along with the first slave identifier, is then encapsulated into a network data packet using the Ethernet protocol, with a frame header and checksum field added to ensure transmission integrity. After the ARM processor transmits the network data packet to the master FPGA, it is converted into a differential signal by the master physical layer chip. The converted differential signal is then forwarded to the slave physical layer chip of the first slave via a network switch. The slave physical layer chip restores the differential signal to an Ethernet frame and sends the Ethernet frame to the slave FPGA. Upon receiving the Ethernet frame, the slave FPGA parses and removes redundant fields to extract the image frame data. This image frame data can then be written to the external memory buffer of the first slave via a high-speed bus, avoiding display anomalies caused by real-time transmission fluctuations. Subsequently, the slave FPGA of the first slave end reads image frame data frame by frame from external memory according to a preset refresh cycle, calls the internal display driver logic to convert it into a drive signal adapted to the display screen, such as an LVDS signal, and drives the display screen of the first slave end to complete a stable image display. By centrally generating and sending image frames by the ARM processor of the master end, and driving the display purely by the slave FPGA, the slave end does not need an operating system or graphics rendering capabilities, thereby significantly reducing its hardware and software complexity and improving the overall performance of the multi-slave system in terms of display consistency, operational stability and long-term reliability.

[0116] In some embodiments, reference Figure 4 Human-computer interaction peripherals also include other peripherals besides the display screen and keyboard. After the step of parsing the user's operation intent through the ARM processor based on user operation location data and the slave display screen, it also includes:

[0117] Step 401: Based on the user's operation intention, generate peripheral control instructions through the ARM processor. The peripheral control instructions include the target peripheral type, control parameters, and the first target slave identifier.

[0118] Step 402: Based on the first target slave identifier, send the peripheral control command to the slave FPGA of the first target slave through the master FPGA and the network switch;

[0119] Step 403: Based on the target peripheral type and control parameters, the target peripheral device of the first target slave end is driven by the slave FPGA of the first target slave end to execute the corresponding operation of the control parameters. Other peripheral devices include the target peripheral device.

[0120] Optionally, based on the target peripheral type and control parameters, the step of driving the target peripheral device of the first target slave end through the slave FPGA of the first target slave end to execute the corresponding operation of the control parameters includes:

[0121] Based on the target peripheral type and control parameters, the state of the target peripheral register corresponding to the target peripheral device in the slave FPGA of the first target slave is modified to drive the target peripheral device to execute the corresponding operation of the control parameters.

[0122] Specifically, other peripheral devices refer to human-computer interaction or function execution devices deployed on the slave end, excluding the display screen and keyboard, such as microphones, speakers, headphones, and indicator lights (e.g., light-emitting diodes, LEDs). Peripheral control instructions are standardized instructions generated by the master end's ARM processor. These instructions include the target peripheral type (e.g., microphone, speaker), control parameters (e.g., microphone on / off, speaker volume), and the first target slave identifier. They are used to precisely control the target peripheral device on a specific slave end. The control parameters are specific operational instructions for the target peripheral device, such as volume level and on / off status.

[0123] As an example, after the master ARM processor parses the user's operation intent (such as "start speaking" or "adjust volume"), it immediately generates standardized peripheral control instructions. These instructions explicitly include the target peripheral type (such as microphone or speaker), specific control parameters, and the first target slave identifier, ensuring the accuracy of the controlled object (i.e., the first target slave). Subsequently, the ARM processor transmits the peripheral control instructions to the master FPGA. The master FPGA encapsulates these instructions into data frames according to a preset protocol, converts them into differential signals by the master physical layer chip, and forwards them via a network switch based on the first target slave identifier to the slave physical layer chip of the first target slave. The slave physical layer chip of the first target slave then restores the differential signal to an Ethernet frame and sends the Ethernet frame to the slave FPGA of the first target slave. After receiving the peripheral control command, the slave FPGA of the first target end parses the target peripheral type and control parameters in the command. By modifying the state of the corresponding control register inside the slave FPGA, it drives the target peripheral device (such as a microphone, speaker, indicator light, etc.) to perform corresponding operations, such as activating the microphone, outputting audio at a set volume, and lighting up the indicator light to indicate the status, thus completing the closed loop of operation intention-peripheral response. By centrally generating precise peripheral control commands at the master end and transmitting them to the target slave FPGA via a hardware link, the target peripheral devices such as microphones and speakers can respond in unison. The slave end does not require complex control logic, simplifying hardware and software design, achieving real-time closed-loop control of operation and peripherals, enriching interactive functions, avoiding misoperation, and thus significantly improving system coordination and overall reliability.

[0124] In some embodiments, reference Figure 5 When the target peripheral device is a microphone, the steps for driving the target peripheral device of the first target slave end to execute the corresponding operation of the control parameters through the slave FPGA of the first target slave end include:

[0125] Step 501: Drive the microphone of the first target slave end through the slave FPGA of the first target slave end to perform an audio acquisition operation, so as to acquire audio signals through the microphone of the first target slave end;

[0126] After step 501, which is the step of driving the microphone of the first target slave end to perform audio acquisition operation through the slave FPGA of the first target slave end, the method further includes:

[0127] Step 502: The first target slave identifier and audio signal are packaged and processed by the slave FPGA of the first target slave end to generate the corresponding audio data packet, and the audio data packet is transmitted to the master FPGA through the network switch;

[0128] Step 503: The received audio data packets are unpacked by the main FPGA to obtain the unpacked audio stream;

[0129] Step 504: Control the main FPGA to play the unpacked audio stream through the main audio output port.

[0130] Specifically, when the target peripheral device is a microphone, the slave FPGA of the first target end parses the peripheral control command issued by the master end and sends a acquisition signal to the microphone of the first target end, driving the microphone of the first target end to start the audio acquisition operation. The microphone of the first target end converts the sound wave of the user's speech into an analog electrical signal, which is then converted into a standardized digital audio signal by the built-in ADC of the first target end and transmitted to the slave FPGA of the first target end in real time. Subsequently, the slave FPGA of the first target end packages the first target end identifier and the audio signal according to the preset audio transmission protocol, adds a frame header (including master and slave end address), a frame tail (including CRC check field) and audio parameter identifier, and generates a structured audio data packet. The audio data packet is converted into a differential signal by the slave physical layer chip of the first target end. The differential signal is forwarded to the master physical layer chip through the network switch. The master physical layer chip converts the differential signal into an audio data packet and sends the audio data packet to the master FPGA. After receiving audio data packets, the master FPGA first performs a CRC check to confirm data integrity, then unpacks the data, stripping away the encapsulation fields to extract the continuous audio stream. Finally, the master FPGA calls the audio output control logic to convert the audio stream into an analog signal conforming to the electrical specifications of the master audio output port, which is then transmitted to external speakers, audio equipment, and other playback devices for real-time audio playback. By implementing controlled microphone acquisition, tagged audio packet transmission, and master hardware decoding playback through FPGA hardware, a low-latency, highly reliable, deterministic audio path without operating system intervention is constructed. This effectively avoids software scheduling jitter and packet loss issues when multiple audio streams are concurrent, improving the real-time performance, stability, and manageability of the conference system's voice transmission.

[0131] In some embodiments, reference Figure 6 When the human-computer interaction peripherals also include microphones and audio players, the control methods for audio and video conferencing discussion systems also include:

[0132] Step 601: Based on the system configuration information, the target audio source is selected from multiple candidate audio sources by the ARM processor, and the second target candidate audio source is determined to include the microphone audio stream uploaded by the slave end or the input audio stream of the master end.

[0133] Step 602: The second target slave identifier and the target audio source of the second target slave end are encapsulated into an audio routing data packet by the ARM processor, and the audio routing data packet is sent to the master FPGA.

[0134] Step 603: Send the audio routing data packet to the slave FPGA of the second target slave end through the master FPGA and network switch;

[0135] Step 604: The audio routing data packet is received and unpacked by the slave FPGA of the second target slave end to obtain the target audio stream corresponding to the unpacked target audio source;

[0136] Step 605: Control the audio player of the second target slave end to play the target audio stream through the slave FPGA of the second target slave end.

[0137] Specifically, the system configuration information consists of pre-stored conference audio routing rules on the master end, such as basic configuration data like speaking permissions, audio playback policies, and the status of each slave device. Candidate audio sources are selectable audio data sources, including audio streams captured and uploaded by slave microphones and externally input audio streams from the master end, such as remote conference audio and background music. The target audio source is the single audio source selected by the ARM processor based on the system configuration information and to be transmitted to the second target slave for playback. The second target slave is the designated slave that needs to receive and play the target audio source; it can be single or multiple, and its identifier is used for directional routing. The audio routing data packet is a standardized transmission data packet encapsulated by the master end's ARM processor, containing the second target slave identifier and the target audio source. The audio player is the audio output device on the slave end, such as a speaker, audio player, or headphones, used to play the target audio stream.

[0138] As an example, when each slave device is equipped with a microphone and an audio player, the system can achieve dynamic audio routing and distribution. The master ARM processor first reads the system configuration information (such as conference speaking permissions, audio distribution policies, and the online status of each slave device), selects the target audio source from the candidate audio sources, including microphone audio streams uploaded by each slave device (such as participants' speech) and external input audio streams from the master device (such as remote conference audio and presentation courseware audio), and determines the second target slave device that needs to receive the target audio source based on the system configuration information. Subsequently, the ARM processor encapsulates the second target slave identifier and the target audio source into an audio routing data packet according to a preset audio transmission protocol. It also adds a frame header (containing master / slave addresses), a checksum field, and audio parameter information to ensure transmission integrity and readability. The audio routing data packet is then sent to the master FPGA. The master FPGA encapsulates the audio routing data packet into an Ethernet frame and sends the Ethernet frame to the master physical layer chip. The master physical layer chip converts the Ethernet frame into a differential signal, which is then forwarded via a network switch to the slave physical layer chip of the second target slave. The slave physical layer chip of the second target slave then converts the differential signal into a differential signal. The signal is restored to an Ethernet frame and sent to the slave FPGA of the second target slave end. After receiving the Ethernet frame, the slave FPGA of the second target slave end parses and removes redundant fields, extracts the audio routing data packet, and performs verification on the audio routing data packet. If the verification confirms that the data is correct, the audio routing data packet is unpacked, and the encapsulation fields are removed to extract the target audio stream. Finally, the slave FPGA calls the audio playback driver logic to convert the digital audio stream into an analog signal that conforms to the electrical specifications of the audio player (such as speakers, audio equipment, headphones, etc.), driving the audio player to accurately play the target audio stream, completing the audio directional distribution and playback closed loop. By centrally deciding audio routing through the ARM processor of the master end and implementing low-latency distribution and playback through FPGA hardware, the system can flexibly and reliably push the target audio source accurately to the target slave end without a slave end operating system, improving the real-time performance, strategy flexibility, and overall system stability of audio distribution.

[0139] In some embodiments, the control method for the audio and video conferencing discussion system further includes:

[0140] When the ARM processor receives a high-priority operation signaling sent by a preset high-priority device for a third target slave, the ARM processor performs authorization verification on the high-priority operation signaling. The preset high-priority device includes a master web management terminal, an external centralized control system, or a slave with chairman privileges.

[0141] After the high-priority operation signaling is verified, the ARM processor parses the high-priority operation signaling, generates high-priority control instructions, and places the high-priority control instructions into the high-priority transmission queue. The high-priority control instructions include the third target identifier, the third peripheral type, and the peripheral control parameters.

[0142] The high-priority transmission queue is preemptively scheduled by the master FPGA, and based on the third target identifier, the high-priority control commands are transmitted to the slave FPGA of the third target slave through the master FPGA and the network switch.

[0143] Based on the third peripheral type and peripheral control parameters, the state of the third target peripheral register corresponding to the third peripheral type in the third target slave FPGA is modified by the slave FPGA of the third target slave end, so as to drive the third peripheral type of human-machine interaction peripheral device to execute the corresponding operation of the peripheral control parameters.

[0144] Specifically, the preset high-priority device is the device with the highest control authority in the meeting, which can include the master web management terminal, external centralized control system, and slave devices with chairman authority. The preset high-priority device can initiate high-priority operation commands. The high-priority operation signaling is the control command sent by the preset high-priority device that needs to be executed first, such as forced mute, chairman priority speaking, emergency mute, etc. The third target slave device is the designated slave device to which the high-priority operation signaling points and which needs to perform the control operation. The high-priority control command is a standardized command generated by the ARM processor after verification, including the third target identifier, the third peripheral type, and the peripheral control parameters. The third target identifier is the third target slave ID (e.g., 07), and the peripheral control parameters can be state=OFF, color=RED, etc. The high-priority transmission queue is a dedicated instruction buffer in the master memory that is independent of the ordinary data stream. The FPGA scheduler assigns it the highest transmission priority. The third target peripheral register is the hardware register inside the slave FPGA of the third target slave device or in the external peripheral device (such as codec, GPIO controller) used to control specific human-computer interaction peripheral devices. Direct writing can change the device state of the human-computer interaction peripheral device.

[0145] As an example, in an audio / video conferencing system, a dedicated control channel is provided to support emergency or high-priority intervention by the chairperson, administrator, or external centralized control system on the conference unit (slave). When the ARM processor receives a high-priority operation signal (e.g., "force mute microphone of representative 05" or "turn on red alarm lights in all units") from a preset high-priority device via a network interface (e.g., HTTP / WebSocket) or serial interface, it first performs strict authorization verification on the high-priority operation signal, including verifying the device identity credentials, operation permission level, legality of the target scope, and validity of parameters to prevent unauthorized or unauthorized operations. Preset high-priority devices include the master web management terminal, external centralized control systems such as Crestron / AMX, or high-privilege slaves with chairperson privileges. After successful verification, the ARM processor parses the high-priority operation signal and generates a structured high-priority control instruction. This high-priority control instruction explicitly includes a third target identifier (i.e., target slave ID, such as 057), a third peripheral type, and specific peripheral control parameters (e.g., state=OFF, brightness=100%). Subsequently, the ARM controller writes the high-priority control instruction into a separately maintained high-priority transmission queue in memory, which is physically isolated from the ordinary display or audio data queue. The master FPGA continuously monitors the status of the high-priority transmission queue and is configured with hardware-level preemptive scheduling logic: once a new instruction is detected in the queue, the currently transmitting ordinary data frame is immediately interrupted, the high-priority instruction is preferentially encapsulated into an Ethernet control packet, and the destination MAC address is determined based on the third target identifier. The packet is then sent to the third target slave end via a network switch. After receiving the control packet, the slave FPGA of the third target end quickly unpacks it to extract the third peripheral type and peripheral control parameters, and directly accesses and modifies the status of the corresponding third target peripheral register in its internal or external peripheral controller. The change in register status immediately triggers the hardware driver logic, driving the human-machine interface peripheral device of the third peripheral type to accurately execute the corresponding operation, completing the full-link response of the high-priority instruction. Beyond regular user operations, through access control, high-priority queues, and FPGA preemption scheduling mechanisms, an emergency intervention channel with preemption capabilities is provided for chairpersons, administrators, or external centralized control systems. This ensures that critical commands (such as forced mute, emergency microphone shutdown, and alarm light activation) can bypass ordinary task queues and reach the target device with minimal latency and maximum reliability, thereby meeting the stringent requirements of professional conference scenarios for security, order, and emergency response.

[0146] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the control method of the audio and video conferencing discussion system of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0147] This application also provides a control device for an audio and video conferencing discussion system; please refer to [reference needed]. Figure 7 The control device for the audio and video conferencing discussion system includes:

[0148] The acquisition module 701 is used to respond to user operation input on the human-machine interaction peripheral device of the first slave end, acquire user operation position data through the human-machine interaction peripheral device of the first slave end, and generate user operation event signaling through the slave FPGA of the first slave end in combination with the first slave end identifier. The first slave end can be any slave end, and the user operation input is touch operation or button operation.

[0149] The signaling transmission module 702 is used to forward user operation event signaling to the ARM processor of the master end through the network switch and the master FPGA;

[0150] The display screen determination module 703 is used to extract the first slave identifier and user operation position data from the user operation event signaling through the ARM processor, and determine the slave display screen when the user operation input occurs based on the first slave identifier;

[0151] The refresh image generation module 704 is used to generate a refresh image of the display screen based on the user operation position data and the display screen of the slave end, by parsing the user operation intention through the ARM processor, and then sending the refresh image of the display screen to the slave FPGA of the first slave end through the master FPGA and the network switch.

[0152] The display update module 705 is used to drive the display screen of the first slave end through the slave FPGA of the first slave end to display the refreshed image of the display screen.

[0153] The control device for the audio-visual conferencing discussion system provided in this application adopts the control method of the audio-visual conferencing discussion system in the above embodiments, which can solve the technical problem of low overall reliability of existing audio-visual conferencing systems due to the complexity of the hardware and software on the slave end. Compared with the prior art, the beneficial effects of the control device for the audio-visual conferencing discussion system provided in this application are the same as the beneficial effects of the control method for the audio-visual conferencing discussion system provided in the above embodiments, and other technical features in the control device for the audio-visual conferencing discussion system are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0154] This application provides a control device for an audio-visual conferencing discussion system. The control device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the control method of the audio-visual conferencing discussion system in the first embodiment described above.

[0155] The following is for reference. Figure 8 The diagram illustrates a structural schematic of a control device suitable for implementing an audio-visual conferencing discussion system according to embodiments of this application. The control device for the audio-visual conferencing discussion system in these embodiments may include, but is not limited to, mobile terminals such as laptops, tablets (Portable Application Description, PADs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 8 The control device of the audio and video conferencing discussion system shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0156] like Figure 8 As shown, the control device of the audio-visual conferencing system may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to programs stored in read-only memory (ROM) 1002 or programs loaded from storage device 1003 into random access memory (RAM) 1004. The random access memory 1004 also stores various programs and data required for the operation of the control device of the audio-visual conferencing system. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the control equipment of the audio / video conferencing discussion system to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows the control equipment of an audio / video conferencing discussion system with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0157] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0158] The control device for the audio-visual conferencing discussion system provided in this application adopts the control method of the audio-visual conferencing discussion system in the above embodiments, which can solve the technical problem of low overall reliability of existing audio-visual conferencing systems due to the complexity of the hardware and software on the slave end. Compared with the prior art, the beneficial effects of the control device for the audio-visual conferencing discussion system provided in this application are the same as the beneficial effects of the control method for the audio-visual conferencing discussion system provided in the above embodiments, and other technical features in the control device for the audio-visual conferencing discussion system are the same as the features disclosed in the method of the previous embodiment, and will not be repeated here.

[0159] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0160] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0161] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the control method of the audio and video conferencing discussion system in the above embodiments.

[0162] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0163] The aforementioned computer-readable storage medium may be included in the control device of the audio / video conferencing discussion system; or it may exist independently and not be assembled into the control device of the audio / video conferencing discussion system.

[0164] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the control device of the audio-visual conferencing system, the control device of the audio-visual conferencing system causes the following actions: In response to user input from a user on a human-machine interface peripheral device at the first slave end, the control device collects user operation location data through the first slave end's human-machine interface peripheral device and, in conjunction with a first slave end identifier, generates user operation event signaling through the slave end's FPGA. The first slave end can be any slave end, and the user operation input can be a touch operation or a button operation. The control device forwards the user operation event signaling to the ARM processor at the master end through a network switch and the master end's FPGA. The ARM processor extracts the first slave end identifier and user operation location data from the user operation event signaling and determines the slave end's display screen at the time of the user operation input based on the first slave end identifier. Based on the user operation location data and the slave end's display screen, the ARM processor parses the user's operation intent, generates a screen refresh image, and sends the screen refresh image to the slave end's FPGA through the master end's FPGA and the network switch. The slave end's FPGA drives the first slave end's display screen to display the screen refresh image.

[0165] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0166] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0167] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0168] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the control method of the above-described audio-visual conferencing discussion system. This solves the technical problem of low overall reliability of multi-slave systems due to the complexity of the hardware and software on the slave end of existing audio-visual conferencing systems. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the control method of the audio-visual conferencing discussion system provided in the above embodiments, and will not be repeated here.

[0169] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the control method for the audio and video conferencing discussion system described above.

[0170] The computer program product provided in this application can solve the technical problem of low overall reliability of existing audio and video conferencing systems due to the complexity of the hardware and software on the slave end. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the control method of the audio and video conferencing discussion system provided in the above embodiments, and will not be repeated here.

[0171] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A control method for an audio-visual conferencing discussion system, characterized in that, An audio-visual conferencing discussion system is provided, comprising a master terminal, a network switch, and at least one slave terminal. The master terminal includes a master FPGA and an ARM processor. Each slave terminal includes a slave FPGA and a human-computer interaction peripheral device, which includes at least a display screen and a keyboard. The control method for the audio-visual conferencing discussion system includes: In response to user operation input on the human-computer interaction peripheral device of the first slave end, user operation position data is collected through the human-computer interaction peripheral device of the first slave end, and combined with the first slave end identifier, user operation event signaling is generated through the slave end FPGA of the first slave end. The first slave end can be any of the slave ends, and the user operation input is a touch operation or a button operation. The user operation event signaling is forwarded to the ARM processor of the main terminal through the network switch and the main FPGA; The ARM processor extracts the first slave identifier and the user operation location data from the user operation event signaling, and determines the slave display screen when the user operation input occurs based on the first slave identifier; Based on the user operation location data and the slave display screen, the ARM processor parses the user operation intention, generates a screen refresh image, and sends the screen refresh image to the slave FPGA of the first slave through the master FPGA and the network switch. The first slave FPGA drives the first slave display to show the refreshed image on the display screen.

2. The control method for the audio and video conferencing discussion system as described in claim 1, characterized in that, Before the step of collecting user operation location data through the first slave-end human-computer interaction peripheral device in response to user operation input on the first slave-end human-computer interaction peripheral device, the method further includes: The ARM processor generates image frame data for the display screen of the first slave end, and encapsulates the image frame data and the first slave end identifier into a network data packet; The network data packets are sent to the slave FPGA of the first slave end through the master FPGA and the network switch; The image frame data in the network data packet is parsed by the slave FPGA of the first slave end, and the image frame data is written into the external memory of the first slave end; The first slave FPGA reads image frame data from the external memory according to a preset refresh cycle, and drives the display screen of the first slave to display the image frame data.

3. The control method for the audio and video conferencing discussion system as described in claim 1, characterized in that, The human-computer interaction peripheral device also includes other peripheral devices besides the display screen and keyboard. After the step of parsing the user's operation intent through the ARM processor based on the user's operation position data and the slave display screen, it further includes: Based on the user's operation intention, the ARM processor generates peripheral control instructions, which include the target peripheral type, control parameters, and a first target slave identifier. Based on the first target slave identifier, the peripheral control command is sent to the slave FPGA of the first target slave through the master FPGA and the network switch; Based on the target peripheral type and the control parameters, the target peripheral device of the first target slave end is driven by the slave FPGA of the first target slave end to execute the corresponding operation of the control parameters, and the other peripheral devices include the target peripheral device.

4. The control method for the audio and video conferencing discussion system as described in claim 3, characterized in that, When the target peripheral device is a microphone, the step of driving the target peripheral device of the first target slave end to execute the corresponding operation of the control parameters through the slave FPGA of the first target slave end includes: The first target slave terminal's microphone is driven by the slave FPGA of the first target slave terminal to perform an audio acquisition operation, so as to acquire audio signals through the microphone of the first target slave terminal; After the step of driving the microphone of the first target slave end to perform audio acquisition operation through the slave FPGA of the first target slave end, the method further includes: The first target slave identifier and the audio signal are packaged and processed by the slave FPGA of the first target slave to generate a corresponding audio data packet, and the audio data packet is transmitted to the master FPGA through the network switch; The received audio data packets are unpacked by the main FPGA to obtain the unpacked audio stream. The main FPGA controls the main audio output port to play the unpacked audio stream.

5. The control method for the audio and video conferencing discussion system as described in claim 1, characterized in that, When the human-computer interaction peripheral device further includes a microphone and an audio player, the control method of the audio and video conferencing discussion system further includes: Based on system configuration information, the ARM processor selects a target audio source from multiple candidate audio sources, and determines that the candidate audio source includes the microphone audio stream uploaded by the slave end or the input audio stream from the master end. The ARM processor encapsulates the second target slave identifier and the target audio source into an audio routing data packet, and sends the audio routing data packet to the master FPGA. The audio routing data packet is sent to the slave FPGA of the second target slave end through the master FPGA and the network switch; The second target slave FPGA receives and unpacks the audio routing data packet to obtain the target audio stream corresponding to the target audio source after unpacking. The target audio stream is played by the audio player of the second target slave terminal controlled by the slave FPGA of the second target slave terminal.

6. The control method for the audio and video conferencing discussion system as described in claim 1, characterized in that, The control method for the audio and video conferencing discussion system also includes: When the ARM processor receives a high-priority operation signaling sent by a preset high-priority device for a third target slave, the ARM processor performs authorization verification on the high-priority operation signaling. The preset high-priority device includes a master web management terminal, an external centralized control system, or a slave with chairman authority. After the high-priority operation signaling is verified, the ARM processor parses the high-priority operation signaling, generates a high-priority control instruction, and places the high-priority control instruction into a high-priority transmission queue. The high-priority control instruction includes a third target identifier, a third peripheral type, and peripheral control parameters. The high-priority transmission queue is preemptively scheduled by the master FPGA, and the high-priority control command is transmitted to the slave FPGA of the third target slave through the master FPGA and the network switch based on the third target identifier. Based on the third peripheral type and the peripheral control parameters, the state of the third target peripheral register corresponding to the human-machine interaction peripheral device of the third peripheral type in the slave FPGA of the third target slave end is modified by the slave FPGA of the third target slave end, so as to drive the human-machine interaction peripheral device of the third peripheral type to execute the corresponding operation of the peripheral control parameters.

7. The control method for the audio and video conferencing discussion system as described in claim 1, characterized in that, The step of determining the slave display screen when the user operation input occurs based on the first slave identifier includes: Based on the first slave identifier and the user interface state model maintained by the ARM processor in the master memory that is synchronized with the actual display content of each slave, the ARM processor retrieves the first slave user interface state model corresponding to the first slave from the master memory. The ARM processor extracts the timestamp of the user operation input from the user operation event signaling, and based on the timestamp, extracts the slave display screen from the first slave user interface state model.

8. The control method for the audio and video conferencing discussion system as described in claim 1, characterized in that, The step of generating a refresh image for the display screen by parsing the user's operation intent through the ARM processor based on the user's operation location data and the slave display screen includes: The ARM processor performs coordinate matching between the user operation location data and the location area information of multiple interface elements in the slave display screen to determine the target interface element to be triggered. The slave display screen includes the location area information of multiple interface elements and the operation functions associated with each interface element. Based on the target operation function associated with the target interface element, the system function logic corresponding to the target operation function is executed by the ARM processor to render and generate a screen refresh image that reflects the execution result of the system function logic.

9. A control device for an audio-visual conferencing discussion system, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the control method for the audio-visual conferencing discussion system as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the control method of the audio and video conferencing discussion system as described in any one of claims 1 to 8.

Images