Video conferencing system and method based on FPGA and CPU shared cache
By adopting a shared cache architecture between FPGA and CPU, the video conferencing system achieves low latency, high synchronization, and visual point-and-click switching, solving the problems of hardware resource redundancy and high cost in existing technologies for multi-channel video input concurrent scenarios, and supporting seamless switching and synchronous display of multiple video inputs.
Patent Information
- Application Number
- CN202611102739.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-23
- Publication Date
- 2026-11-17
AI Technical Summary
Existing video conferencing systems struggle to achieve low latency, high synchronization, and visual point-and-click switching in scenarios with multiple concurrent video inputs, and also suffer from hardware resource redundancy and high costs.
The system adopts an FPGA and CPU shared cache architecture. The FPGA is responsible for the parallel acquisition and preprocessing of multiple video signals, generating video frames in a unified format and writing them into the shared cache. The CPU is responsible for generating a visual preview interface and switching instructions. The switching operation is executed through the FPGA, and seamless switching is achieved by using previous frame preloading and fade-in/fade-out algorithms.
It achieves seamless frame-level switching of multiple video inputs, ultra-low latency, visual operation, reduced hardware costs and power consumption, supports multiple concurrent inputs, ensures synchronization between preview and output screens, and enhances the meeting operation experience.
Smart Images

Figure CN123123533A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image communication technology, and in particular to a video conferencing system and method based on FPGA and CPU shared cache. Background Technology
[0002] With the widespread adoption of large and medium-sized video conferencing applications, multi-channel video access, visual control, and low-latency video switching have become core requirements. Currently, mainstream video conferencing switching equipment and heterogeneous video processing solutions in the industry fall into three categories, all of which have significant technical shortcomings, as detailed below: 1. Pure FPGA Video Switching Equipment: This type of equipment relies on an FPGA hardware pipeline to achieve video acquisition, synthesis, image overlay, and output. It boasts strong real-time performance, low latency, and can support multiple high frame rate video streams concurrently. However, the equipment only possesses low-level video processing capabilities and cannot achieve multi-channel sub-screen splitting, visual preview interface generation, or human-computer interaction. Operators can only switch channels using channel numbers. In conference scenarios with multiple video input terminals participating concurrently, this is highly prone to misoperation and fails to meet the visualization and convenient operation requirements of modern video conferencing.
[0003] 2. Pure CPU / ARM architecture video processing equipment: This type of equipment can realize functions such as preview interface rendering, human-computer interaction, and command issuance. However, due to the limited number of processor video input interfaces and computing power bottlenecks, when facing 6 or more 1080P and 4K high frame rate HDMI videos to be captured, synthesized and switched at the same time, there are generally problems such as high screen latency, stuttering, frame dropping and poor synchronization of multiple screens, which makes it difficult to ensure the stable operation of large and medium-sized video conferences.
[0004] 3. Split Preview + Main Output Solution: This solution configures independent acquisition links for the preview video stream and the main output video stream, resulting in redundant hardware resources, complex system architecture, and high hardware costs. At the same time, the independent acquisition of dual links will cause inconsistencies in timing between the preview screen and the main output screen in the meeting room, as well as large delay deviations. The host cannot accurately predict the switching effect through the preview screen, which seriously affects the meeting operation experience.
[0005] In summary, existing technologies generally suffer from the technical contradiction of being unable to balance real-time processing performance and visual interaction capabilities, and lack a video conferencing solution that features low latency, high synchronization, visual point-and-click switching, and simultaneous support for multiple concurrent inputs. Summary of the Invention
[0006] The purpose of this invention is to provide a video conferencing system and method based on FPGA and CPU shared cache, in order to solve the above-mentioned problems in the prior art.
[0007] This invention provides a video conferencing system based on a shared cache between an FPGA and a CPU, comprising: The FPGA processing unit is used to acquire multiple video source signals in parallel, preprocess the multiple video source signals to generate multiple video frames of the same source with uniform format, receive switching instructions, and perform switching operations between the output channel and the video source based on the switching instructions. A shared cache unit, connected to the FPGA processing unit and the CPU processing unit, is used to cache the same source video frames written by the FPGA processing unit and provide them to the CPU processing unit for reading. The CPU processing unit is used to read the same source video frames from the shared cache unit and generate a visual preview interface, and to generate the switching instruction in response to the operation instruction input by the external input and send it to the FPGA processing unit; The FPGA processing unit is the only processing unit in the system that acquires the signals from the multiple video sources, while the CPU processing unit has no video signal acquisition interface.
[0008] This invention provides a video conferencing method based on a shared cache between an FPGA and a CPU, comprising: The FPGA processing unit acquires multiple video source signals in parallel, preprocesses the multiple video source signals to generate multiple homogeneous video frames with a unified format, and writes the homogeneous video frames into a shared buffer unit. The CPU processing unit reads the same source video frames from the shared cache unit, generates and displays a visual preview interface based on the same source video frames, generates a switching instruction in response to an external input operation instruction, and sends the switching instruction to the FPGA processing unit. The FPGA processing unit receives the switching instruction and performs a switching operation between the output channel and the video source based on the switching instruction. The FPGA processing unit is the only processing unit in the system that acquires the signals from the multiple video sources, while the CPU processing unit has no video signal acquisition interface.
[0009] This invention also provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the steps of the video conferencing method based on FPGA and CPU shared cache described above.
[0010] This invention also provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, implements the steps of the video conferencing method based on FPGA and CPU shared cache.
[0011] The embodiments of this invention can include the following beneficial effects: This invention proposes a conference system and method based on FPGA-based multi-channel video full-acquisition and same-source preview switching. This system is implemented using an FPGA+CPU heterogeneous architecture. The FPGA, as the system's sole original video acquisition and hardware processing unit, independently completes all multi-channel video signal acquisition, timing recovery, global frame synchronization, color space unification, video scaling and compositing, etc. The FPGA stores the processed same-source video frames in a shared buffer unit. The CPU reads video data from the shared buffer, without participating in any original video acquisition, decoding, or timing processing. It is responsible for splitting sub-screens, generating an input visual preview interface, receiving human-machine operations, and issuing switching commands. Operators can trigger switching by clicking on the preview sub-screen. The FPGA switches the video input source bound to the main / auxiliary output ports according to the command (rather than adjusting the window layout). Relying on same-source frame multiplexing technology, it achieves zero-frame-difference synchronization between the preview and output screens, and combined with a frame-level transition algorithm, it achieves ultra-low latency seamless switching. Attached Figure Description
[0012] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0013] Figure 1 This is a schematic diagram of a video conferencing system based on a shared cache between an FPGA and a CPU, according to an embodiment of the present invention. Figure 2 This is a block diagram of an FPGA-based multi-channel video full-acquisition same-source preview switching conference system according to an embodiment of the present invention; Figure 3 This is a business process diagram of the FPGA-based multi-channel video full-acquisition same-source preview switching conference system according to an embodiment of the present invention; Figure 4 This is a flowchart of a video conferencing method based on FPGA and CPU shared cache according to an embodiment of the present invention. Detailed Implementation
[0014] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.
[0015] System Implementation Examples According to embodiments of the present invention, a video conferencing system based on a shared cache between an FPGA and a CPU is provided. Figure 1 This is a schematic diagram of a video conferencing system based on a shared cache between FPGA and CPU, according to an embodiment of the present invention. Figure 1 As shown, the video conferencing system based on FPGA and CPU shared cache according to an embodiment of the present invention specifically includes: The FPGA processing unit 10 is used to acquire multiple video source signals in parallel, preprocess the multiple video source signals to generate multiple video frames of the same source with uniform format, and receive switching instructions, and perform switching operations between the output channel and the video source based on the switching instructions; specifically including: The acquisition interface subunit is used to receive multiple video source signals in parallel; The timing recovery subunit is connected to the acquisition interface subunit and is used to correct the line synchronization signal deviation and field synchronization signal deviation of each video source signal respectively. The frame synchronization subunit, connected to the timing recovery subunit, is used to align the video frames of each video source signal to the same clock reference point using the system's unified reference clock as the standard. The format conversion subunit, connected to the frame synchronization subunit, is used to convert the video source signals of each channel into a standard color space format; The scaling and compositing subunit, connected to the format conversion subunit, is used to scale the video source signals of each channel and combine the scaled video source signals into a single video frame from the same source.
[0016] A shared cache unit 12, connected to the FPGA processing unit and the CPU processing unit, is used to cache the same-source video frames written by the FPGA processing unit and provide them to the CPU processing unit for reading; specifically, it includes: A high-speed cache partition, connected to the FPGA processing unit and the CPU processing unit, is used to cache real-time video frames; A large-capacity cache partition, connected to the FPGA processing unit and the CPU processing unit, is used to cache historical video frames and composited preview video frames; The write arbitration subunit is connected to the high-speed cache partition and the large-capacity cache partition. When multiple source video frames corresponding to multiple video source signals are simultaneously written to the shared cache unit, the write requests are sorted according to the preset channel priority, and each channel is authorized in sequence to write the source video frames to the high-speed cache partition or the large-capacity cache partition. The amount of data continuously written after a single authorization does not exceed a preset threshold.
[0017] CPU processing unit 14 is configured to read the same source video frames from the shared cache unit and generate a visual preview interface, and to generate the switching instruction in response to an externally input operation instruction and send it to the FPGA processing unit; specifically including: A cache reading subunit, connected to the shared cache unit, is used to read the same-source video frames from the shared cache unit; The preview generation subunit, connected to the cache reading subunit, is used to split the same source video frame into multiple sub-screens according to the preset preview layout; wherein each sub-screen uniquely corresponds to one video source signal; and the multiple sub-screens are adapted to the display size respectively, and the adapted sub-screens are combined into a visual preview interface; An interactive response subunit, connected to the preview generation subunit, is used to receive external input operation instructions, determine the target video source based on the operation instructions, generate a switching instruction based on the target video source, and send it to the FPGA processing unit.
[0018] The FPGA processing unit is the only processing unit in the system that acquires the signals from the multiple video sources, while the CPU processing unit has no video signal acquisition interface.
[0019] The FPGA processing unit performs a switching operation between the output channel and the video source based on the switching instruction, specifically including: The FPGA processing unit responds to the switching command and preloads the frame of the target video source indicated by the switching command into the output buffer, maintaining the normal output of the current output frame during the preloading period; After preloading is complete, a fade-in / fade-out transition is performed to smoothly switch between the current output screen and the screen of the target video source. The total time from the moment the switching command is received to the completion of the fade-in / fade-out transition does not exceed one video frame cycle.
[0020] When the FPGA processing unit writes the same source video frame into the shared buffer unit, it adds the same frame sequence number and acquisition timestamp to the main video frame, auxiliary video frame and preview video frame at the same acquisition time to identify that the three come from the same acquisition time.
[0021] After the CPU processing unit reads the same source video frame from the shared cache unit, it verifies whether the frame sequence numbers of the currently read main video frame, auxiliary video frame and preview video frame are consistent. If they match, then the currently read video frame will be used normally. If there is a discrepancy, the currently read video frame is discarded.
[0022] The following describes in detail the above-mentioned technical solutions of the present invention with reference to the specific circumstances of the video conferencing system based on FPGA and CPU shared cache in the embodiments of the present invention.
[0023] like Figure 2 As shown, this system mainly consists of three core units: FPGA real-time video processing unit, shared cache unit, and CPU preview control unit. Each unit has a clear division of labor and is independently decoupled.
[0024] 1. FPGA Real-time Video Processing Unit The FPGA real-time video processing unit is the only original video input and hardware processing core of this system, integrating multiple video acquisition interfaces and internal hardware logic modules.
[0025] A. Hardware Interface It integrates multiple video acquisition interfaces (HDMI, DVI, SDI and other common ultra-high-definition and high-definition signal interfaces) and high-speed inter-board communication interfaces; B. Internal Logic Module Timing recovery module, global frame synchronization module, color space conversion module, video scaling and compositing module, seamless switching module, frame transition processing module; C. Core Functions a. Parallel acquisition of multiple video input signals; b. Correct timing deviations in various video signals and achieve global frame synchronization based on the unified conference clock; c. Standardize the color space format of all video signals to eliminate color differences in the image; d. Scale and split the video frames to generate homogeneous video frames in a unified format; e. Write the same source video frames to the shared buffer unit; f. Receive the switching command from the CPU, preload the target channel image into the output buffer, and then use a preloading + fade-in / fade-out algorithm to complete a seamless frame-level switching, driving the main and auxiliary output interfaces to output video images. The timing relationship between preloading and fade-in / fade-out is as follows: after the switching command arrives, the target channel image preloading and the current image output are performed in parallel. After preloading is complete, a fade-in / fade-out transition is initiated, and after the transition is complete, the channel binding switch is finished. The entire process is strictly controlled to be completed within one frame period.
[0026] 2. Shared cache unit Located between the FPGA and the CPU, it enables cross-chip video data sharing, and is divided into two independent partitions: on-chip high-speed cache (BRAM) and off-chip DDR cache, configured with dedicated read / write arbitration and conflict avoidance strategies. A. On-chip cache It stores real-time video frames with high read and write speeds, ensuring real-time screen refresh. B. Off-chip DDR cache It stores historical video frames and synthesizes preview video frames, has a large cache capacity, supports multi-channel concurrent data reading and writing, and meets the needs of cross-chip video data sharing between FPGA and CPU; C. Scheduling Strategy For multi-channel concurrent video, a time-division read / write queue is set up, and a weighted round-robin scheduling algorithm is used to authorize bus access rights in order of channel priority. After a single authorization, a fixed length of data is continuously written in bursts. The bus occupancy time of a single channel does not exceed the preset maximum time slot, ensuring that low-priority channels are not starved, while high-priority channels get sufficient bandwidth. It automatically avoids data conflicts, frame drops, and screen distortion caused by simultaneous read / write of multiple channels.
[0027] 3. CPU Preview Control Unit It has no original video acquisition interface, does not participate in video input signal decoding or timing calculation, and is only responsible for human-computer interaction and command control.
[0028] A. Hardware peripherals Touchscreen display, inter-board communication, video output interface; B. Internal Core Module The module includes a cache reading module, a preview sub-screen splitting module, a visualization control module, a video encoding / decoding module, and a video output module. C. Core Functions a. Periodically read preview, main video, and auxiliary video frame data from the shared cache unit; b. Split the entire video frame into multiple sub-frames, and bind each sub-frame to one video input source; c. Combine sub-screens to generate a visual preview interface and display it on the touch screen; d. Real-time monitoring of touch click operations, parsing of target video source IDs, generation of standard switching instructions and sending them to the FPGA; e. Bind local primary and secondary video inputs, encode the video, and transmit it to the remote end.
[0029] like Figure 3 As shown, this method is applied to a multi-channel video conferencing system based on FPGA full acquisition for simultaneous preview and seamless switching of video sources. The overall logic follows a closed loop: hardware acquisition → unified processing → cache sharing → preview rendering → human-computer interaction → instruction execution. The system uses the FPGA as the sole core for original video acquisition and hardware processing, while the CPU is only responsible for image preview and human-computer instruction parsing. A simultaneous video frame reuse mechanism is used throughout to ensure complete consistency in timing between the preview, main output, and auxiliary output images. It supports concurrent input of multiple 1080P / 4K high frame rate videos, and features frame-level seamless switching, anti-flicker, and multi-channel concurrent conflict avoidance capabilities. The specific implementation steps are as follows: 1. After the system is powered on and started, the initialization process is executed, which sequentially executes the initialization of the FPGA video real-time processing unit, the shared buffer unit, and the CPU preview control unit. After the initialization is completed, the business process is entered.
[0030] 2. FPGA real-time video processing unit initialization includes hardware interface acquisition initialization, clock system initialization, internal logic module initialization, and resource initialization: Hardware interface initialization: Configure video input interface level and timing parameters.
[0031] Clock system initialization: Start the system global clock and video synchronization clock to generate a unified reference clock for video conferencing, which serves as the reference for the synchronization of all video frames. The unified reference clock source is the FPGA's local high-precision crystal oscillator. After frequency multiplication and division by a phase-locked loop, multiple clocks with the same source are output to drive video acquisition, frame synchronization, and output interfaces respectively, ensuring that the clocks of the entire system are from the same source.
[0032] Internal logic module initialization: sequentially enable the timing recovery module, global frame synchronization module, color space conversion module, video scaling and compositing module, seamless switching module, and frame transition module, and reset the internal registers and state machine.
[0033] Resource initialization: Clear the internal data buffer, initialize the channel status table, and mark all video input channels as idle and not connected.
[0034] 3. After the FPGA initialization is complete, a system ready message is sent to the CPU.
[0035] 4. Shared memory unit initialization, mainly including partitioning, read / write strategy initialization, and cache initialization: Partitioning: The cache is divided into two independent partitions: the on-chip high-speed BRAM cache of the FPGA and the off-chip large-capacity DDR cache.
[0036] Read / write strategy initialization: Start the read / write arbitration module, time-sharing read / write queue, and data conflict detection module, and configure the concurrent read / write priority rules for multiple video signals. The priority rules are as follows: the video channel number is used as the priority basis, and the smaller the number, the higher the priority; write operations take precedence over read operations within the same channel; and the maximum length of a single burst write is set to ensure that the bus occupancy time of any channel at one time does not exceed the preset maximum time slot.
[0037] Cache initialization: Clear all historical data in the cache area and set the read and write pointers to the initial position.
[0038] 5. CPU preview control unit initialization, mainly including communication interface initialization, display driver initialization, and interaction module initialization: Communication interface initialization: Configure the high-speed inter-board bus that interfaces with the FPGA and establish a bidirectional data and command communication link.
[0039] Display driver initialization: Start the touch screen driver, UI, WEB rendering engine and other UI drivers, and load the basic layout template of the preview interface.
[0040] Interaction module initialization: Enable touch click, UI interaction, WEB and other interaction detection and instruction encapsulation modules, and reset the channel mapping table.
[0041] 6. After the CPU is ready, check the FPGA ready message. Once the FPGA is ready, configure the FPGA channel mapping table, video preview synthesis parameter configuration, main video, and auxiliary video input channels.
[0042] Channel mapping table: Ensures consistent video signal mapping relationships between FPGA acquisition, CPU preview sub-screen, and main / auxiliary video switching.
[0043] Video preview compositing parameters: Ensure a one-to-one mapping between CPU preview sub-frames and video input channels.
[0044] 7. The CPU configures the video preview compositing parameters to the FPGA according to the preset preview interface basic layout. The parameters include [compositing mode, number of composite frames, compositing resolution, [channel sub-frame 1 [compositing position, sub-frame width, sub-frame height], channel sub-frame 2 [compositing position, sub-frame width, sub-frame height], ..., channel sub-frame n [compositing position, sub-frame width, sub-frame height]]].
[0045] 8. FPGA parallel acquisition and processing: After the FPGA detects a valid video signal in a channel, it updates the channel status table to show that it is online and normal, and synchronously updates the input channel status register to provide the CPU with the signal status and input signal resolution.
[0046] 9. The FPGA acquires the original HDMI video streams of all online channels in parallel through a hardware pipeline, and then processes them sequentially through timing recovery processing → global frame synchronization → color space conversion to generate multiple source video frames with unified timing and format.
[0047] Timing recovery processing: For timing deviations in video signals from different terminals, the timing recovery module corrects the horizontal and vertical synchronization signals to unify the basic timing of each video channel.
[0048] Global frame synchronization: Using a unified reference clock for video conferencing as the standard, all video frames are aligned to ensure that all channels output complete video frames at the same time.
[0049] Color space conversion: Converts various video signals of different formats into a standard format, such as YUV422, to eliminate differences in color and color gamut.
[0050] 10. The FPGA scales, fills, and synthesizes preview video frames for each channel video signal according to the video preview synthesis parameters. When multiple video signals are written simultaneously, the read / write arbitration module enables a time-division read / write queue and writes sequentially according to channel priority. A single continuous write to a single channel does not exceed the preset maximum burst length. After a single write is completed, the bus is released and the channel is rotated to the next priority channel. Multiple channels are prohibited from simultaneously occupying the buffer bus to avoid frame drops and screen distortion.
[0051] 11. According to the cache partitioning rules, the FPGA splits the main video, auxiliary video, and preview frames and writes them into the shared memory. The FPGA adds a frame sequence number, a collection timestamp (with a unified reference clock) and the corresponding channel number to each frame of data before writing it. The frame sequence number increases monotonically in the collection order, and the collection timestamp is accurate to the pixel clock cycle. The three sets of data (main, auxiliary, and preview) use the same frame sequence number to identify that they come from the same collection time. After all the data of a single frame is written to the cache, the FPGA sends a cache data update notification to the CPU through the inter-board bus.
[0052] 12. The CPU reads the input signal status and input resolution of each channel in real time and provides a UI interface to display the real-time signal status.
[0053] 13. After receiving the cache update notification, the CPU actively initiates a read command to read the main video frame, auxiliary video frame, and preview video frame from the external DDR cache. After reading, the CPU verifies the frame sequence number and timestamp of the read frame to confirm that the sequence numbers of the main, auxiliary, and preview frames are consistent. If they are consistent, they can be used normally; if the sequence numbers are inconsistent, the batch is discarded and the CPU waits for the next frame update notification to ensure that the preview frame used by the CPU and the main / auxiliary frame currently output by the FPGA strictly come from the same acquisition time. The main and auxiliary video frames are used for video encoding and local signal display.
[0054] 14. The CPU, based on the preset preview layout, splits the preview video frames into independent sub-frames. Each sub-frame is uniquely bound to a single video input channel, and sub-frame coordinates are established. There is a one-to-one mapping relationship between video channel IDs, and channels without signal are filled with image content from the CPU's internal non-signal system.
[0055] 15. Adapt the size of the segmented sub-screens to the touch screen, UI, or WEB preview, and synchronize each sub-screen to the UI visual preview interface.
[0056] 16. The CPU monitors the UI video switching operation. When the main and auxiliary video switching signals are triggered, the CPU configures the switching parameters in the FPGA switching register. The switching parameter is the channel ID.
[0057] 17. The FPGA monitors the switching register. When a switching signal is received, it updates the current main output channel flag and performs a video source switching action.
[0058] 18. The FPGA employs a pre-loading of the previous frame plus a fade-in / fade-out transition algorithm to achieve seamless switching within one frame cycle, without black screens or flickering. The specific timing is as follows: After the switching command arrives, the pre-loading of the target channel image is immediately initiated (the current frame or the next frame of the target channel is written to the output buffer). During the pre-loading period, the current output image continues to output normally. After the pre-loading is completed, the fade-in / fade-out transition is initiated. The transparency of the original output image gradually decreases from 100% to 0%, while the transparency of the target channel image gradually increases from 0% to 100%. The transition duration is a preset N pixel clock cycles (N is set according to the frame rate to ensure that the transition completion time falls exactly within the frame blanking period). After the transition is completed, the FPGA officially binds the channel port to the new target video input source. If the pre-loading is not completed, the switching action waits until the pre-loading is ready before initiating the transition, ensuring that there are no frame losses during the switching process.
[0059] Status broadcast: After the switch is completed, the FPGA updates the global output status and sends a successful switch response message to the CPU.
[0060] Performance constraints: The entire switching action must be strictly controlled within one video frame cycle (≤16.7ms for 1080P@60fps scenes).
[0061] 19. After receiving the FPGA switching success response, the CPU synchronously updates the video channel selection status on the system interface; at the same time, the CPU associates the switching result with the current output frame sequence number and records it in the local log for subsequent anomaly tracing and system status monitoring.
[0062] 20. The system returns to step 8 and continues to execute the next round of video acquisition, preprocessing, cache writing, and preview refresh process.
[0063] 21. Full-process guarantee: The preview screen, main output screen, and auxiliary output screen always use the same original source frame. This is ensured by the same frame sequence number added in step 11 and the frame sequence number verification mechanism in step 13, so that the frame difference is always 0.
[0064] Method Implementation Examples According to embodiments of the present invention, a video conferencing method based on a shared cache between an FPGA and a CPU is provided. Figure 4 This is a flowchart of a video conferencing method based on a shared cache between FPGA and CPU according to an embodiment of the present invention, as shown below. Figure 4 As shown, the video conferencing method based on FPGA and CPU shared cache according to an embodiment of the present invention specifically includes: Step S401: The FPGA processing unit acquires multiple video source signals in parallel, preprocesses the multiple video source signals to generate multiple homogeneous video frames with uniform format, and writes the homogeneous video frames into a shared buffer unit. Step S402: The CPU processing unit reads the same source video frame from the shared cache unit, generates and displays a visual preview interface based on the same source video frame, generates a switching instruction in response to an external input operation instruction, and sends the switching instruction to the FPGA processing unit. Step S403: Receive the switching instruction through the FPGA processing unit, and perform a switching operation between the output channel and the video source based on the switching instruction; The FPGA processing unit is the only processing unit in the system that acquires the signals from the multiple video sources, while the CPU processing unit has no video signal acquisition interface.
[0065] The embodiments of the present invention are method embodiments corresponding to the system embodiments described above. The specific operations of each step can be understood by referring to the description of the system embodiments, and will not be repeated here.
[0066] In summary, compared with the prior art, the beneficial effects of the embodiments of the present invention include: 1. Seamless frame-level switching with ultra-low latency: Relying on FPGA hardware pipeline + previous frame preloading and fade-in / fade-out transition algorithms, the video source switching latency is controlled within one video frame cycle. There is no black screen, flickering, or stuttering during the switching process, which meets the high real-time requirements of video conferencing. Among them, previous frame preloading ensures that the target channel image is ready in the output buffer before switching, and together with the fade-in / fade-out transition, it ensures that there is no frame loss during the switching process. 2. High-level video synchronization: The entire system performs video capture only once. The preview stream, main output stream, and auxiliary output stream all use the original frames captured at the same time. The frame difference between the preview and output is 0, which completely solves the problem of asynchrony in split dual acquisition links. When the FPGA writes each frame to the buffer, it adds a frame sequence number and a capture timestamp. When the CPU reads the frame, it verifies the frame sequence number to ensure that the read preview frame and the current main / auxiliary output frame strictly correspond to the same capture time. At the same time, multiplexing of one acquisition reduces the consumption of encoding and decoding resources and lowers hardware costs and device power consumption. 3. Visualized operation, zero accidental touches: Abandoning the traditional channel number switching, direct WYSIWYG point-and-click switching; 4. High concurrency capability: Stably supports simultaneous input of multiple video streams, meeting the needs of large and medium-sized video conferencing scenarios; 5. Strong compatibility with domestic products: The entire architecture can be implemented based on domestic FPGAs, eliminating dependence on imported chips and improving equipment controllability and security; 6. Extremely high resource utilization: One-time acquisition and bidirectional multiplexing avoids repeated decoding, reducing power consumption and hardware costs.
[0067] Device Example 1 This invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it performs the steps described in the method embodiment.
[0068] Device Example 2 This invention provides a computer-readable storage medium storing an information transmission implementation program, which, when executed by a processor, performs the steps described in the method embodiment.
[0069] The computer-readable storage media described in this embodiment include, but are not limited to, ROM, RAM, disk, or optical disk.
[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A video conferencing system based on FPGA and CPU shared cache, characterized in that, include: The FPGA processing unit is used to acquire multiple video source signals in parallel, preprocess the multiple video source signals to generate multiple video frames of the same source with uniform format, receive switching instructions, and perform switching operations between the output channel and the video source based on the switching instructions. A shared cache unit, connected to the FPGA processing unit and the CPU processing unit, is used to cache the same source video frames written by the FPGA processing unit and provide them to the CPU processing unit for reading. The CPU processing unit is used to read the same source video frames from the shared cache unit and generate a visual preview interface, and to generate the switching instruction in response to the operation instruction input by the external input and send it to the FPGA processing unit; The FPGA processing unit is the only processing unit in the system that acquires the signals from the multiple video sources, while the CPU processing unit has no video signal acquisition interface.
2. The system according to claim 1, characterized in that, The FPGA processing unit specifically includes: The acquisition interface subunit is used to receive multiple video source signals in parallel; The timing recovery subunit is connected to the acquisition interface subunit and is used to correct the line synchronization signal deviation and field synchronization signal deviation of each video source signal respectively. The frame synchronization subunit, connected to the timing recovery subunit, is used to align the video frames of each video source signal to the same clock reference point using the system's unified reference clock as the standard. The format conversion subunit, connected to the frame synchronization subunit, is used to convert the video source signals of each channel into a standard color space format; The scaling and compositing subunit, connected to the format conversion subunit, is used to scale the video source signals of each channel and combine the scaled video source signals into a single video frame from the same source.
3. The system according to claim 1, characterized in that, The shared cache unit specifically includes: A high-speed cache partition, connected to the FPGA processing unit and the CPU processing unit, is used to cache real-time video frames; A large-capacity cache partition, connected to the FPGA processing unit and the CPU processing unit, is used to cache historical video frames and composited preview video frames; The write arbitration subunit is connected to the high-speed cache partition and the large-capacity cache partition. When multiple source video frames corresponding to multiple video source signals are simultaneously written to the shared cache unit, the write requests are sorted according to the preset channel priority, and each channel is authorized in sequence to write the source video frames to the high-speed cache partition or the large-capacity cache partition. The amount of data continuously written after a single authorization does not exceed a preset threshold.
4. The system according to claim 1, characterized in that, The CPU processing unit specifically includes: A cache reading subunit, connected to the shared cache unit, is used to read the same-source video frames from the shared cache unit; The preview generation subunit, connected to the cache reading subunit, is used to split the same source video frame into multiple sub-screens according to the preset preview layout; wherein each sub-screen uniquely corresponds to one video source signal; and the multiple sub-screens are adapted to the display size respectively, and the adapted sub-screens are combined into a visual preview interface; An interactive response subunit, connected to the preview generation subunit, is used to receive external input operation instructions, determine the target video source based on the operation instructions, generate a switching instruction based on the target video source, and send it to the FPGA processing unit.
5. The system according to claim 1, characterized in that, The FPGA processing unit performs a switching operation between the output channel and the video source based on the switching instruction, specifically including: The FPGA processing unit responds to the switching command and preloads the frame of the target video source indicated by the switching command into the output buffer, maintaining the normal output of the current output frame during the preloading period; After preloading is complete, a fade-in / fade-out transition is performed to smoothly switch between the current output screen and the screen of the target video source. The total time from the moment the switching command is received to the completion of the fade-in / fade-out transition does not exceed one video frame cycle.
6. The system according to claim 1, characterized in that, When the FPGA processing unit writes the same source video frame into the shared buffer unit, it adds the same frame sequence number and acquisition timestamp to the main video frame, auxiliary video frame and preview video frame at the same acquisition time to identify that the three come from the same acquisition time.
7. The system according to claim 6, characterized in that, After the CPU processing unit reads the same source video frame from the shared cache unit, it verifies whether the frame sequence numbers of the currently read main video frame, auxiliary video frame and preview video frame are consistent. If they match, then the currently read video frame will be used normally. If there is a discrepancy, the currently read video frame is discarded.
8. A video conferencing method based on FPGA and CPU shared cache, characterized in that, include: The FPGA processing unit acquires multiple video source signals in parallel, preprocesses the multiple video source signals to generate multiple homogeneous video frames with a unified format, and writes the homogeneous video frames into a shared buffer unit. The CPU processing unit reads the same source video frames from the shared cache unit, generates and displays a visual preview interface based on the same source video frames, generates a switching instruction in response to an external input operation instruction, and sends the switching instruction to the FPGA processing unit. The FPGA processing unit receives the switching instruction and performs a switching operation between the output channel and the video source based on the switching instruction. The FPGA processing unit is the only processing unit in the system that acquires the signals from the multiple video sources, while the CPU processing unit has no video signal acquisition interface.
9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the video conferencing method based on FPGA and CPU shared cache as described in claim 8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an implementation program for information transmission, which, when executed by a processor, implements the steps of the video conferencing method based on FPGA and CPU shared cache as described in claim 8.
