Audio and video processing system and method based on embedded hybrid matrix and related device

Abstract

The embodiment of the invention provides an audio and video processing system and method based on an embedded hybrid matrix and a related device, and relates to the technical field of audio and video, the audio and video processing system based on the embedded hybrid matrix comprises a micro-control unit, an FPGA chip and an audio and video processing chip, the audio and video processing chip is connected with the micro-control unit and the FPGA chip, and the micro-control unit is connected with the FPGA chip. Wherein the micro-control unit is used for controlling the audio and video processing chip and the FPGA chip; the audio and video processing chip is used for acquiring an audio and video source, decoding the audio and video source, encoding the audio and video source after signal image processing, and outputting the encoded audio and video source to the display device; and the FPGA chip is used for carrying out signal image processing on the decoded audio and video sources. According to the invention, the system complexity and the cost can be reduced.

Description

Technical Field

[0001] This disclosure relates to the field of audio and video technology, and in particular to an audio and video processing system, method and related apparatus based on an embedded hybrid matrix. Background Technology

[0002] Currently, most modern conference rooms and command and control centers adopt a hybrid matrix solution, which supports almost all mainstream audio and video interfaces such as HDMI (High-Definition Multimedia Interface), DVI (Digital Visual Interface), VGA (Video Graphics Array), and SDI (Serial Digital Interface). Without the need for converters, all audio and video signals are uniformly connected, enabling non-blocking, low-latency processing of all audio and video signals, as well as large-screen display and video conferencing.

[0003] Current hybrid matrix solutions employ FPGA (Field-Programmable Gate Array) and backplane architecture. Due to the highly specialized nature of the initial system integration, configuration, and debugging work, technicians need in-depth knowledge of HDCP (High-bandwidth Digital Content Protection), EDID (Extended Display Identification Data), video signal formats, encoding and decoding, and other related expertise, resulting in high technical complexity. At the same time, the involvement of additional peripherals such as backplanes, chassis, matching cables, and converters leads to high hardware costs. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides an audio and video processing system, method, and related apparatus based on an embedded hybrid matrix.

[0005] This disclosure provides an audio / video processing system based on an embedded hybrid matrix. The system integrates a microcontroller unit, an FPGA chip, and an audio / video processing chip. The audio / video processing chip is connected to both the microcontroller unit and the FPGA chip. The microcontroller unit controls the audio / video processing chip and the FPGA chip. The audio / video processing chip acquires audio / video sources, decodes them, encodes the processed audio / video sources, and outputs the encoded audio / video sources to a display device. The FPGA chip performs signal / image processing on the decoded audio / video sources.

[0006] This implementation integrates an FPGA chip, audio / video processing chip, and microcontroller unit onto a single board, saving on hardware procurement and wiring costs for multiple peripherals. By utilizing the audio / video processing chip and microcontroller unit to share the audio / video processing load of the FPGA chip, the development cycle is shortened, reducing system complexity and development difficulty, as well as manufacturing and maintenance costs. Furthermore, the parallel operation of the FPGA chip and audio / video processing chip reduces signal attenuation and latency, enabling low-latency processing of multiple audio / video signals and meeting the real-time requirements of new video playback.

[0007] In one possible implementation, the audio / video processing chip includes at least one first chip, at least one second chip, and at least one third chip that operate independently in a cycle. The first chip is used to acquire and decode the audio / video source; the second chip is used to encode the processed audio / video source; and the third chip is used to output the encoded audio / video source to a display device.

[0008] In this implementation, multiple chips divide the work of multiple audio and video processing tasks, with each chip handling a single link. This avoids the performance bottleneck of multi-tasking on a single chip. A failure of a single chip only affects the corresponding functional link and will not cause the entire audio and video link to be paralyzed. This facilitates rapid fault location and improves the real-time performance and stability of multi-channel high-definition signal processing. At the same time, the number of input and output channels can be expanded by increasing or decreasing the number of various chips according to actual needs, adapting to various scale scenarios.

[0009] In one possible implementation, the audio / video processing chip includes a first number of first chips, a second number of second chips, and a third number of third chips, each audio / video processing chip having a different application programming interface; wherein the first number is greater than the second number.

[0010] In this implementation, corresponding application programming interfaces are set for various chips, which enables multiple chips to complement each other and ensures that data is not lost, mixed, or erroneous during exchange.

[0011] In one possible implementation, the first chip is used to acquire the input signal and clock timing of the audio and video source, configure the configuration information of the audio and video source according to the configuration items, and send the configured audio and video source to the FPGA chip; wherein, the configuration information includes status information, output channel and output phase; the second chip is used to receive the audio and video source after signal image processing, encode the audio and video source according to the configuration information, and send the encoded audio and video source to the third chip; the third chip is used to configure the output port based on the extended display recognition data and configuration information of the input port, and output the encoded audio and video to the display device.

[0012] In this implementation, the first chip, the second chip, and the third chip work together to achieve the decoding, encoding, and output of audio and video data, respectively. This can improve fault isolation capabilities and debugging efficiency, while also adapting to the flexible expansion requirements of multiple inputs, meeting the high-definition, low-latency audio and video processing needs of scenarios such as conference rooms and command centers.

[0013] In one possible implementation, the FPGA chip is used to receive the audio and video sources configured by each first chip, reconfigure the clock timing of the audio and video sources through a phase-locked loop frequency multiplier, determine the output second chip according to the configuration information, and send the processed audio and video sources to the second chip.

[0014] In this implementation, the hardware parallel processing capability of the FPGA and the clock reconfiguration function of the phase-locked loop frequency multiplier are used to realize clock synchronization and timing calibration of multiple audio and video sources, eliminate timing deviations of different first chip input signals, and ensure signal stability in subsequent encoding stages.

[0015] In one possible implementation, the FPGA chip includes a display buffer, a pre-stored buffer, and a backup buffer; wherein the pre-stored buffer is used to write the current audio / video source, the display buffer is used to read the output audio / video source, and the backup buffer is used to write the next audio / video source.

[0016] In this implementation, a three-buffer mechanism of display buffer, pre-buffer, and backup buffer is set in the FPGA chip to achieve audio and video read and write separation. This can avoid screen tearing and stuttering when switching audio and video sources and ensure seamless connection of multi-channel signal switching. At the same time, the three buffers have clear division of labor, which effectively improves the reading and writing efficiency and timing synchronization of audio and video data, and meets the low latency and high stability large screen display requirements of hybrid matrix systems.

[0017] This disclosure also provides an FPGA chip data caching method, applied to an FPGA chip. The FPGA chip data caching method includes: when the FPGA chip receives the current audio and video source, caching the current frame data of the current audio and video source in a pre-stored cache; determining whether the data frame in the display cache has been read completely; if the data frame in the display cache has been read completely, converting the pre-stored cache into a new display cache and converting the display cache into a new pre-stored cache; reading the current frame data in the new display cache and caching the next frame data of the current video source in the new pre-stored cache.

[0018] In this implementation, the FPGA triple buffer mechanism is used to quickly switch between the pre-stored buffer and the display buffer after the current frame is read, by judging the data reading status of the display buffer in real time. This enables parallel operation of reading the current frame and writing the next frame, avoiding screen tearing and stuttering during frame switching.

[0019] In one possible implementation, the FPGA chip data caching method further includes: when the FPGA chip receives the next audio / video source, caching the current frame data of the next audio / video source in a spare buffer; determining whether an audio / video source switching instruction has been received; if an audio / video source switching instruction has been received, determining whether the data frame in the display buffer has been read completely; if the data frame in the display buffer has been read completely, converting the spare buffer into a new display buffer, converting the display buffer into a new pre-stored buffer, clearing the data in the pre-stored buffer and converting it into a new spare buffer; reading the current frame data in the new display buffer, caching the next frame data of the next video source in the new pre-stored buffer, and leaving the new spare buffer idle and waiting for a new video source.

[0020] In this implementation, an FPGA triple buffer mechanism is used to pre-buffer the next audio and video source data through a backup buffer. The buffer role is switched only when a switching command is received and the current display buffer frame is read. This achieves seamless switching of audio and video sources and avoids problems such as screen tearing, black screen, or frame loss during the switching process. At the same time, through the dynamic switching of buffer roles and data clearing mechanism, the buffer resources are efficiently recycled to meet the low latency and high stability requirements of the hybrid matrix system for fast switching of multiple signals.

[0021] This disclosure also provides a computing device, which includes: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement the FPGA chip data caching method provided in this disclosure.

[0022] This disclosure also provides a computer-readable storage medium storing a computer program for executing the FPGA chip data caching method provided in this disclosure. Attached Figure Description

[0023] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0024] Figure 1 A schematic diagram of an audio / video processing system based on an embedded hybrid matrix provided in an embodiment of this disclosure; Figure 2 A flowchart illustrating the audio / video receiving module provided in an embodiment of this disclosure; Figure 3 A flowchart illustrating the audio and video processing module provided in an embodiment of this disclosure; Figure 4 A flowchart illustrating the audio / video encoding module provided in this embodiment of the disclosure; Figure 5 A flowchart illustrating the audio / video output module provided in an embodiment of this disclosure; Figure 6 A flowchart illustrating an FPGA chip data caching method provided in this embodiment of the disclosure; Figure 7 A flowchart illustrating another FPGA chip data caching method provided in this embodiment of the disclosure; Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure. Detailed Implementation

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0027] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0028] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0029] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0030] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0031] Current hybrid matrix solutions employ FPGA and backplane architectures. The high level of technical complexity stems from the highly specialized nature of the initial system integration, configuration, and debugging work, requiring technicians to possess in-depth knowledge of HDCP, EDID, video signal formats, and encoding / decoding. Furthermore, the involvement of additional peripherals such as backplanes, chassis, cables, and converters increases hardware costs. Additionally, during signal switching, the backend display devices may experience signal instability due to the loss of synchronization signals, resulting in delays and black screens.

[0032] To address the aforementioned issues, this disclosure provides an audio / video processing system based on an embedded hybrid matrix. This system integrates a microcontroller unit, an FPGA chip, and an audio / video processing chip. The audio / video processing chip is connected to both the microcontroller unit and the FPGA chip. The microcontroller unit controls both the audio / video processing chip and the FPGA chip. The audio / video processing chip acquires audio / video sources, decodes them, encodes the processed audio / video sources, and outputs the encoded audio / video sources to a display device. The FPGA chip performs signal / image processing on the decoded audio / video sources. By integrating the FPGA chip, audio / video processing chip, and microcontroller unit onto a single board, the hardware procurement and wiring costs of multiple peripherals are reduced. The audio / video processing chip and microcontroller unit share the audio / video processing load of the FPGA chip, shortening the development cycle, reducing system complexity and development difficulty, and lowering manufacturing and maintenance costs. Furthermore, the parallel operation of the FPGA chip and the audio / video processing chip reduces signal attenuation and latency, enabling low-latency processing of multiple audio / video signals and meeting the real-time requirements of new video playback.

[0033] The method will be described below with reference to specific embodiments.

[0034] Figure 1 This is a schematic diagram of an audio / video processing system based on an embedded hybrid matrix, provided in an embodiment of this disclosure. Figure 1 As shown, the audio and video processing system based on an embedded hybrid matrix includes a microcontroller unit, an FPGA chip, and an audio and video processing chip, which are connected to the microcontroller unit and the FPGA chip, respectively.

[0035] A microcontroller unit (MCU) is a compact embedded processor that integrates a central processing unit (CPU), memory, general-purpose I / O ports, and various peripheral interfaces onto a single chip. It is used to control audio and video processing chips and FPGA chips.

[0036] Audio and video processing chips are application-specific integrated circuits (ASICs) used to acquire, decode, encode, convert, enhance, transmit, and output audio and video signals. Examples of such chips include the MS9331, MS7200, and MS7210. These chips acquire audio and video sources, decode them, encode the processed audio and video signals, and output the encoded audio and video signals to a display device.

[0037] FPGA chips are highly flexible and programmable semiconductor devices. FPGA chips are used for signal and image processing of decoded audio and video sources.

[0038] Specifically, the audio and video processing chip includes at least one first chip, at least one second chip, and at least one third chip that operate independently, with the microcontroller unit controlling each of the three chips.

[0039] In one possible implementation, the microcontroller unit (MCU) is equipped with a flag protection mechanism. When multiple IIC buses of multiple chips are operating concurrently, the MCU sends control commands to ensure that when sending data commands, the data streams of different chips are transmitted within the same IIC link, thus ensuring the stability and reliability of the IIC link data.

[0040] In one possible implementation, the microcontroller unit (MCU) incorporates a critical protection zone mechanism. Specifically, the IIC instruction transmission process for a single chip is encapsulated as an indivisible critical section. When the MCU sends an instruction to a single chip, it enters a critical protection state, blocking other tasks from preempting or interfering with the IIC bus. Once the instruction transmission is complete, it exits the critical protection state, ensuring that IIC data transmission is not interrupted by other tasks and avoiding instruction transmission chaos.

[0041] In one possible implementation, the three specialized chips run independently in multiple threads, reducing the delay and waiting time for the second chip to start working after the first chip starts working, and finally the third chip to start working. This application ensures the reasonable progress of work among the same chips, and when the status label changes, the corresponding chip will start working immediately.

[0042] In this implementation, multiple chips divide the work of multiple audio and video processing tasks, with each chip handling a single link. This avoids the performance bottleneck of multi-tasking on a single chip. A failure of a single chip only affects the corresponding functional link and will not cause the entire audio and video link to be paralyzed. This facilitates rapid fault location and improves the real-time performance and stability of multi-channel high-definition signal processing. At the same time, the number of input and output channels can be expanded by increasing or decreasing the number of various chips according to actual needs, adapting to various scale scenarios.

[0043] In one possible implementation, the audio / video processing chip includes a first number of first chips, a second number of second chips, and a third number of third chips, each with a different application programming interface (API). Specifically, this application provides a separate API interface for each chip, ensuring that data exchange is error-free, non-mixed, and without data loss.

[0044] The first quantity is greater than the second quantity. For example, the audio / video processing chip includes eight first chips, four second chips, and four third chips to achieve eight-channel input switching to four-channel output.

[0045] Correspondingly, the audio and video processing system architecture of this application includes an audio and video receiving module, an audio and video processing module, an audio and video encoding module, and an audio and video output module.

[0046] The audio / video receiving module acquires the audio / video source through the first chip and decodes the audio / video source. For example, the first chip is an MS7200 chip. For details, please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a flowchart illustrating the audio / video receiving module provided in an embodiment of this disclosure.

[0047] like Figure 2 As shown, after the audio / video receiving module is initialized, it powers on the first chip, acquires the external input voltage and clock timing of the audio / video source, achieves internal voltage regulation, and enters standby mode. For example, the external input voltage is 5V.

[0048] Furthermore, the first chip processes the audio and video sources according to the configuration items. For example, it performs signal equalization compensation and audio / video separation on the audio and video sources.

[0049] Furthermore, the first chip configures the status information, output channel, and output phase of the audio and video source according to the configuration items, and transmits the audio and video source to the FPGA chip through the output channel. The configuration status information includes operating status, such as normal operation, standby, and error reporting; data validity flags, such as data signal enable; and error status, such as signal loss and HPD (hot-plug detection) failure. The output channel is the output channel of the audio and video source, clearly defining the data flow.

[0050] Understandably, each first chip processes one audio / video source and sends multiple audio / video sources to the FPGA chip.

[0051] The audio and video processing module uses an FPGA chip to perform signal and image processing on the decoded audio and video sources. For details, please refer to [link / reference needed]. Figure 3 , Figure 3 This is a flowchart illustrating the audio and video processing module provided in an embodiment of this disclosure.

[0052] like Figure 3 As shown, after the audio / video processing module is initialized, the FPGA chip is powered on. The FPGA chip receives the audio / video sources configured by each first chip, parses the audio / video, and reconfigures the clock timing of the audio / video sources through a phase-locked loop (PLL) frequency multiplier. Specifically, the FPGA chip generates a standard pixel clock based on its own high-precision external crystal oscillator through a PLL, and fixes timing parameters such as horizontal and vertical synchronization and data enable, forming a fixed output timing independent of the input source. For example, the standard pixel is 1080P@60Hz.

[0053] Furthermore, the second chip for output is determined based on the configuration information, and the processed audio and video sources are sent to the second chip. Specifically, the corresponding second chip is selected for output based on the output channel information in the configuration information.

[0054] The audio / video encoding module encodes the processed audio and video source using a second chip. For example, the second chip is an MS7210 chip. See [link to relevant documentation] for details. Figure 4 , Figure 4 This is a flowchart illustrating the audio / video encoding module provided in an embodiment of this disclosure.

[0055] like Figure 4 As shown, after the audio / video encoding module is initialized, the second chip is powered on. The second chip receives the audio / video source after signal image processing and encodes the audio / video source according to the configuration information. Specifically, the audio / video source is subjected to channel switching, phase adjustment, and timing calibration according to the configuration information. Channel switching selects the currently valid input path and adapts to the multiple audio / video sources output by the FPGA to ensure that the output of complete frame data is uninterrupted during switching. Phase adjustment adjusts the phase offset between the output TMDS (Transition-Minimized Differential Signaling) clock and data to compensate for transmission line delay and keep the display device in a stable sampling window. Timing calibration configures the occlusion length and synchronization pulse width to achieve synchronized audio and video output.

[0056] Furthermore, the second chip sends the encoded audio and video source to the third chip.

[0057] The audio / video output module outputs the encoded audio / video source to the display device via a third chip. For example, the third chip is an MS9331 chip. See [link to documentation] for details. Figure 5 , Figure 5 This is a flowchart illustrating the audio / video output module provided in an embodiment of this disclosure.

[0058] like Figure 5 As shown, after the audio / video output module is initialized, the third chip is powered on. The third chip configures the output ports based on the extended display identification data and configuration information of the input ports, and outputs the encoded audio and video to the display device. Specifically, the control unit controls the third chip to detect the input port signals. When an input port signal is detected, it immediately pulls the HPD signal lines of each output port high to trigger and complete hot-plug detection. The microcontroller unit controls the third chip to read the EDID of each output port, parses and processes it, and uses the optimal EDID data as the EDID of the input port. Further, the control unit controls the third chip to configure the output ports and output the encoded video signal to the display.

[0059] In this implementation, the functional layering and collaborative operation of the first, second, and third chips respectively realize the decoding, encoding, and output of audio and video data. This improves fault isolation capabilities and debugging efficiency, while also adapting to the flexible expansion requirements of multiple inputs, meeting the high-definition, low-latency audio and video processing needs of scenarios such as conference rooms and command centers. Utilizing the hardware parallel processing capabilities of the FPGA and the clock reconfiguration function of the phase-locked loop frequency multiplier, clock synchronization and timing calibration of multiple audio and video sources are achieved, eliminating timing deviations in the input signals of different first chips and ensuring signal stability in subsequent encoding stages.

[0060] This application also incorporates a triple-buffer mechanism in the FPGA chip. This triple-buffer mechanism, consisting of a display buffer, a pre-buffer, and a backup buffer, separates audio and video read / write operations, preventing screen tearing and stuttering during audio / video source switching and ensuring seamless switching between multiple signals. Furthermore, the clear division of labor among the three buffers effectively improves the efficiency and timing synchronization of audio and video data read / write operations, meeting the low-latency, high-stability large-screen display requirements of hybrid matrix systems.

[0061] Specifically, the pre-stored buffer is used to write to the current audio and video source, the display buffer is used to read the output audio and video source, and the spare buffer is used to write to the next audio and video source.

[0062] In one possible implementation, specifically, Figure 6 This is a flowchart illustrating an FPGA chip data caching method according to an embodiment of the present disclosure. The method can be executed by an FPGA chip, which can be implemented in software and / or hardware, and is generally integrated into a computing device. Figure 6 As shown, the method includes: S601. When the FPGA chip receives the current audio and video source, it caches the current frame data of the current audio and video source in the pre-stored cache.

[0063] Specifically, when the FPGA chip receives the current video source, it caches a complete frame of data in the pre-stored buffer and reads the data in the display buffer. The data in the display buffer can be the previous frame of data from the current video source or other pre-stored startup audio and video.

[0064] Furthermore, it is determined whether the FPGA chip has received the next audio / video source. When the FPGA chip has not received the next audio / video source, the idle backup buffer is cleared.

[0065] S602. Determine whether the data frame in the display cache has been read completely. If the data frame in the display cache has been read completely, convert the pre-stored cache to a new display cache and convert the display cache to a new pre-stored cache.

[0066] It continuously checks whether the data frames in the display cache have been read completely. If the data frames in the display cache have not been read completely, it waits to continue reading. If the data frames in the display cache have been read completely, it converts the pre-stored cache and the display cache, converting the pre-stored cache into a new display cache and the display cache into a new pre-stored cache.

[0067] S603: Read the current frame data from the new display cache and cache the next frame data of the current video source in the new pre-stored cache.

[0068] Continue reading data frames from the new display buffer and caching the new data frames in the new pre-buffer. Repeat the above steps to swap the pre-buffer and display buffer. The pre-buffer is used only for writing, and the display buffer is used only for reading. When only the current audio / video source is present, the standby buffer is idle, waiting to receive the next audio / video source.

[0069] In this implementation, the FPGA triple buffer mechanism is used to quickly switch between the pre-stored buffer and the display buffer after the current frame is read, by judging the data reading status of the display buffer in real time. This enables parallel operation of reading the current frame and writing the next frame, avoiding screen tearing and stuttering during frame switching.

[0070] In one possible implementation, specifically, Figure 7 This is a flowchart illustrating another FPGA chip data caching method provided in an embodiment of this disclosure. This method can be executed by an FPGA chip, which can be implemented in software and / or hardware, and is generally integrated into a computing device. Figure 7 As shown, the method includes: S701. When the FPGA chip receives the current audio and video source, it caches the current frame data of the current audio and video source in the pre-stored cache.

[0071] Specifically, when the FPGA chip receives the current video source, it caches a complete frame of data in the pre-stored buffer and reads the data in the display buffer. The data in the display buffer can be the previous frame of data from the current video source or other pre-stored startup audio and video.

[0072] Furthermore, it determines whether the FPGA chip has received the next audio / video source.

[0073] S702. When the FPGA chip receives the next audio / video source, it caches the current frame data of the next audio / video source in the backup buffer.

[0074] Specifically, when the FPGA chip receives the next video source, it caches a complete frame of data in the backup buffer.

[0075] S703. Determine whether an audio / video source switching command has been received. If an audio / video source switching command has been received, determine whether the data frames in the display buffer have been read completely. If the data frames in the display buffer have been read completely, convert the pre-stored buffer to a new display buffer, convert the display buffer to a new pre-stored buffer, clear the data in the pre-stored buffer and convert it to a new standby buffer.

[0076] Determine if an audio / video source switching command has been received. This command instructs the user to switch from the current audio / video source to the next. If no such command is received, continue transmitting from the current data source. (See reference...) Figure 6 The method shown determines whether the data frames in the display cache have been read completely. If the data frames in the display cache have been read completely, the pre-stored cache is converted into a new display cache, and the display cache is converted into a new pre-stored cache.

[0077] If an audio / video source switching command is received, the next data source is transmitted. Specifically, it is determined whether the data frames in the display buffer have been completely read. If the data frames in the display buffer have been completely read, the standby buffer is converted into a new display buffer, the display buffer is converted into a new pre-stored buffer, and the data in the pre-stored buffer is cleared and converted into a new standby buffer. The new standby buffer is then idle, waiting to receive subsequent audio / video sources.

[0078] S704: Read the current frame data in the new display cache, cache the next frame data of the next video source in the new pre-stored cache, and leave the new standby cache idle while waiting for a new video source.

[0079] Continue reading data frames from the new display buffer and cache new data frames in the new pre-stored buffer. If no new audio / video source switching command is received, proceed according to... Figure 6 The method shown swaps the pre-stored cache and the display cache.

[0080] In this implementation, an FPGA triple buffer mechanism is used to pre-buffer the next audio and video source data through a backup buffer. The buffer role is switched only when a switching command is received and the current display buffer frame is read. This achieves seamless switching of audio and video sources and avoids problems such as screen tearing, black screen, or frame loss during the switching process. At the same time, through the dynamic switching of buffer roles and data clearing mechanism, the buffer resources are efficiently recycled to meet the low latency and high stability requirements of the hybrid matrix system for fast switching of multiple signals.

[0081] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the FPGA chip data caching method in the above embodiments.

[0082] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure.

[0083] The following is a detailed reference. Figure 8 The diagram illustrates a structural schematic suitable for implementing the computing device 800 in the embodiments of this disclosure. The computing device 800 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The computing device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0084] like Figure 8 As shown, the computing device 800 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 802 or a program loaded from memory 808 into random access memory (RAM) 803. The RAM 803 also stores various programs and data required for the operation of the computing device 800. The processor 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0085] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows computing device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 8 A computing device 800 with various devices is shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.

[0086] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a memory 808, or installed from a ROM 802. When the computer program is executed by the processor 801, it performs the functions defined in the FPGA chip data caching method of embodiments of this disclosure.

[0087] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0088] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0089] The aforementioned computer-readable medium may be included in the aforementioned computing device; or it may exist independently and not assembled into the computing device.

[0090] The aforementioned computer-readable medium carries one or more programs, which, when executed by the computing device, cause the computing device to perform the aforementioned FPGA chip data caching method.

[0091] The computing device can be programmed with computer program code in one or more programming languages ​​or a combination thereof to perform the operations of this disclosure. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0093] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0094] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0095] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0096] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0097] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0098] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An audio and video processing system based on an embedded hybrid matrix, characterized in that, The system integrates a microcontroller unit, an FPGA chip, and an audio / video processing chip. The audio / video processing chip is connected to both the microcontroller unit and the FPGA chip. The microcontroller unit is used to control the audio / video processing chip and the FPGA chip; The audio and video processing chip is used to acquire audio and video sources, decode the audio and video sources, encode the audio and video sources after signal image processing, and output the encoded audio and video sources to the display device. The FPGA chip is used to perform signal and image processing on the decoded audio and video sources.

2. The audio and video processing system based on an embedded hybrid matrix according to claim 1, characterized in that, The audio and video processing chip includes at least one first chip, at least one second chip, and at least one third chip that operate independently in a cycle. The first chip is used to acquire the audio and video source and decode the audio and video source; The second chip is used to encode the audio and video sources after signal image processing; The third chip is used to output the encoded audio and video source to the display device.

3. The audio and video processing system based on an embedded hybrid matrix according to claim 2, characterized in that, The audio and video processing chip includes a first number of first chips, a second number of second chips, and a second number of third chips, each of which is provided with a different application programming interface; wherein the first number is greater than the second number.

4. The audio and video processing system based on an embedded hybrid matrix according to claim 3, characterized in that, The first chip is used to acquire the input signal and clock timing of the audio and video source, configure the configuration information of the audio and video source according to the configuration items, and send the configured audio and video source to the FPGA chip; wherein, the configuration information includes status information, output channel and output phase; The second chip is used to receive the audio and video source after signal image processing, encode the audio and video source according to the configuration information, and send the encoded audio and video source to the third chip; The third chip is used to configure the output port based on the extended display identification data of the input port and the configuration information, and to output the encoded audio and video to the display device.

5. The audio and video processing system based on an embedded hybrid matrix according to claim 4, characterized in that, The FPGA chip is used to receive the audio and video source configured by each of the first chips, and reconfigure the clock timing of the audio and video source through a phase-locked loop frequency multiplier; The second chip for output is determined based on the configuration information, and the processed audio and video source is sent to the second chip.

6. The audio and video processing system based on an embedded hybrid matrix according to claim 5, characterized in that, The FPGA chip includes a display cache, a pre-stored cache, and a backup cache; wherein, the pre-stored cache is used to write the current audio / video source, the display cache is used to read the output audio / video source, and the backup cache is used to write the next audio / video source.

7. A data caching method for an FPGA chip, characterized in that, Applied to the FPGA chip of claim 6, the method includes: When the FPGA chip receives the current audio and video source, it caches the current frame data of the current audio and video source in the pre-stored cache; Determine whether the data frames in the display cache have been read completely. If the data frames in the display cache have been read completely, convert the pre-stored cache into a new display cache and the display cache into a new pre-stored cache. Read the current frame data from the new display cache, and cache the next frame data of the current video source in the new pre-stored cache.

8. The FPGA chip data caching method according to claim 7, characterized in that, The method further includes: When the FPGA chip receives the next audio / video source, it caches the current frame data of the next audio / video source in the backup cache; Determine whether an audio / video source switching instruction has been received. If the audio / video source switching instruction has been received, determine whether the data frames in the display cache have been read completely. If the data frames in the display cache have been read completely, convert the standby cache into a new display cache, convert the display cache into a new pre-stored cache, clear the data in the pre-stored cache, and convert it into a new standby cache. Read the current frame data from the new display cache, cache the next frame data of the next video source in the new pre-stored cache, and leave the new standby cache idle and wait for a new video source.

9. A computing device, characterized in that, The computing device includes: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the FPGA chip data caching method as described in claim 7 or 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to execute the FPGA chip data caching method of claim 7 or 8.

Images