Multimedia chip prototype verification device and method based on FPGA

Abstract

The invention relates to the technical field of chip prototype verification, in particular to a multimedia chip prototype verification device and method based on an FPGA (Field Programmable Gate Array), the whole multimedia architecture is accommodated in a manner of cascading three FPGA platforms, so that the function and the hierarchical structure are more distinct; different functional modules are distributed on the three FPGAs, so that the resource advantages of the FPGAs can be fully utilized, and the prototype verification capability of the multimedia chip is improved. The first FPGA is mainly responsible for starting related things by a system, the second FPGA is responsible for processing services of the multimedia module, the coding and decoding module has relatively large resources, relatively independent functions and low coupling with other modules, so that the third FPGA independently accommodates the coding and decoding module, and the three FPGA platforms are matched for use, so that the cost is reduced, and the efficiency is improved. The multimedia system can work in a high-performance mode as much as possible, and the problem that the whole multimedia system cannot be accommodated due to insufficient resources of a single chip of the FPGA is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of chip prototype verification, and in particular to a multimedia chip prototype verification device and method based on FPGA. Background Art

[0002] With the rapid development of multimedia technology and the dramatic increase in multimedia chip shipments, people are placing increasing emphasis on the verification stage of multimedia chips after their design is complete. Field Programmable Gate Arrays (FPGAs) are a key verification method before chip tape-out. Prototype verification, performed based on chip prototypes, is the most realistic approach to actual chip performance available. However, FPGA prototype verification also presents certain bottlenecks, particularly in large multimedia systems. This can lead to issues such as insufficient resources for a single FPGA chip, complex architecture, low performance, and implementation difficulties.

[0003] In the chip industry, multimedia chips are large in scale and complex in architecture, resulting in long development and verification cycles. FPGA prototype verification, a core step in chip verification, often presents the problem of insufficient resources on a single FPGA chip to support the hardware resources of the entire multimedia chip system.

[0004] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in this technical field. Summary of the Invention

[0005] The technical problem to be solved by the present invention is how to solve the problem that a single FPGA chip cannot bear the hardware resources of the entire multimedia chip system.

[0006] The present invention adopts the following technical solutions: In a first aspect, a multimedia chip prototype verification device based on an FPGA is provided, comprising: a first FPGA, a second FPGA, and a third FPGA, wherein the first FPGA comprises a control module, a first bus, and a first storage module; the second FPGA comprises a second bus, a receiving module, a selection processing module, and a reading and output module; and the third FPGA comprises a codec module; The control module is connected to the first bus, and the first bus is respectively connected to the second bus, the first storage module and the codec module; The second bus is connected to the receiving module, the selection processing module and the reading and outputting module respectively; the receiving module is connected to the selection processing module; The control module is used to send configuration signals, and the first bus and the second bus are used to configure downstream modules respectively according to the configuration signals; The receiving module is used to receive first video data, the selection and processing module is used to process the first video data to obtain second video data, and the second video data is stored in the first storage module after passing through the second bus and the first bus in sequence; The encoding and decoding module is used to encode and decode the second video data in the first storage module to obtain third video data, and store the third video data in the first storage module; The reading and outputting module is used to read the third video data in the first storage module and output it in a corresponding display manner.

[0007] Preferably, the first bus includes a first configuration bus and a first data bus, and the second bus includes a second configuration bus and a second data bus; the first configuration bus and the first data bus are respectively connected to the control module; the first configuration bus is also respectively connected to the second configuration bus and the codec module; the first data bus is also respectively connected to the second data bus, the first storage module, and the codec module; The second configuration bus is connected to the receiving module, the selection processing module and the reading and outputting module respectively; the second data bus is connected to the selection processing module and the reading and outputting module respectively; The first configuration bus and the second configuration bus are used to configure downstream modules respectively according to the configuration signal; The second data bus is used to receive the second video data and transmit the second video data to the first data bus, and the first data bus is used to transmit the second video data to the first storage module for storage; The first data bus is further used to transmit the second video data to the encoding and decoding module for processing to obtain third video data, and transmit the third video data to the first storage module; The reading and outputting module is used to read the third video data in the first storage module through the second data bus and the first data bus in sequence.

[0008] Preferably, the first FPGA includes a first interconnection module and a second interconnection module, the second FPGA includes a third interconnection module, and the third FPGA includes a fourth interconnection module; Each interconnection module includes a bus transmission unit and a serial-parallel conversion unit connected to each other, wherein the bus transmission unit is used to receive a corresponding configuration signal or video data signal, and the serial-parallel conversion unit is used to convert the configuration signal or video data signal into a parallel signal and transmit the parallel signal to the serial-parallel conversion unit in other interconnection modules; The serial-to-parallel conversion unit is further used to convert the parallel signal into a configuration signal or a video data signal, and the bus transmission unit is further used to transmit the configuration signal or the video data signal to the corresponding module; The bus transmission units in the first interconnection module and the second interconnection module are connected to the first configuration bus and the first data bus respectively; The bus transmission unit in the third interconnection module is connected to the second configuration bus and the second data bus respectively; the bus transmission unit in the fourth interconnection module is connected to the codec module; The serial-parallel conversion unit in the first interconnection module is connected to the serial-parallel conversion unit in the third interconnection module; the serial-parallel conversion unit in the second interconnection module is connected to the serial-parallel conversion unit in the fourth interconnection module.

[0009] Preferably, the second FPGA further includes a second storage module and a data selection module, wherein a common end of the data selection module is connected to the second data bus, a first branch end of the data selection module is connected to the first data bus, and a second branch end of the data selection module is connected to the second storage module; The data selection module is used to transmit the second video data from the second data bus to the first storage module or the second storage module; When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module is used to transmit the second video data to the first storage module through the first data bus, the encoding and decoding module is used to read the second video data in the first storage module through the first data bus, encode and decode the second video data to obtain third video data, and store the third video data in the first storage module through the first data bus; the reading and output module is used to read the third video data in the first storage module through the second data bus and the first data bus and output the data; When the resolution of the video data is less than a first preset threshold and / or the frame rate of the video data is less than a second preset threshold, the third FPGA is turned off, the data selection module is used to transfer the second video data to the second storage module for storage, and the read output module is used to read the second video data in the second storage module through the second data bus and output it.

[0010] Preferably, the reading and output module includes a reading unit and an output unit, the reading unit is connected to the second data bus, and the output unit is connected to the reading unit; The reading unit is configured to read the third video data in the first storage module, or read the second video data in the second storage module; The output unit is used to output the third video data or the second video data to a display screen of a corresponding format through a corresponding cable to output an image.

[0011] Preferably, the second FPGA further includes a calculation module, and the calculation module is connected to the second data bus; The calculation module is used to read the third video data in the first storage module, or read the second video data in the second storage module, perform precision calculation on the third video data or the second video data, and re-store the third video data or the second video data after the precision calculation into the first storage module or the second storage module; Among them, when the reading unit reads the third video data from the first storage module or reads the second video data from the second storage module, if the image output through the output unit and the corresponding display screen is a distorted image, the third video data is read from the first storage module or the second video data is read from the second storage module through the calculation module, and the corresponding video data is processed by the calculation module and then sent back to the first storage module or the second storage module; it is re-read by the reading unit and re-displayed through the output unit and the display screen. At this time, the image displayed on the display screen is the image after the distortion is corrected.

[0012] In a second aspect, a method for verifying a multimedia chip prototype based on an FPGA is provided. The method is applicable to the FPGA-based multimedia chip prototype verification device described in the first aspect, comprising: The control module sends a configuration signal, and the first bus and the second bus configure downstream modules respectively according to the configuration signal; The receiving module receives first video data, the selection and processing module processes the first video data to obtain second video data, and the second video data is sequentially passed through the second bus and the first bus and then stored in the first storage module; The encoding and decoding module encodes and decodes the second video data in the first storage module to obtain third video data, and stores the third video data in the first storage module; The reading and outputting module reads the third video data in the first storage module and outputs the data in a corresponding display manner.

[0013] Preferably, the first bus includes a first configuration bus and a first data bus, and the second bus includes a second configuration bus and a second data bus; and the reading and outputting module reading the third video data from the first storage module includes: The first configuration bus and the second configuration bus respectively configure downstream modules according to the configuration signal; The second data bus receives the second video data and transmits the second video data to the first data bus, and the first data bus transmits the second video data to the first storage module for storage; The first data bus further transmits the second video data to the encoding and decoding module for processing to obtain third video data, and transmits the third video data to the first storage module; The reading and outputting module sequentially reads the third video data in the first storage module through the second data bus and the first data bus.

[0014] Preferably, the second FPGA further includes a second storage module and a data selection module; the method further includes: The data selection module transmits the second video data from the second data bus to the first storage module or the second storage module; When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module transmits the second video data to the first storage module through the first data bus, the codec module reads the second video data in the first storage module through the first data bus, encodes and decodes the second video data to obtain third video data, and stores the third video data in the first storage module through the first data bus; the read and output module reads the third video data in the first storage module through the second data bus and the first data bus and outputs the data; When the resolution of the video data is less than a first preset threshold and / or the frame rate of the video data is less than a second preset threshold, the third FPGA is turned off, the data selection module transfers the second video data to the second storage module for storage, and the reading and output module reads the second video data in the second storage module through the second data bus and outputs it.

[0015] Preferably, the second FPGA further includes a computing module; and the method further includes: The calculation module reads the third video data in the first storage module, or reads the second video data in the second storage module, and performs precision calculation on the third video data or the second video data, and stores the third video data or the second video data after precision calculation back into the first storage module or the second storage module.

[0016] Compared with the prior art, the present invention has the following beneficial effects: Because a single FPGA chip has limited resources, the present invention utilizes three cascaded FPGA platforms to accommodate the entire multimedia architecture, resulting in a more distinct function and hierarchy. By distributing different functional modules across the three FPGAs, the resource advantages of each FPGA are fully utilized, improving the prototype verification capabilities of the multimedia chip. The first FPGA is primarily responsible for system startup-related tasks, while the second handles multimedia module operations. Because the codec module itself has relatively large resources and relatively independent functions, and lacks sufficient coupling with other modules, the third FPGA houses the codec module. The coordinated use of these three FPGA platforms ensures the highest possible performance for the multimedia system, resolving the issue of insufficient resources on a single FPGA chip to accommodate the entire multimedia system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 This is a schematic structural diagram of a multimedia chip prototype verification device based on FPGA provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the connection structure of the configuration bus and the data bus of an FPGA-based multimedia chip prototype verification device provided by an embodiment of the present invention; Figure 3This is a schematic diagram of the connection structure of interconnection modules of an FPGA-based multimedia chip prototype verification device provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of the specific structure of an FPGA-based multimedia chip prototype verification device provided by an embodiment of the present invention; Figure 5 This is a flow chart of a multimedia chip prototype verification method based on FPGA provided by an embodiment of the present invention; Figure 6 This is another flowchart of a multimedia chip prototype verification method based on FPGA provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] Unless the context requires otherwise, throughout the specification and claims, the term "including" is to be interpreted as meaning open inclusion, that is, "including, but not limited to". In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiments", "example", "specific example" or "some examples" and the like are intended to indicate that the specific features, structures, materials or characteristics associated with the embodiment or example are included in at least one embodiment or example of the present disclosure. The schematic representation of the above terms does not necessarily refer to the same embodiment or example. In addition, the specific features, structures, materials or characteristics may be included in any one or more embodiments or examples in any appropriate manner, that is, although they may be carried in the embodiments or examples of the above terms due to reasons such as the order and position of appearance, it is not limited to that they can be carried in combination by one embodiment or example.

[0021] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, for example, the description may also use the method of adding "A" and "B" at the end to describe the same type of nouns as two independent individuals. In this case, the corresponding features defined as "A" and "B" are only used to distinguish the description purposes of the same type of individuals, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0022] When describing some embodiments, the expressions “coupled”, “coupled” and “connected” and their derivatives may be used. For example, when describing some embodiments, the term “connected” may be used to indicate that two or more components are in direct physical or electrical contact with each other. For another example, when describing some embodiments, the term “coupled” may be used to indicate that two or more components are in direct physical or electrical contact. However, the term “connected” or “coupled” may also mean that two or more components are not in direct contact with each other, but still cooperate or interact with each other, such as “optical coupling”, “wireless connection”, etc. The embodiments disclosed herein are not necessarily limited to the contents of the present invention.

[0023] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0024] The existing technology often uses a single FPGA to perform prototype verification on multimedia chips. Due to the limited resources of a single FPGA chip, it cannot bear the hardware resources of the entire multimedia chip system, resulting in unsatisfactory prototype verification results for the multimedia chip. In order to solve the above problem, the embodiment of the present invention provides a multimedia chip prototype verification device based on FPGA, such as Figure 1 As shown, it includes: a first FPGA, a second FPGA and a third FPGA, the first FPGA includes a control module, a first bus and a first storage module; the second FPGA includes a second bus, a receiving module, a selection processing module and a read output module; the third FPGA includes a codec module; the control module is connected to the first bus, the first bus is respectively connected to the second bus, the first storage module and the codec module; the second bus is respectively connected to the receiving module, the selection processing module and the read output module; the receiving module is connected to the selection processing module.

[0025] The control module is used to send down a configuration signal, and the first bus and the second bus are used to configure the downstream modules respectively according to the configuration signal; the receiving module is used to receive first video data, and the selection processing module is used to process the first video data to obtain second video data, and the second video data is stored in the first storage module after passing through the second bus and the first bus in sequence; the encoding and decoding module is used to encode and decode the second video data in the first storage module to obtain third video data, and store the third video data in the first storage module; the reading and output module is used to read the third video data in the first storage module and output it through a corresponding display method.

[0026] The control module plays an initial control role within the entire device, issuing configuration signals that are then transmitted via the first bus. The control module can be a central processing unit (CPU), which coordinates the parallel operation of software and hardware. Users can write software drivers to instruct the CPU to issue read and write processing instructions, configure downstream modules, and handle related transactions in the system. Downstream modules refer to all modules in the prototype verification device other than the control module, the first bus, and the second bus.

[0027] The first bus is connected to multiple modules. On the one hand, it transmits configuration signals to the first storage module and encoding and decoding module in the first FPGA where it is located to perform corresponding initialization configuration, etc. On the other hand, the first bus is also connected to the second bus, and further transmits the configuration signal to the second bus, which initializes and configures the receiving module, selection processing module and reading output module.

[0028] After initialization and configuration of all downstream modules are complete, the receiving module begins operation, receiving first video data from an external source, such as first video data from a camera. The receiving module transmits the first video data to the selection and processing module, which processes the first video data according to pre-set algorithms and rules, such as performing format conversion and filtering, to generate second video data. The second video data is then transmitted sequentially via the second bus and the first bus, and ultimately stored in the first storage module for temporary storage.

[0029] The codec module encodes and decodes the second video data stored in the first storage module. The encoding process compresses or converts the video data into a specific format for storage and transmission; the decoding process restores the encoded video data to a playable format. The encoding and decoding operations generate third video data, which the codec module then stores in the first storage module.

[0030] The reading and outputting module reads the third video data from the first storage module and outputs it through a corresponding display mode, such as connecting to a display screen for image display or transmitting the video data to an external device through other output interfaces for further processing.

[0031] Through the above structure and corresponding workflow, the modules in the three FPGAs cooperate with each other to realize the functions of video data reception, processing, storage, encoding and decoding, and output in the prototype verification process of the multimedia chip, thus realizing efficient prototype verification of the multimedia chip.

[0032] The prototype verification device proposed in the embodiment of the present invention has the following functions: During multimedia chip development, this prototype verification device can help developers quickly verify chip functionality and performance. By receiving different types of video data, processing and encoding / decoding it, and outputting the results for observation and analysis, problems in chip design can be identified and optimized and improved.

[0033] During the integration of multimedia systems, the prototype verification device can serve as an important testing tool. By connecting to different multimedia devices and receiving their output video data for verification, the compatibility and stability of the entire multimedia system can be ensured.

[0034] In related teaching and research fields, the prototype verification device can serve as an intuitive teaching tool to help students understand the working principles of multimedia chips and prototype verification methods. At the same time, researchers can also use the device to conduct research and innovation in multimedia technology.

[0035] Next, the structure of the prototype verification device will be described in detail.

[0036] In the entire device, in order to transmit configuration signals and data signals, in one embodiment, Figure 2 As shown, the first bus includes a first configuration bus and a first data bus, and the second bus includes a second configuration bus and a second data bus; the first configuration bus and the first data bus are respectively connected to the control module; the first configuration bus is also respectively connected to the second configuration bus and the codec module; the first data bus is also respectively connected to the second data bus, the first storage module and the codec module.

[0037] The second configuration bus is connected to the receiving module, the selection processing module and the reading and outputting module respectively; the second data bus is connected to the selection processing module and the reading and outputting module respectively.

[0038] The first configuration bus and the second configuration bus are used to configure the downstream modules respectively according to the configuration signal; the second data bus is used to receive the second video data and transmit the second video data to the first data bus, and the first data bus is used to transmit the second video data to the first storage module for storage; the first data bus is also used to transmit the second video data to the codec module for processing to obtain third video data, and transmit the third video data to the first storage module; the read output module is used to read the third video data in the first storage module through the second data bus and the first data bus in sequence.

[0039] In one embodiment, the control module may use an open source Reduced Instruction Set Computing (RISC) model. This type of CPU has two buses, namely a configuration bus and a data bus. The configuration bus is configured to different modules after passing through the first configuration bus, and the data bus can read and write data in the first storage module after passing through the first data bus. In addition, the control module can also perform other operations such as interrupt processing and debugging.

[0040] The first configuration bus, second configuration bus, first data bus, and second data bus are all Network on Chip (NOC), a common interconnect bus in SOC systems. Four NOC buses are used in this embodiment. The NOC in the first FPGA is tailored from an Application Specific Integrated Circuit (ASIC) design to reduce resource overhead by pruning some memory. Bus pruning typically involves reducing some cache, meaning the bus's existing memory is reduced or even partially removed. This does not affect functionality but only degrades performance. However, FPGA prototyping generally doesn't prioritize performance, only functionality. The NOC in the second FPGA is a newly added bus designed to adapt to the system architecture. The first and second configuration buses serve as configuration paths, while the first and second data buses serve as data paths. They are independent and do not interact with each other.

[0041] The control module sends corresponding signals to the first configuration bus and the first data bus, respectively. After receiving the configuration signal from the control module, the first configuration bus passes it to the connected second configuration bus and codec module. The second configuration bus then distributes the configuration signal to the receiving module, the selection and processing module, and the reading and output module, thereby configuring these downstream modules. Each module can adjust its operating parameters and status based on the configuration signal to meet different prototype verification requirements.

[0042] After the receiving module receives the first video data and processes it through the selection and processing module to obtain the second video data, the second video data first enters the second data bus. The second data bus acts as a bridge for data transmission, transferring the second video data to the first data bus. Based on system requirements, the first data bus is used to transfer the second video data to the first storage module for storage and subsequent reading and processing. Furthermore, the codec module reads the second video data via the first data bus and performs encoding and decoding operations on the second video data. The codec module processes the second video data to obtain third video data, which is then transferred to the first data bus. The third video data is then returned to the first storage module via the first data bus for storage.

[0043] When video data needs to be output, the reading and outputting module sequentially reads the third video data from the first storage module via the second data bus and the first data bus. The above design ensures the orderly flow of data between the modules and improves the stability and reliability of the system.

[0044] In order to realize the interconnection between each FPGA, in one embodiment, as shown in FIG. Figure 3 The first FPGA includes a first interconnection module and a second interconnection module, the second FPGA includes a third interconnection module, and the third FPGA includes a fourth interconnection module; each interconnection module includes a bus transmission unit and a serial-to-parallel conversion unit connected to each other, the bus transmission unit is used to receive a corresponding configuration signal or video data signal, the serial-to-parallel conversion unit is used to convert the configuration signal or video data signal into a parallel signal, and transmit the parallel signal to the serial-to-parallel conversion unit in the other interconnection module; the serial-to-parallel conversion unit is also used to convert the parallel signal into a configuration signal or video data signal, and the bus transmission unit is also used to transmit the configuration signal or video data signal to the corresponding module; the bus transmission units in the first interconnection module and the second interconnection module are respectively connected to the first configuration bus and the first data bus; the bus transmission unit in the third interconnection module is respectively connected to the second configuration bus and the second data bus; the bus transmission unit in the fourth interconnection module is connected to the codec module; the serial-to-parallel conversion unit in the first interconnection module is connected to the serial-to-parallel conversion unit in the third interconnection module; and the serial-to-parallel conversion unit in the second interconnection module is connected to the serial-to-parallel conversion unit in the fourth interconnection module.

[0045] The bus transmission unit is C2C (Chip to Chip), and the serial-to-parallel conversion unit is Aurora. C2C can transmit the configuration bus and data bus, while Aurora mainly converts serial data into parallel data.

[0046] In the first FPGA, the bus transmission units of the first and second interconnect modules are connected to the first configuration bus and the first data bus, respectively. This enables efficient transmission of configuration signals and video data from the control module between the different FPGAs. The bus transmission units provide fast and stable data transmission channels, ensuring timely information exchange between the FPGAs and meeting the high-volume data transmission requirements during multimedia chip prototype verification.

[0047] During the data transmission process, different modules may require different data formats. When the signal is transmitted between different FPGAs, the serial-to-parallel conversion unit can convert the serial signal into a parallel signal, or convert the parallel signal into a serial signal, to adapt to the input and output requirements of different modules. For example, when the signal in the first FPGA is to be transmitted to the second FPGA, the serial-to-parallel conversion unit in the first interconnection module converts the serial signal from the first FPGA into a parallel signal, and then transmits the parallel signal to the third interconnection module through the connection with the serial-to-parallel conversion unit in the third interconnection module. The serial-to-parallel conversion unit in the third interconnection module then converts the parallel signal into a serial signal as needed or directly transmits it to the corresponding module in the second FPGA in the form of a parallel signal. The serial-to-parallel conversion function improves the compatibility and flexibility of the system, enabling different FPGAs and modules to work better together.

[0048] The interconnection module ensures that configuration signals and video data can be transmitted smoothly between the three FPGAs, and through the flexible conversion of the serial-to-parallel conversion unit, it adapts to the requirements of different modules for signal forms, thereby improving the compatibility and stability of the system.

[0049] In one embodiment, Figure 4As shown, the first FPGA also includes multiple low-speed modules, including: Customer Relationship Management (CRM), Watchdog Timer (WDT), Inter Integrated Circuit (IIC), Timer, Serial Peripheral Interface (SPI), General Purpose Input / Output (GPIO), Universal Asynchronous Receiver / Transmitter (UART), and Pulse Width Modulation (PWM). Most of these modules are essential for system startup and normal operation. Among them: CRM manages the clock and reset of the entire system architecture. In the data circuit, clock and reset are very important parts, affecting system function and stability; WDT can regularly check the internal status of the chip and send a restart signal to the chip if an error occurs. The watchdog command has the highest priority among program interrupts. The IIC provides a set of IIC interfaces from the chip for connecting external IIC devices. Timers are used for various timing and frame retransmission tasks. The SPI provides a set of IIC interfaces from the chip for connecting external SPI devices, primarily flash memory. GPIO can be used as the chip's input and output ports, often used for debugging. The UART is used for serial communication, bridging the gap between the software driver and the CPU. The PWM outputs voltage levels for external devices. The low-speed module has little to do with the multimedia system, but it is an important component of the system. Further details are not provided in this embodiment.

[0050] Since the video stream data volume is large under high resolution and high frame rate, the codec module is required to perform encoding and decoding to reduce the data bandwidth pressure. However, when the resolution and frame rate of the video data are not high, in order to improve the performance and test efficiency of the prototype verification device, in one embodiment, Figure 4As shown, the second FPGA also includes a second storage module and a data selection module, the common end of the data selection module is connected to the second data bus, the first branch end of the data selection module is connected to the first data bus, and the second branch end of the data selection module is connected to the second storage module; the data selection module is used to transmit the second video data from the second data bus to the first storage module or the second storage module; when the resolution of the video data is greater than or equal to the first preset threshold and / or the frame rate of the video data is greater than or equal to the second preset threshold, the third FPGA is turned on, the data selection module is used to transmit the second video data to the first storage module through the first data bus, and the codec module is used to transmit the second video data to the first storage module through the first preset threshold. A data bus reads the second video data in the first storage module, encodes and decodes the second video data to obtain third video data, and stores the third video data in the first storage module through the first data bus; the reading and output module is used to read the third video data in the first storage module through the second data bus and the first data bus and output it; when the resolution of the video data is less than a first preset threshold and / or the frame rate of the video data is less than a second preset threshold, the third FPGA is turned off, the data selection module is used to transfer the second video data to the second storage module for storage, and the reading and output module is used to read the second video data in the second storage module through the second data bus and output it.

[0051] In some embodiments, reference Figure 4 The first branch end of the data selection module is connected to the first data bus through the third interconnection module and the first interconnection module in sequence.

[0052] In one embodiment, when the frame rate and resolution are high, such as 4K 60fps, the data volume is very large, processing is time-consuming, and bandwidth pressure increases. Therefore, after encoding and decoding the second video data using the codec module in the third FPGA, the data volume is much smaller. Therefore, it is not necessary to enable the third FPGA. At high resolutions and frame rates, using the codec module in the third FPGA will improve performance, and not enabling the codec module will not affect functionality. In other words, from the perspective of FPGA prototyping, whether to enable the codec module can be selected regardless of the resolution and frame rate. For video data with high resolutions and frame rates, performance is higher. For video data with low resolutions and frame rates, enabling the third FPGA is not necessary, but it can be enabled. The functionality is built into the third FPGA and can be determined based on actual needs.

[0053] In one embodiment, the data selection module is controlled by a select signal. In the second FPGA, the select signals are connected to the FPGA's dip switch, and 0-way data is selected by default. The data selection module is relatively flexible and can support 1 input and 2 outputs, or 2 inputs and 1 output. It can also be cascaded and expanded to achieve multiple inputs and multiple outputs.

[0054] The first and second storage modules serve as the primary cache modules for the entire prototype verification device. They cache information including CPU kernel boot image data, video streams or image data written by various modules, and more. This embodiment uses a Xilinx Double Data Rate (DDR) controller with 4GB memory granules on the platform. All platforms with Direct Memory Access (DMA) or Advanced eXtensible Interface (AXI) bus interfaces can access the storage modules. In the first FPGA, the first storage module can be directly accessed by the CPU via the NOC bus. Other FPGAs require cross-platform access via the bus transfer unit from other FPGAs. This includes all multimedia modules in the second FPGA and the codec module in the third FPGA, which must rely on the bus transfer unit and serial-to-parallel conversion unit to access the first storage module in the first FPGA. Similarly, the second storage module in the second FPGA can only be directly accessed by modules on the local platform via the NOC bus, and modules in other FPGAs also require the bus transfer unit and serial-to-parallel conversion unit to access it.

[0055] In one embodiment, Figure 4As shown, the upstream of the receiving module is the camera, and the downstream of the receiving module is the selection and processing module. The selection and processing module includes a first MUX (full name: Multiplexer) module, a second MUX module and an Image Signal Processing (ISP) module. The first video data from the camera passes through the first MUX module and is then given to the ISP module for processing. The ISP module is mainly used in multimedia systems. Its main function is to process image signals. The ISP module can enrich the application scenarios of videos. The ISP Core in the ISP module can further improve the quality of video data to obtain second video data. Considering the resource limitations of FPGA chips, up to two different ISP modules are supported, which can be from different manufacturers. The images processed by the first MUX module are processed separately by two different ISP modules, and the image quality effects of the two ISP modules are compared. The upstream module of the ISP module is the data stream selected by the first MUX module, and the downstream is given to the second MUX module. The data selection module then selects to cache the second video data in the second storage module or the first storage module. It can be used flexibly according to software requirements. Figure 4 As shown, when the select signal is 0, the second video data is cached in the second storage module, and when the select signal is 1, the second video data is cached in the first storage module.

[0056] When the video data has a high resolution or frame rate, the data volume is large. In this case, the data selection module transfers the second video data to the first storage module. The codec module in the third FPGA can encode and decode the second video data stored in the first storage module, effectively reducing data bandwidth pressure. Through the encoding and decoding process, the video data can be compressed or converted into a format more suitable for transmission and storage, ensuring that the system can still operate and process the video data stably even with large data volumes. When the video data has a low resolution and frame rate, the data volume is relatively small, placing less pressure on the bandwidth. To improve device performance and test efficiency, the third FPGA is shut down at this time to reduce unnecessary resource usage. The data selection module transfers the second video data to the second storage module for storage. The read and output module directly reads and outputs the second video data from the second storage module without processing by the codec module. This reduces data processing steps, improves data transmission speed, and improves system response time.

[0057] Through the above method, the system's working mode is dynamically adjusted according to different video data characteristics, so that the prototype verification device can perform optimally in various situations and meet the verification requirements of video data with different resolutions and frame rates.

[0058] In order to read the video data in the storage module and output it for display, in one embodiment, Figure 4 As shown, the reading and output module includes a reading unit and an output unit, the reading unit is connected to the second data bus, and the output unit is connected to the reading unit; the reading unit is used to read the third video data in the first storage module, or read the second video data in the second storage module; the output unit is used to output the third video data or the second video data, and output it to a display screen of a corresponding format through a corresponding cable to output an image.

[0059] Among them, the output formats include High Definition Multimedia Interface (HDMI), DisplayPort (DP), Video Graphics Array (VGA) and Mobile Industry Processor Interface TX (MIPI TX).

[0060] Among them, the reading unit, as a part of the reading output module, reads the video data from the corresponding storage module according to the current working status of the system and the data storage location. In one embodiment, as long as the corresponding data exists in the first storage module or the second storage module, the reading unit will read it. If the video data has been processed by the codec module and stored in the first storage module, the reading unit reads the third video data from the first storage module. If the resolution and frame rate of the video data are low and stored in the second storage module, the reading unit reads the second video data from the second storage module. The output unit is connected to the reading unit and receives the video data read by the reading unit. The output unit has the ability to output multiple formats and can convert video data into different formats such as HDMI, DP, VGA and MIPI TX. These formats cover various common display interface types, so that the prototype verification device can be connected to different types of displays.

[0061] Once the output unit determines the output format, it transmits the video data to a display screen of the corresponding format via the appropriate cable. The display screen processes and displays the received video data, producing an image. Developers and testers can visually observe the output effects of the video data and evaluate and analyze the multimedia chip prototype verification process.

[0062] In order to further improve the output effect of video data, in one embodiment, Figure 4 As shown, the second FPGA also includes a computing module, which is connected to the second data bus; the computing module is used to read the third video data in the first storage module, or read the second video data in the second storage module, and perform precision calculation on the third video data or the second video data, and re-store the third video data or the second video data after precision calculation into the first storage module or the second storage module.

[0063] The accuracy calculation includes Lens Distortion Correction (LDC), Oriented FAST and Rotated BRIEF (ORB), and Semi-Global Matching (SGM).

[0064] When further processing of the video data is required, the computing module, through connection to the second data bus, can access the third video data after codec processing in the first storage module, or, when the video data resolution and frame rate are low, access the second video data in the second storage module. Alternatively, the computing module operates before the codec module.

[0065] The functions of the calculation module will be further explained below with reference to specific examples.

[0066] In one embodiment, when the reading unit reads the third video data from the first storage module or reads the second video data from the second storage module, if the image output through the output unit and the corresponding display screen is a distorted image, the third video data is read from the first storage module or the second video data is read from the second storage module through the calculation module, and the corresponding video data is processed by the LDC in the calculation module and then sent back to the first storage module or the second storage module, and then re-read by the reading unit and re-displayed through the output unit and the display screen. At this time, the image displayed on the display screen is the image after the distortion is corrected.

[0067] In one embodiment, if the data is directly output on the display screen without being processed by the ORB in the computing module, the content output by the display screen is the content recorded by the camera. The ORB is used to extract key information from the video data in specific situations. In one embodiment, if the camera is used to capture vehicle violation information, after the camera obtains the corresponding video data, it is stored in the first storage module or the second storage module. After ORB processing, it can be used to identify which vehicle has violated the law (running a red light or illegally parked, etc.). After extracting the relevant vehicle information, the corresponding vehicle is indicated by marking it with a red frame or other means, and then stored again in the first storage module or the second storage module. The data is re-read by the reading unit and re-displayed by the output unit and the display screen. At this time, the image displayed on the display screen can indicate the relevant illegal vehicle.

[0068] In one embodiment, when the camera is used in an artificial intelligence (AI) robot, the images captured by the camera do not need to be displayed on a display screen. For example, while walking, the robot uses the camera to capture images of the road ahead to determine its proper course. Without the SGM processing the video data captured by the camera, the robot would only know that there is a curve ahead and needs to turn, but would not know how many steps it will take to reach the curve. The SGM processes the video data captured by the camera and then stores it back in the first or second storage module. The robot's control system then acquires and analyzes the relevant video data to determine the distance to the curve and the number of steps required.

[0069] In summary, the calculation module is used to perform the aforementioned precision calculations on the corresponding video data. This improves the quality of the video data and enhances the output, providing developers and testers with more accurate and clearer images, and facilitating better evaluation of the performance and functionality of multimedia chips. The specific processes of each precision algorithm will not be described in detail in this embodiment.

[0070] In summary, the present invention uses three FPGA platforms in cascade to accommodate the entire multimedia architecture, making the functions and hierarchical structure more distinct. By distributing different functional modules across three FPGAs, the resource advantages of each FPGA can be fully utilized, improving the prototype verification capability of the multimedia chip. The first FPGA is primarily responsible for matters related to system startup, the second FPGA is responsible for processing the multimedia module's business, and the third FPGA independently accommodates the codec module. The three FPGA platforms are used in conjunction to achieve the highest possible performance for the multimedia system, solving the problem of insufficient resources on a single FPGA chip to accommodate the entire multimedia system. Ultimately, the present invention utilizes less than 70% of the resources of each FPGA chip.

[0071] In the above embodiment, a multimedia chip prototype verification device based on FPGA is proposed. In this embodiment, a multimedia chip prototype verification method based on FPGA is proposed. Figure 5 Shown, including: Step 101: The control module sends a configuration signal, and the first bus and the second bus configure downstream modules respectively according to the configuration signal.

[0072] Step 101 is primarily responsible for initializing and configuring the system. The control module, serving as the system's control center, issues configuration signals, which are transmitted to their respective downstream modules via the first and second buses. Downstream modules connected to the first bus include the first storage module and the codec module, while downstream modules connected to the second bus include the receiving module, the selection processing module, and the read and output module. The configuration signals set parameters and initialize the functions of these modules, ensuring they operate as intended. For example, the configuration signals can set the input data format of the receiving module, the processing algorithm of the selection processing module, and the encoding and decoding parameters of the codec module. This configuration process lays the foundation for subsequent video data processing and output.

[0073] Step 102: The receiving module receives first video data, the selection and processing module processes the first video data to obtain second video data, and the second video data is sequentially passed through the second bus and the first bus and then stored in the first storage module.

[0074] The receiving module receives first video data from an external source. The first video data can be a raw video signal from a camera, a video file, or other video source. The receiving module transmits the received first video data to the selection and processing module. The selection and processing module processes the first video data according to preset algorithms and rules, such as performing operations such as format conversion, filtering, and enhancement, to obtain second video data. The processed second video data has a format and quality that is more suitable for subsequent processing and storage. The second video data is then transmitted sequentially via the second bus and the first bus. The second bus transmits the second video data from the selection and processing module to the first bus, which then transmits the second video data to the first storage module for storage.

[0075] Step 103: the encoding and decoding module encodes and decodes the second video data in the first storage module to obtain third video data, and stores the third video data in the first storage module.

[0076] The codec module reads the second video data from the first storage module. The codec module encodes the second video data according to a preset encoding algorithm, converting it into a more compact format to reduce data volume. This encoded video data can be stored and transmitted more efficiently. Then, when the video data needs to be output, the codec module decodes the encoded video data to restore it to a playable format. The third video data obtained after the encoding and decoding process is stored again in the first storage module, awaiting reading and output by the read output module.

[0077] Step 104: The reading and outputting module reads the third video data in the first storage module and outputs it in a corresponding display manner.

[0078] The first bus includes a first configuration bus and a first data bus, and the second bus includes a second configuration bus and a second data bus; the first configuration bus and the second configuration bus respectively configure the downstream modules according to the configuration signal; the second data bus receives the second video data and transmits the second video data to the first data bus, and the first data bus transmits the second video data to the first storage module for storage; the first data bus also transmits the second video data to the codec module for processing to obtain third video data, and transmits the third video data to the first storage module; the reading and output module reads the third video data in the first storage module through the second data bus and the first data bus in sequence.

[0079] The read / output module can select different display modes for output based on system settings and user needs. For example, video data can be output to a display screen via interfaces such as HDMI, DP, VGA, or MIPI TX. The output video data can be high-quality video after codec processing or a variety of video data with different resolutions and frame rates. This allows users to intuitively observe the multimedia chip's processing of video data, thereby evaluating and verifying the chip's performance.

[0080] In summary, this FPGA-based multimedia chip prototype verification method verifies the functionality and performance of a multimedia chip. From system configuration to receiving, processing, encoding, decoding, and outputting video data, the entire process encompasses the key functions required for a multimedia chip in practical applications, providing an effective means for multimedia chip development and optimization.

[0081] In one embodiment, Figure 6 As shown, the second FPGA further includes a second storage module and a data selection module; the method further includes: Step 201: The data selection module transmits the second video data from the second data bus to the first storage module or the second storage module.

[0082] The data selection module receives the second video data from the second data bus and determines whether to transfer the second video data to the first storage module or the second storage module based on the system's operating status and the characteristics of the video data. This allows for flexible management of video data storage paths, improving system efficiency and performance.

[0083] Step 202: When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module transmits the second video data to the first storage module through the first data bus, the codec module reads the second video data in the first storage module through the first data bus, encodes and decodes the second video data to obtain third video data, and stores the third video data in the first storage module through the first data bus.

[0084] The reading and outputting module reads and outputs the third video data in the first storage module through the second data bus and the first data bus.

[0085] When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, it means that the amount of video data is large and encoding and decoding processing is required to reduce data bandwidth pressure. In this case, the third FPGA is turned on. The data selection module transfers the second video data to the first storage module so that the encoding and decoding module can encode and decode it. The processing of the encoding and decoding module can effectively compress and decompress the video data, improving the efficiency of data transmission and storage. In this way, even in the case of high resolution and high frame rate, the system can stably process and transmit video data.

[0086] Step 203: When the resolution of the video data is less than a first preset threshold and / or the frame rate of the video data is less than a second preset threshold, the third FPGA is turned off, the data selection module transfers the second video data to the second storage module for storage, and the reading and output module reads the second video data in the second storage module through the second data bus and outputs it.

[0087] When the resolution of the video data is less than the first preset threshold and / or the frame rate of the video data is less than the second preset threshold, it indicates that the amount of video data is relatively small and the pressure on the bandwidth is also small. At this time, in order to improve the performance and test efficiency of the device, the third FPGA is turned off to reduce unnecessary resource occupation. The data selection module transfers the second video data to the second storage module for storage. The reading output module directly reads the second video data in the second storage module through the second data bus and outputs it without being processed by the encoding and decoding module. This can reduce the data processing links, improve the data transmission speed and the response time of the system. In this way, according to the different characteristics of the video data, the system can automatically adjust the working mode, achieve optimal allocation of resources, and improve overall performance.

[0088] In one embodiment, the second FPGA further includes a computing module. In both step 202 and step 203, the method further includes: the computing module reading the third video data from the first storage module or the second video data from the second storage module, performing precision calculations on the third video data or the second video data, and re-storing the calculated third video data or the second video data into the first storage module or the second storage module. When further processing of the video data is required, the computing module, through a connection to the second data bus, can access the encoded and decoded third video data in the first storage module, or, if the video data resolution and frame rate are low, access the second video data in the second storage module. The computing module can improve the quality of video data and enhance output quality, providing developers and testers with more accurate and clear images, thereby facilitating better evaluation of the performance and functionality of multimedia chips.

[0089] The specific structure of the FPGA-based multimedia chip prototype verification device is shown in the above embodiment and will not be described in detail in this embodiment.

[0090] In the above embodiment, a multimedia chip prototype verification device based on FPGA is proposed. In this embodiment, the prototype verification device will be further described based on its actual usage scenario. In one embodiment, the third FPGA is used as an example: Figure 2 As shown: Step S1: After the hardware is ready, power on the platform.

[0091] Step S2: Confirm the dial status and select the data path. The select signals of the first MUX module and the second MUX module on the left and right of the ISP module are controlled by the dial 0 on the FPGA platform. The default is 0 and remains unchanged, that is, ISP0 is selected; the data selection module is controlled by the dial 1 of the second FPGA platform. When the default is 0, the second storage module is selected; when the dial is adjusted to 1, the first storage module in the first FPGA is selected.

[0092] Step S3: After the version is burned, the first storage module, the second storage module, the bus transmission unit and the serial-to-parallel conversion unit are initialized respectively.

[0093] Step S4: Users can write software to configure the driver and store it in the read-only memory (ROM) space within the control module. Upon powering on, the system automatically reads the ROM instructions in the control module and begins the control module's boot process (typically three stages, ultimately reaching the kernel).

[0094] Step S5: During the startup process, the control module initializes and configures the low-speed module, receiving module, selection processing module, calculation module, reading and output module, and encoding and decoding module through the first configuration bus and the second configuration bus, and finally enters the kernel.

[0095] Step S6: After the control module is started and other system modules are initialized, the user can rewrite the software driver to call each module to work under the kernel.

[0096] Step S7: Turn on the camera, the first video data is transmitted from the camera to the receiving module, and then output to the first MUX module. Since the dial switch is set to 0, the ISP0 module is selected to obtain the second video data. After passing through the second mux module, the second video data is given to the data selection module through the second data bus. Since the mux selection of the data selection module is 1, the second video data will be given to the C2C in the third interconnection module, and then to the Aurora of the third interconnection module, and then to the Aurora and C2C of the first interconnection module through the Serdes interface, and then through the first data bus, and finally the second video data is written to the first storage module.

[0097] Step S8: The computing module reads and writes data. After ISP0 processes the first video data, the computing module needs to read the second video data processed by ISP0 and stored in the first storage module through the second data bus and the first data bus in sequence (it can also read the third video data after encoding and decoding, provided that the encoding and decoding module works first and the computing module works later, and the selection is made according to different application scenarios). After the computing module processes the second video data, it writes it back to the first storage module through the second data bus and the first data bus in sequence.

[0098] Step S9: The codec module in the third FPGA reads the second video data from the first storage module through the fourth interconnection module and the second interconnection module, encodes the second video data, and then writes it back to the first storage module (it should be noted that the addresses written to the first storage module by different modules are different and there must be no conflict. The encoding and decoding stages of the codec module are relatively independent and can be operated as long as there is data in the first storage module. This step can also be performed after the ISP module completes writing and before the computing module reads).

[0099] Step S10: the codec module reads the encoded second video data from the first storage module through the fourth interconnection module and the second interconnection module, decodes the data to obtain third video data, and writes the third video data back to the first storage module.

[0100] Step S11: The reading unit reads the decoded third video data in the first storage module (if the codec module works before the computing module, the path for the reading unit to obtain data is consistent with the path for the computing module to obtain data, and the data can be read from the storage unit where the computing module writes the data). The reading unit supports multiple sets of video interface format outputs.

[0101] Step S12: The reading unit outputs the data to different output units, and displays the data on a display screen of a corresponding format via a cable.

[0102] The overall process when the third FPGA is not involved can be simply derived from the above process. Only in step S7, the data selection module's mux selection 1 is changed to 0, and the second video data is directly sent to the second storage module. The remaining operations are then performed. For more details, refer to the above embodiment and will not be repeated here.

[0103] The specific structure of the FPGA-based multimedia chip prototype verification device is shown in the above embodiment and will not be described in detail in this embodiment.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multimedia chip prototype verification device based on FPGA, characterized in that: include: a first FPGA, a second FPGA, and a third FPGA, wherein the first FPGA includes a control module, a first bus, and a first storage module; the second FPGA includes a second bus, a receiving module, a selection processing module, and a reading and output module; and the third FPGA includes a codec module; The control module is connected to the first bus, and the first bus is respectively connected to the second bus, the first storage module and the codec module; The second bus is connected to the receiving module, the selection processing module and the reading and outputting module respectively; the receiving module is connected to the selection processing module; The control module is used to send configuration signals, and the first bus and the second bus are used to configure downstream modules respectively according to the configuration signals; The receiving module is used to receive first video data, the selection and processing module is used to process the first video data to obtain second video data, and the second video data is stored in the first storage module after passing through the second bus and the first bus in sequence; The encoding and decoding module is used to encode and decode the second video data in the first storage module to obtain third video data, and store the third video data in the first storage module; The reading and outputting module is used to read the third video data in the first storage module and output it in a corresponding display manner.

2. The FPGA-based multimedia chip prototype verification device according to claim 1, characterized in that: The first bus includes a first configuration bus and a first data bus, and the second bus includes a second configuration bus and a second data bus; the first configuration bus and the first data bus are respectively connected to the control module; the first configuration bus is also respectively connected to the second configuration bus and the codec module; the first data bus is also respectively connected to the second data bus, the first storage module, and the codec module; The second configuration bus is connected to the receiving module, the selection processing module and the reading and outputting module respectively; the second data bus is connected to the selection processing module and the reading and outputting module respectively; The first configuration bus and the second configuration bus are used to configure downstream modules respectively according to the configuration signal; The second data bus is used to receive the second video data and transmit the second video data to the first data bus, and the first data bus is used to transmit the second video data to the first storage module for storage; The first data bus is further used to transmit the second video data to the encoding and decoding module for processing to obtain third video data, and transmit the third video data to the first storage module; The reading and outputting module is used to read the third video data in the first storage module through the second data bus and the first data bus in sequence.

3. The FPGA-based multimedia chip prototype verification device according to claim 2, characterized in that: The first FPGA includes a first interconnection module and a second interconnection module, the second FPGA includes a third interconnection module, and the third FPGA includes a fourth interconnection module; Each interconnection module includes a bus transmission unit and a serial-parallel conversion unit connected to each other, wherein the bus transmission unit is used to receive a corresponding configuration signal or video data signal, and the serial-parallel conversion unit is used to convert the configuration signal or video data signal into a parallel signal and transmit the parallel signal to the serial-parallel conversion unit in other interconnection modules; The serial-to-parallel conversion unit is further used to convert the parallel signal into a configuration signal or a video data signal, and the bus transmission unit is further used to transmit the configuration signal or the video data signal to the corresponding module; The bus transmission units in the first interconnection module and the second interconnection module are connected to the first configuration bus and the first data bus respectively; The bus transmission unit in the third interconnection module is connected to the second configuration bus and the second data bus respectively; the bus transmission unit in the fourth interconnection module is connected to the codec module; The serial-parallel conversion unit in the first interconnection module is connected to the serial-parallel conversion unit in the third interconnection module; the serial-parallel conversion unit in the second interconnection module is connected to the serial-parallel conversion unit in the fourth interconnection module.

4. The FPGA-based multimedia chip prototype verification device according to claim 2, characterized in that: The second FPGA further includes a second storage module and a data selection module, wherein a common end of the data selection module is connected to the second data bus, a first branch end of the data selection module is connected to the first data bus, and a second branch end of the data selection module is connected to the second storage module; The data selection module is used to transmit the second video data from the second data bus to the first storage module or the second storage module; When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module is used to transmit the second video data to the first storage module through the first data bus, the codec module is used to read the second video data in the first storage module through the first data bus, encode and decode the second video data to obtain third video data, and store the third video data in the first storage module through the first data bus; The reading and outputting module is used for reading and outputting the third video data in the first storage module through the second data bus and the first data bus; When the resolution of the video data is less than a first preset threshold and / or the frame rate of the video data is less than a second preset threshold, the third FPGA is turned off, the data selection module is used to transfer the second video data to the second storage module for storage, and the read output module is used to read the second video data in the second storage module through the second data bus and output it.

5. The FPGA-based multimedia chip prototype verification device according to claim 4, characterized in that: The reading and output module includes a reading unit and an output unit, the reading unit is connected to the second data bus, and the output unit is connected to the reading unit; The reading unit is configured to read the third video data in the first storage module, or read the second video data in the second storage module; The output unit is used to output the third video data or the second video data to a display screen of a corresponding format through a corresponding cable to output an image.

6. The FPGA-based multimedia chip prototype verification device according to claim 5, characterized in that: The second FPGA further includes a calculation module, and the calculation module is connected to the second data bus; The calculation module is used to read the third video data in the first storage module, or read the second video data in the second storage module, perform precision calculation on the third video data or the second video data, and re-store the third video data or the second video data after the precision calculation into the first storage module or the second storage module; When the reading unit reads the third video data from the first storage module or reads the second video data from the second storage module, if the image outputted by the output unit and the corresponding display screen is a distorted image, the third video data is read from the first storage module or the second video data is read from the second storage module by the computing module, and the corresponding video data is processed by the computing module before being sent back to the first storage module or the second storage module; The image is read again by the reading unit and displayed again by the output unit and the display screen. At this time, the image displayed on the display screen is the image after the distortion is corrected.

7. A multimedia chip prototype verification method based on FPGA, characterized in that: The method is applicable to the FPGA-based multimedia chip prototype verification device according to any one of claims 1 to 6, comprising: The control module sends a configuration signal, and the first bus and the second bus configure downstream modules respectively according to the configuration signal; The receiving module receives first video data, the selection and processing module processes the first video data to obtain second video data, and the second video data is sequentially passed through the second bus and the first bus and then stored in the first storage module; The encoding and decoding module encodes and decodes the second video data in the first storage module to obtain third video data, and stores the third video data in the first storage module; The reading and outputting module reads the third video data in the first storage module and outputs the data in a corresponding display manner.

8. The FPGA-based multimedia chip prototype verification method according to claim 7, characterized in that: The first bus includes a first configuration bus and a first data bus, and the second bus includes a second configuration bus and a second data bus; The reading and outputting module reading the third video data from the first storage module includes: The first configuration bus and the second configuration bus respectively configure downstream modules according to the configuration signal; The second data bus receives the second video data and transmits the second video data to the first data bus, and the first data bus transmits the second video data to the first storage module for storage; The first data bus further transmits the second video data to the encoding and decoding module for processing to obtain third video data, and transmits the third video data to the first storage module; The reading and outputting module sequentially reads the third video data in the first storage module through the second data bus and the first data bus.

9. The FPGA-based multimedia chip prototype verification method according to claim 8, characterized in that: The second FPGA further includes a second storage module and a data selection module; the method further includes: The data selection module transmits the second video data from the second data bus to the first storage module or the second storage module; When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module transmits the second video data to the first storage module through the first data bus, the codec module reads the second video data in the first storage module through the first data bus, encodes and decodes the second video data to obtain third video data, and stores the third video data in the first storage module through the first data bus; the read and output module reads the third video data in the first storage module through the second data bus and the first data bus and outputs the data; When the resolution of the video data is less than a first preset threshold and / or the frame rate of the video data is less than a second preset threshold, the third FPGA is turned off, the data selection module transfers the second video data to the second storage module for storage, and the reading and output module reads the second video data in the second storage module through the second data bus and outputs it.

10. The FPGA-based multimedia chip prototype verification method according to claim 9, characterized in that: The second FPGA further includes a calculation module; and the method further includes: The calculation module reads the third video data in the first storage module, or reads the second video data in the second storage module, and performs precision calculation on the third video data or the second video data, and stores the third video data or the second video data after precision calculation back into the first storage module or the second storage module.

Images