A device and method for prototyping a multimedia chip based on FPGA

Abstract

The present application relates to chip prototype verification technical field, especially to a kind of multimedia chip prototype verification device and method based on FPGA, the whole multimedia architecture is housed using the way of three FPGA platform cascades, so that function and hierarchy are also more distinct;By distributing different functional modules on three FPGAs, the resource advantages of each FPGA can be fully utilized, and the prototype verification capability of the multimedia chip is improved. The first FPGA is mainly responsible for system startup related matters, the second FPGA is responsible for processing multimedia module business, since the coding and decoding module itself resource is relatively large and the function is relatively independent, and the coupling with other modules is not large, therefore, the third FPGA separately accommodates the coding and decoding module, three FPGA platforms are used in cooperation, to achieve the multimedia system works in the way of as high performance as possible, solve the problem that the resource of single-chip FPGA is not enough, and the whole multimedia system cannot be accommodated.

Description

TECHNICAL FIELD

[0001] The present application relates to the chip prototype verification technical field, in particular to a kind of multimedia chip prototype verification device and method based on FPGA. BACKGROUND

[0002] With the rapid development of multimedia technology and the increasing of multimedia chip shipment, people pay more and more attention to the verification stage of multimedia chip after design completion. Field programmable logic gate array (FPGA) is one of the important verification methods before chip tape-out. Prototype verification is based on chip prototype, which is the closest way to the real situation of chip in the prior art. However, FPGA prototype verification also has certain bottleneck problem, especially in large multimedia system, there are problems such as insufficient single FPGA chip resources, complex architecture, low performance and difficult implementation.

[0003] In the chip industry, the scale of multimedia chip is relatively large, and the architecture is relatively complex. Whether it is development cycle or verification cycle, it is relatively long. FPGA prototype verification, as one of the core steps of chip verification, usually has the problem of insufficient single FPGA chip resources when verifying multimedia chip, which cannot bear the hardware resources of the whole system of multimedia chip.

[0004] Therefore, it is urgent to overcome the defects of the prior art in the technical field. SUMMARY

[0005] The technical problem to be solved by the present application is how to solve the problem that single FPGA chip cannot bear the hardware resources of the whole system of multimedia chip.

[0006] The present application adopts the following technical solutions:

[0007] In a first aspect, a multimedia chip prototype verification device based on FPGA is provided, comprising: a first FPGA, a second FPGA and a third FPGA, the first FPGA comprising a control module, a first bus and a first storage module; the second FPGA comprising a second bus, a receiving module, a selection processing module and a reading output module; the third FPGA comprising a codec module;

[0008] The control module is connected with the first bus, and the first bus is connected with the second bus, the first storage module and the codec module respectively;

[0009] The second bus is connected with the receiving module, the selection processing module and the reading output module respectively; the receiving module is connected with the selection processing module;

[0010] The control module is configured to issue a configuration signal, and the first bus and the second bus are configured to configure downstream modules according to the configuration signal respectively.

[0011] The receiving module is configured to receive first video data, and the selection processing module is configured to process the first video data to obtain second video data, which is sequentially stored in the first storage module through the second bus and the first bus.

[0012] The encoding and decoding module is configured to encode and decode the second video data in the first storage module to obtain third video data, and store the third video data in the first storage module.

[0013] The reading and outputting module is configured to read the third video data in the first storage module and output the third video data through a corresponding display mode.

[0014] Preferably, the first bus includes a first configuration bus and a first data bus, and the second bus includes a second configuration bus and a second data bus; the first configuration bus and the first data bus are connected with the control module respectively; the first configuration bus is further connected with the second configuration bus and the encoding and decoding module respectively; the first data bus is further connected with the second data bus, the first storage module and the encoding and decoding module respectively.

[0015] The second configuration bus is connected with the receiving module, the selection processing module and the reading and outputting module respectively; and the second data bus is connected with the selection processing module and the reading and outputting module respectively.

[0016] The first configuration bus and the second configuration bus are configured to configure downstream modules according to the configuration signal respectively.

[0017] The second data bus is configured to receive the second video data and transmit the second video data to the first data bus, and the first data bus is configured to transmit the second video data to the first storage module for storage.

[0018] The first data bus is further configured to transmit the second video data to the encoding and decoding module for processing to obtain third video data, and transmit the third video data to the first storage module.

[0019] The reading and outputting module is configured to sequentially read the third video data in the first storage module through the second data bus and the first data bus.

[0020] Preferably, the first FPGA comprises a first interconnection module and a second interconnection module, the second FPGA comprises a third interconnection module, and the third FPGA comprises a fourth interconnection module;

[0021] Each interconnection module comprises a bus transmission unit and a serial-parallel conversion unit connected to each other, the bus transmission unit is used for receiving a corresponding configuration signal or video data signal, and the serial-parallel conversion unit is used for converting the configuration signal or video data signal into a parallel signal and transmitting the parallel signal to the serial-parallel conversion unit in another interconnection module;

[0022] The serial-parallel conversion unit is also used for converting the parallel signal into a configuration signal or video data signal, and the bus transmission unit is also used for transmitting the configuration signal or video data signal to a corresponding module;

[0023] The bus transmission units in the first interconnection module and the second interconnection module are connected with the first configuration bus and the first data bus respectively;

[0024] The bus transmission units in the third interconnection module are connected with the second configuration bus and the second data bus respectively, and the bus transmission units in the fourth interconnection module are connected with the codec module;

[0025] The serial-parallel conversion unit in the first interconnection module is connected with the serial-parallel conversion unit in the third interconnection module, and the serial-parallel conversion unit in the second interconnection module is connected with the serial-parallel conversion unit in the fourth interconnection module.

[0026] Preferably, the second FPGA further comprises a second storage module and a data selection module, the common end of the data selection module is connected with the second data bus, the first branch end of the data selection module is connected with the first data bus, and the second branch end of the data selection module is connected with the second storage module;

[0027] The data selection module is used for transmitting second video data from the second data bus to the first storage module or the second storage module;

[0028] When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module is configured to transmit the second video data to the first storage module through the first data bus, the encoding and decoding module is configured to read the second video data in the first storage module through the first data bus, encode and decode the second video data to obtain third video data, and store the third video data in the first storage module through the first data bus; and the reading and outputting module is configured to read the third video data in the first storage module through the second data bus and the first data bus and output the third video data.

[0029] When the resolution of the video data is less than the first preset threshold and / or the frame rate of the video data is less than the second preset threshold, the third FPGA is turned off, the data selection module is configured to transmit the second video data to the second storage module for storage, and the reading and outputting module is configured to read the second video data in the second storage module through the second data bus and output the second video data.

[0030] Preferably, the reading and outputting module comprises a reading unit and an output unit, the reading unit is connected with the second data bus, and the output unit is connected with the reading unit.

[0031] The reading unit is configured to read the third video data in the first storage module or read the second video data in the second storage module.

[0032] The output unit is configured to output the third video data or the second video data to a display screen of a corresponding format through a corresponding cable to output an image.

[0033] Preferably, the second FPGA further comprises a calculation module, and the calculation module is connected with the second data bus.

[0034] The calculation module is configured to read the third video data in the first storage module or read the second video data in the second storage module, perform precision calculation on the third video data or the second video data, and store the third video data or the second video data after the precision calculation in the first storage module or the second storage module again.

[0035] When the reading unit reads third video data from the first storage module or reads second video data from the second storage module, if the image output by the output unit and the corresponding display screen is a distorted image, the third video data is read from the first storage module or the second video data is read from the second storage module by the calculation module, and the corresponding video data is processed by the calculation module and then sent back to the first storage module or the second storage module; the reading unit is re-read, and the output unit and the display screen are re-displayed, and the image displayed on the display screen is the corrected image.

[0036] In a second aspect, a multimedia chip prototype verification method based on FPGA is provided, and the method is suitable for the multimedia chip prototype verification device based on FPGA in the first aspect, and the method comprises the following steps:

[0037] The control module issues a configuration signal, and the first bus and the second bus respectively configure downstream modules according to the configuration signal;

[0038] The receiving module receives first video data, the selection processing module processes the first video data to obtain second video data, and the second video data is sequentially stored in the first storage module through the second bus and the first bus;

[0039] The encoding and decoding module encodes and decodes the second video data in the first storage module to obtain third video data, and stores the third video data in the first storage module;

[0040] The reading and outputting module reads the third video data in the first storage module and outputs the third video data through a corresponding display mode.

[0041] Preferably, the first bus comprises a first configuration bus and a first data bus, and the second bus comprises a second configuration bus and a second data bus; the reading and outputting module reading the third video data from the first storage module comprises the following steps:

[0042] The first configuration bus and the second configuration bus respectively configure downstream modules according to the configuration signal;

[0043] The second data bus receives the second video data and transmits the second video data to the first data bus, and the first data bus transmits the second video data to the first storage module for storage;

[0044] The first data bus also transmits the second video data to the codec module for processing to obtain third video data, and transmits the third video data to the first storage module.

[0045] The reading output module reads the third video data in the first storage module through the second data bus and the first data bus in sequence.

[0046] Preferably, the second FPGA further comprises a second storage module and a data selection module; the method further comprises:

[0047] The data selection module transmits the second video data from the second data bus to the first storage module or the second storage module;

[0048] When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module transmits the second video data to the first storage module through the first data bus, the codec module reads the second video data in the first storage module through the first data bus, and encodes and decodes the second video data to obtain third video data, and stores the third video data in the first storage module through the first data bus; the reading output module reads the third video data in the first storage module through the second data bus and the first data bus and outputs the third video data;

[0049] When the resolution of the video data is less than the first preset threshold and / or the frame rate of the video data is less than the second preset threshold, the third FPGA is turned off, the data selection module transmits the second video data to the second storage module for storage, and the reading output module reads the second video data in the second storage module through the second data bus and outputs the second video data.

[0050] Preferably, the second FPGA further comprises a calculation module; the method further comprises:

[0051] The calculation module reads the third video data in the first storage module or the second video data in the second storage module, performs precision calculation on the third video data or the second video data, and re-stores the third video data or the second video data after precision calculation in the first storage module or the second storage module.

[0052] Compared with the prior art, the present application has the following beneficial effects:

[0053] Due to the limited resources of a single FPGA chip, the whole multimedia architecture is accommodated by using a mode of cascading three FPGA platforms, so that the functions and hierarchy are more distinct, and by distributing different functional modules on the three FPGAs, the resource advantages of the respective FPGAs can be fully utilized to improve the prototype verification capability of the multimedia chip. The first FPGA is mainly responsible for the system startup related matters, the second FPGA is responsible for processing the business of the multimedia module, and since the coding and decoding module itself has relatively large resources and relatively independent functions, and has little coupling with other modules, the third FPGA separately accommodates the coding and decoding module, the three FPGA platforms are used in cooperation to make the multimedia system work in the most high-performance mode, and the problem that the resources of a single FPGA chip are insufficient to accommodate the whole multimedia system is solved. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0055] Figure 1 is a structure schematic diagram of a multimedia chip prototype verification device based on FPGA provided by the embodiment of the present application;

[0056] Figure 2 is a connection structure schematic diagram of a configuration bus and a data bus of a multimedia chip prototype verification device based on FPGA provided by the embodiment of the present application;

[0057] Figure 3 is a connection structure schematic diagram of an interconnection module of a multimedia chip prototype verification device based on FPGA provided by the embodiment of the present application;

[0058] Figure 4 is a specific structure schematic diagram of a multimedia chip prototype verification device based on FPGA provided by the embodiment of the present application;

[0059] Figure 5 is a flow schematic diagram of a multimedia chip prototype verification method based on FPGA provided by the embodiment of the present application;

[0060] Figure 6 is another flow schematic diagram of a multimedia chip prototype verification method based on FPGA provided by the embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0062] Unless otherwise required by context, the term "comprises" or "comprising" as used in this specification is taken to mean the inclusion since but not limited to. In the description of the specification, the terms "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example" or "some examples" are intended to indicate that the described implementation, implementation or example is included in at least one embodiment or example of the present disclosure. The illustrative representation of the above terms does not necessarily mean the same embodiment or example. In addition, the specific features, structures, materials or characteristics described can be included in any one or more embodiments or examples in any appropriate manner, i.e. although they are carried in the embodiment or example of the above terms due to the order of appearance and location, they are not limited to the combination of one embodiment or example.

[0063] In the description of the present application, the terms "first", "second" are only used for description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features limited by "first", "second" can be explicitly or implicitly included in one or more features. In the description of the embodiments of the present disclosure, unless otherwise stated, the meaning of "multiple" is two or more. In addition, for example, in the description, the same type of nouns can also be described as two independent individuals by adding "A", "B" at the end, in which case the features limited by "A", "B" are only used for the purpose of distinguishing the same type of individual description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0064] In describing some embodiments, "coupled", "coupled" and "connected" and their derivatives can be used. For example, the term "connected" can be used to describe some embodiments to indicate that two or more components have direct physical or electrical contact with each other. For example, the term "coupled" can be used to describe some embodiments to indicate that two or more components have direct physical or electrical contact. However, the term "connected" or "coupled" can also refer to two or more components that do not have direct contact with each other, but still cooperate or interact with each other, such as "optical coupling", "wireless connection", etc. The embodiments disclosed herein are not necessarily limited to the content of the present application.

[0065] In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0066] The prior art often uses a single FPGA to perform prototype verification on a multimedia chip. Since the single FPGA chip has limited resources, it cannot bear the hardware resources of the entire system of the multimedia chip, resulting in an unsatisfactory prototype verification effect on the multimedia chip. In order to solve the above problems, the embodiment of the application provides a multimedia chip prototype verification device based on FPGA, as shown in Figure 1 The control module is used to issue a configuration signal, and the first bus and the second bus are used to configure downstream modules according to the configuration signal. The receiving module is used to receive first video data, and the selection processing module is used to process the first video data to obtain second video data, which is stored in the first storage module in sequence through the second bus and the first bus. The codec module is used to encode and decode the second video data in the first storage module to obtain third video data, and store the third video data in the first storage module. The read output module is used to read the third video data in the first storage module and output it through a corresponding display mode.

[0067] The control module is used to issue a configuration signal, and the first bus and the second bus are used to configure downstream modules according to the configuration signal. The receiving module is used to receive first video data, and the selection processing module is used to process the first video data to obtain second video data, which is stored in the first storage module in sequence through the second bus and the first bus. The codec module is used to encode and decode the second video data in the first storage module to obtain third video data, and store the third video data in the first storage module. The read output module is used to read the third video data in the first storage module and output it through a corresponding display mode.

[0068] The control module can be a central processing unit (CPU), which can coordinate the parallel work of software and hardware. Users can write software drivers to let the CPU send read-write processing instructions, configure downstream modules, and handle related transactions in the system. Downstream modules refer to all modules in the entire prototype verification device except the control module, the first bus, and the second bus.

[0069] The first bus is connected with multiple modules, on one hand, it transmits configuration signals to the first storage module and the encoding and decoding module in the first FPGA where the first bus is located, and performs corresponding initialization configuration, etc.; on the other hand, the first bus is also connected with the second bus, and transmits the configuration signals to the second bus, so that the second bus performs initialization configuration on the receiving module, the selection processing module and the reading and outputting module.

[0070] After the initialization configuration of all downstream modules is completed, the receiving module starts to work, receives first video data from the outside, for example, receives first video data from a camera. The receiving module transmits the first video data to the selection processing module, and the selection processing module processes the first video data according to a preset algorithm and rule, for example, performs format conversion, filtering and other operations, to obtain second video data. The second video data is transmitted in turn through the second bus and the first bus, and finally stored in the first storage module for temporary storage.

[0071] The encoding and decoding module encodes and decodes the second video data stored in the first storage module. The encoding process can compress or convert the video data into a specific format, so as to facilitate storage and transmission; the decoding process restores the encoded video data into a playable format. After the encoding and decoding operation, third video data is obtained, and the encoding and decoding module stores the third video data into the first storage module.

[0072] The reading and outputting module reads the third video data from the first storage module, and outputs through a corresponding display mode, for example, connects a display screen to display an image or transmits the video data to an external device through other output interfaces for further processing.

[0073] Through the above structure and the corresponding working process, the modules in the three FPGAs cooperate with each other, realize the functions of video data receiving, processing, storage, encoding and decoding and outputting in the prototype verification process of the multimedia chip, and realize efficient prototype verification of the multimedia chip.

[0074] The prototype verification device provided by the embodiment of the application has the following effects:

[0075] In the research and development process of the multimedia chip, the prototype verification device can help the developer to quickly verify the functions and performance of the chip. By receiving different types of video data, processing and encoding and decoding the video data, and outputting the results for observation and analysis, problems in the chip design can be found in time, and optimization and improvement can be performed.

[0076] In the integration process of the multimedia system, the prototype verification device can be used as an important test tool. By connecting different multimedia devices and receiving the output video data for verification, the compatibility and stability of the entire multimedia system can be ensured.

[0077] In the related teaching and research field, the prototype verification device can be used as an intuitive teaching tool to help students understand the working principle of the multimedia chip and the prototype verification method. Meanwhile, researchers can also use the device to conduct research and innovation of multimedia technology.

[0078] Next, the structure of the prototype verification device will be described in detail.

[0079] In the entire device, in order to be able to transmit configuration signals and data signals, in an embodiment, as shown in Figure 2 The first bus includes a first configuration bus and a first data bus, and the second bus includes a second configuration bus and a second data bus; the first configuration bus and the first data bus are connected with the control module respectively; the first configuration bus is also connected with the second configuration bus and the codec module respectively; the first data bus is also connected with the second data bus, the first storage module and the codec module respectively.

[0080] The second configuration bus is connected with the receiving module, the selection processing module and the reading output module respectively; the second data bus is connected with the selection processing module and the reading output module respectively.

[0081] The first configuration bus and the second configuration bus are used to configure the downstream modules according to the configuration signals respectively; the second data bus is used to receive the second video data and transmit the second video data to the first data bus; the first data bus is used to transmit the second video data to the first storage module for storage; the first data bus is also used to transmit the second video data to the codec module for processing to obtain third video data and transmit the third video data to the first storage module; the reading output module is used to read the third video data in the first storage module through the second data bus and the first data bus in turn.

[0082] In an embodiment, the control module can be selected from open-source Reduced Instruction Set Computing (RISC) models. Such a CPU has two buses, namely a configuration bus and a data bus. After passing through the first configuration bus, the configuration bus is configured to different modules. After passing through the first data bus, the data bus can read and write access data in the first storage module. In addition, the control module can also perform some interrupt processing, debug and other operations.

[0083] The first configuration bus, the second configuration bus, the first data bus and the second data bus are all Network On Chip (NOC), which is a commonly used interconnection bus in SOC system. In the embodiment, four NOC buses are used. The NOC in the first FPGA is designed and tailored based on Application Specific Integrated Circuit (ASIC) to reduce resource consumption and delete part of the memory. Generally, the way to tailor the bus is to reduce part of the cache, that is, to make the original memory of the bus smaller or even to remove part of the memory. In this way, the function will not be affected, and only the performance will be reduced. However, generally, the FPGA prototype verification does not care much about the performance, and only the function is verified. The NOC in the second FPGA is a newly added bus to adapt to the system architecture. The first configuration bus and the second configuration bus are used for configuration path, and the first data bus and the second data bus are used for data path. The two are independent and have no data interaction.

[0084] The control module sends corresponding signals to the first configuration bus and the first data bus respectively. After receiving the configuration signal from the control module, the first configuration bus transmits it to the second configuration bus and the codec module connected thereto. The second configuration bus further distributes the configuration signal to the receiving module, the selection processing module and the reading output module, so as to realize the configuration of these downstream modules. Each module can adjust its working parameters and state according to the configuration signal to adapt to different prototype verification requirements.

[0085] When the receiving module receives the first video data and the selection processing module processes the first video data to obtain the second video data, the second video data first enters the second data bus. The second data bus plays a role of a bridge for data transmission and transmits the second video data to the first data bus. According to the requirements of the system, the first data bus is used to transmit the second video data to the first storage module for storage, so as to be read and processed subsequently. On the other hand, the codec module reads the second video data through the first data bus and performs encoding and decoding operation on the second video data. The codec module processes the second video data to obtain the third video data and transmits the third video data to the first data bus again. The first data bus sends the third video data back to the first storage module for storage.

[0086] When the video data needs to be output, the reading output module reads the third video data from the first storage module through the second data bus and the first data bus in sequence. The above design ensures the orderly flow of data between the modules and improves the stability and reliability of the system.

[0087] To realize the interconnection between the FPGAs, in one embodiment, as Figure 3 described, the first FPGA includes a first interconnection module and a second interconnection module, the second FPGA includes a third interconnection module, and the third FPGA includes a fourth interconnection module; each interconnection module includes a bus transmission unit and a serial-parallel conversion unit connected with each other, the bus transmission unit is used to receive a corresponding configuration signal or a video data signal, and the serial-parallel conversion unit is used to convert the configuration signal or the video data signal into a parallel signal and transmit the parallel signal to a serial-parallel conversion unit in another interconnection module; the serial-parallel conversion unit is also used to convert the parallel signal into the configuration signal or the video data signal, and the bus transmission unit is also used to transmit the configuration signal or the video data signal to a corresponding module; the bus transmission units in the first interconnection module and the second interconnection module are connected with the first configuration bus and the first data bus respectively; the bus transmission units in the third interconnection module are connected with the second configuration bus and the second data bus respectively; the bus transmission unit in the fourth interconnection module is connected with the codec module; the serial-parallel conversion unit in the first interconnection module is connected with the serial-parallel conversion unit in the third interconnection module; and the serial-parallel conversion unit in the second interconnection module is connected with the serial-parallel conversion unit in the fourth interconnection module.

[0088] In the above embodiment, the bus transmission unit is C2C (Chip To Chip), and the serial-parallel conversion unit is Aurora. C2C can transmit a configuration bus and a data bus, and Aurora is mainly used for converting serial data and parallel data.

[0089] In the first FPGA, the bus transmission units of the first interconnection module and the second interconnection module are connected with the first configuration bus and the first data bus respectively, so that the configuration signal issued by the control module and the video data can be efficiently transmitted between different FPGAs. The bus transmission unit can provide a fast and stable data transmission channel, so that information can be timely exchanged between the FPGAs, and the demand for large amount of data transmission in the multimedia chip prototype verification process can be met.

[0090] During data transmission, different modules may require different data formats. When signals are transmitted between different FPGAs, the serial-parallel conversion unit can convert serial signals into parallel signals or convert parallel signals into serial signals to adapt to the input and output requirements of different modules. For example, when a signal in the first FPGA needs to be transmitted to the second FPGA, the serial-parallel conversion unit in the first interconnection module converts the serial signal from the first FPGA into a parallel signal, and then transmits the parallel signal to the third interconnection module through the connection with the serial-parallel conversion unit in the third interconnection module. The serial-parallel conversion unit in the third interconnection module converts the parallel signal into a serial signal as needed or directly transmits the parallel signal to the corresponding module in the second FPGA. The serial-parallel conversion function improves the compatibility and flexibility of the system, enabling different FPGAs and modules to work better together.

[0091] The interconnection module ensures that the configuration signal and the video data can be smoothly transmitted between the three FPGAs, and through the flexible conversion of the serial-parallel conversion unit, it adapts to the requirements of different modules for signal form, improving the compatibility and stability of the system.

[0092] In one embodiment, as Figure 4As shown, the first FPGA further includes a plurality of low-speed modules, including: Customer Relationship Management (CRM), Watch Dog Timer (WDT), Inter Integrated Circuit (IIC), Timer, Serial Peripheral Interface (SPI), General Purpose Input / Output (GPIO), Universal Asynchronous Receiver / Transmitter (UART), and Pulse Width Modulation (PWM), most of which are indispensable modules for system startup and normal operation. Among them: CRM manages the clock and reset of the entire system architecture, and the clock and reset are very important parts in the data circuit, which are related to the system function and stability; WDT can periodically check the internal situation of the chip and send a restart signal to the chip once an error occurs. The watchdog command has the highest priority in the program interruption; IIC can lead a group of IIC interfaces from the chip for connecting external IIC devices; Timer is used for various timing and frame retransmission tasks; SPI can lead a group of IIC interfaces from the chip for connecting external spi devices, mainly flash; GPIO can be used as the input and output IO port of the chip, commonly used for debug function; UART is used for serial communication and is the bridge for software drivers and CPU communication; PWM can output a level and can be used by external devices. The low-speed module has little relationship with the multimedia system, but it is an important part of the system. More specifically, it is not described in detail in this embodiment.

[0093] Due to the large amount of video stream data under high resolution and high frame rate, a codec module is needed to code and decode to reduce the data bandwidth pressure. However, when the resolution and frame rate of the video data are not high, in order to improve the performance of the prototype verification device and the test efficiency, in one embodiment, as shown in Figure 4As shown, the second FPGA further comprises a second storage module and a data selection module, a common end of the data selection module is connected with the second data bus, a first branch end of the data selection module is connected with the first data bus, and a second branch end of the data selection module is connected with the second storage module; the data selection module is configured to transmit second video data from the second data bus to the first storage module or the second storage module; when the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is opened, the data selection module is configured to transmit the second video data to the first storage module through the first data bus, the codec module is configured to read the second video data in the first storage module through the first data bus, encode and decode the second video data to obtain third video data, and store the third video data in the first storage module through the first data bus; the reading and outputting module is configured to read the third video data in the first storage module through the second data bus and the first data bus and output the third video data; when the resolution of the video data is less than the first preset threshold and / or the frame rate of the video data is less than the second preset threshold, the third FPGA is closed, the data selection module is configured to transmit the second video data to the second storage module for storage, and the reading and outputting module is configured to read the second video data in the second storage module through the second data bus and output the second video data.

[0094] In some embodiments, with reference to Figure 4 , the first branch end of the data selection module is connected with the first data bus through the third interconnection module and the first interconnection module in sequence.

[0095] In one embodiment, when the frame rate and the resolution are large, such as 4K 60 frames, the data amount is very large, the processing time is long, and the bandwidth pressure is large, so the data amount is much smaller after the second video data is encoded and decoded by the codec module in the third FPGA, and it is not necessary to open the third FPGA. In the case of large resolution and frame rate, the codec module in the third FPGA can improve the performance, and the function will not be affected without opening the codec module. That is, from the perspective of FPGA prototype verification, whether the codec module is opened or not can be selected regardless of the resolution and the frame rate. For video data with large resolution and frame rate, the performance is higher, and for video data with small resolution and frame rate, it is not necessary to open the third FPGA, but the third FPGA can also be opened. The function is made inside, and the actual demand can be determined.

[0096] In one embodiment, the data selection module is controlled by a select signal, in the second FPGA, the select signal is connected to the DIP switch of the FPGA, and the default selection is 0-way data. The data selection module is flexible and can support 1-in-2-out or 2-in-1-out. It can also be cascaded and expanded for use, achieving multiple-in and multiple-out.

[0097] The first storage module and the second storage module are main cache modules of the entire prototype verification device. The cached information includes image data of a CPU startup kernel, video stream or image data written by each module, and the like. In this embodiment, a double data rate (DDR) controller of xilinx is used, and the particle size of the 4G memory on the platform is used. All modules with direct memory access (DMA) function or advanced eXtensible Interface (AXI) bus interface can access the storage module. In the first FPGA, the first storage module can be directly accessed by the CPU through the NOC bus. The remaining modules need to be accessed from other FPGAs across the platform through the bus transmission unit. All multimedia modules in the second FPGA and the codec module in the third FPGA must rely on the bus transmission unit and the serial-parallel conversion unit to access the first storage module in the first FPGA. Similarly, the second storage module in the second FPGA can only be directly accessed by the local platform module through the NOC bus. The modules in other FPGAs also need to pass through the bus transmission unit and the serial-parallel conversion unit to access.

[0098] In one embodiment, as Figure 4As shown, upstream of the receiving module is a camera, and downstream of the receiving module is a selection processing module, which includes a first MUX module, a second MUX module, and an image signal processing (ISP) module. The first video data from the camera is processed by the first MUX module and then sent to the ISP module for processing. The ISP module is mainly applied in multimedia systems and mainly functions to process image signals. The ISP module can enrich the application scenarios of video, and the ISP Core in the ISP module can further improve the quality of video data to obtain second video data. Considering the resource limitations of the FPGA chip, at most 2 different ISP modules can be supported, which can be from different manufacturers. The images processed by the two ISP modules are compared in terms of quality. The upstream module of the ISP module is the data stream after selection by the first MUX module, and the downstream is the second MUX module. The second video data is cached to the second storage module or the first storage module through the data selection module, which can be flexibly used according to software requirements. Figure 4 As shown, when the select signal is 0, the second video data is cached to the second storage module, and when the select signal is 1, the second video data is cached to the first storage module.

[0099] When the resolution or frame rate of the video data is high, the data amount is large, and at this time, the data selection module transmits the second video data to the first storage module. The encoding and decoding module in the third FPGA can perform encoding and decoding operations on the second video data stored in the first storage module, effectively reducing the data bandwidth pressure. Through the encoding and decoding process, the video data can be compressed or converted into a format more suitable for transmission and storage, so that the system can still stably run and process video data in the case of large data amount. When the resolution and frame rate of the video data are low, the data amount is relatively small, and the pressure on the bandwidth is also small. In order to improve the performance and test efficiency of the device, the third FPGA is turned off at this time to reduce unnecessary resource occupation. The data selection module transmits the second video data to the second storage module for storage. The read output module directly reads the second video data from the second storage module and outputs it without the need for processing by the encoding and decoding module, thereby reducing the data processing link and improving the data transmission speed and system response time.

[0100] In the above manner, according to different video data characteristics, the working mode of the system is dynamically adjusted, so that the prototype verification device can exert the best performance in various situations and meet the verification needs of video data with different resolutions and frame rates.

[0101] In order to read and output the video data in the storage module for display, in an embodiment, as shown in Figure 4 The read output module includes a reading unit and an output unit. The reading unit is connected with the second data bus, and the output unit is connected with the reading unit. The reading unit is configured to read the third video data in the first storage module or the second video data in the second storage module. The output unit is configured to output the third video data or the second video data to a display screen of a corresponding format through a corresponding cable to output an image.

[0102] The output format includes a High Definition Multimedia Interface (HDMI), a DisplayPort (DP), a Video Graphics Array (VGA), and a Mobile Industry Processor Interface TX (MIPI TX).

[0103] The reading unit is part of the read output module and reads the video data from the corresponding storage module according to the current working state of the system and the data storage location. In an embodiment, the reading unit reads the data as long as the corresponding data exists in the first storage module or the second storage module. If the video data is processed by the codec module and stored in the first storage module, the reading unit reads the third video data from the first storage module. If the resolution and frame rate of the video data are low, the video data is stored in the second storage module, and the reading unit reads the second video data from the second storage module. The output unit is connected with the reading unit and receives the video data read by the reading unit. The output unit has the ability to output in multiple formats and can convert the video data into different formats such as HDMI, DP, VGA, and MIPI TX. These formats cover various display screen interface types commonly used at present, so that the prototype verification device can be connected with different types of display screens.

[0104] After the output unit determines the output format, the video data is transmitted to the display screen of the corresponding format through the corresponding cable. After the display screen receives the video data, it performs corresponding processing and display to output an image. The developers and testers can directly observe the output effect of the video data and evaluate and analyze the prototype verification process of the multimedia chip.

[0105] In order to further improve the output effect of the video data, in an embodiment, as shown inFigure 4 As shown, the second FPGA further comprises a calculation module connected with the second data bus; the calculation module is used to read third video data in the first storage module or second video data in the second storage module, and perform precision calculation on the third video data or the second video data, and re-store the third video data or the second video data after precision calculation into the first storage module or the second storage module.

[0106] The precision calculation includes Lens Distortion Correction (LDC), Oriented FAST and Rotated BRIEF (ORB) and Semi Global Matching (SGM).

[0107] When further processing of the video data is needed, the calculation module can access the third video data after codec processing in the first storage module through the connection with the second data bus, or access the second video data in the second storage module when the video data resolution and frame rate are low. Alternatively, the calculation module works before the codec module.

[0108] The function of the calculation module will be further described below in combination with specific examples.

[0109] In one embodiment, when the reading unit reads the third video data from the first storage module or the second video data from the second storage module, if the image output through the output unit and the corresponding display screen is a distorted image, the calculation module reads the third video data from the first storage module or the second video data from the second storage module, and processes the corresponding video data through the LDC in the calculation module before sending it back to the first storage module or the second storage module, then the reading unit reads again, and the output unit and the display screen display again, at this time, the image displayed on the display screen is the image after distortion correction.

[0110] In one embodiment, if the output is directly on the display screen without processing by the ORB in the calculation module, the content output by the display screen is the content recorded by the camera, and the ORB is used to extract key information in the video data in specific situations. In one embodiment, if the camera is used to capture vehicle violation information, after the camera obtains the corresponding video data, it is stored in the first storage module or the second storage module, and after ORB processing, it can be used to identify which vehicle is in violation (red light running or illegal parking, etc.). After the relevant vehicle information is extracted, the corresponding vehicle is indicated by methods such as red box, and then re-stored in the first storage module or the second storage module. The reading unit is re-read, and the output unit and the display screen are re-displayed. At this time, the image displayed on the display screen can indicate the relevant violation vehicle.

[0111] In one embodiment, when the camera is used for an artificial intelligence (AI) robot, the image captured by the camera does not need to be displayed through the display screen. For example, the robot determines how to proceed by capturing the image of the road ahead during walking. If the video data captured by the camera is not processed by the SGM, the robot only knows that there is a bend ahead that needs to be turned, but does not know how many steps to take to reach the bend. After the SGM processes the video data captured by the camera, the relevant video data is re-stored in the first storage module or the second storage module. By obtaining and analyzing the relevant video data through the control system of the robot, it can be determined how far away the bend is and how many steps need to be taken.

[0112] In summary, the calculation module is used to perform the above precision calculation on the corresponding video data. The calculation module can improve the quality of the video data, improve the output effect, and provide developers and testers with more accurate and clearer images, which helps to better evaluate the performance and functions of multimedia chips. The specific process of each precision algorithm is not described in detail in this embodiment.

[0113] In summary, the application uses three FPGA platforms in cascade to accommodate the entire multimedia architecture, so that the function and hierarchy are more distinct. By distributing different functional modules on the three FPGAs, the resource advantages of each FPGA can be fully utilized, and the prototype verification capability of the multimedia chip is improved. The first FPGA is mainly responsible for system startup related matters, the second FPGA is responsible for processing multimedia module business, and therefore the third FPGA separately accommodates the codec module. The three FPGA platforms are used in cooperation to achieve the highest performance of the multimedia system, solve the problem that the resources of a single FPGA chip are not enough to accommodate the entire multimedia system, and the resource utilization of each FPGA chip is below 70%.

[0114] In the above embodiment, a FPGA-based multimedia chip prototype verification device is proposed, and in the present embodiment, a FPGA-based multimedia chip prototype verification method will be proposed, as shown in the figure, comprising: Figure 5

[0115] Step 101: The control module issues a configuration signal, and the first bus and the second bus respectively configure the downstream modules according to the configuration signal.

[0116] Among them, the step 101 is mainly responsible for the initialization configuration of the system. The control module as the control center of the system issues a configuration signal, and the configuration signal is transmitted to the respective downstream modules through the first bus and the second bus. The downstream modules connected by the first bus include the first storage module, the codec module and the like, and the downstream modules connected by the second bus include the receiving module, the selection processing module and the reading output module. The function of the configuration signal is to set the parameters and initialize the functions of these modules, so as to ensure that they can work in the predetermined way. For example, the configuration signal can set the input data format of the receiving module, the processing algorithm of the selection processing module, the encoding and decoding parameters of the codec module, etc. Through the configuration process, the foundation is laid for subsequent video data processing and output.

[0117] Step 102: The receiving module receives the first video data, the selection processing module processes the first video data to obtain the second video data, and the second video data is stored in the first storage module in sequence through the second bus and the first bus.

[0118] ​The receiving module receives first video data from outside. The first video data can be raw video signals from a camera, a video file or other video sources. The receiving module transmits the received first video data to the selection processing module. The selection processing module processes the first video data according to preset algorithms and rules, such as format conversion, filtering, enhancement and other operations, to obtain second video data. The processed second video data has a format and quality more suitable for subsequent processing and storage. Then, the second video data is transmitted through the second bus and the first bus in turn. The second bus transmits the second video data from the selection processing module to the first bus, and the first bus transmits the second video data to the first storage module for storage.

[0119] Step 103: The codec module encodes and decodes the second video data in the first storage module to obtain third video data, and stores the third video data in the first storage module.

[0120] The codec module reads the second video data from the first storage module. The codec module encodes the second video data according to a preset encoding algorithm to convert it into a more compact format to reduce the amount of data. The encoded video data can be more efficiently stored and transmitted. Then, when the video data needs to be output, the codec module decodes the encoded video data to restore it to a playable format. The third video data obtained after the encoding and decoding process is stored in the first storage module again, waiting to be read and output by the reading and output module.

[0121] Step 104: The reading and output module reads the third video data in the first storage module and outputs it through a corresponding display mode.

[0122] The first bus includes a first configuration bus and a first data bus, and the second bus includes a second configuration bus and a second data bus; the first configuration bus and the second configuration bus configure downstream modules according to the configuration signal respectively; the second data bus receives the second video data and transmits the second video data to the first data bus, and the first data bus transmits the second video data to the first storage module for storage; the first data bus also transmits the second video data to the codec module for processing to obtain third video data, and transmits the third video data to the first storage module; the reading and output module reads the third video data in the first storage module through the second data bus and the first data bus in turn.

[0123] The reading and output module can select different display modes for output according to the settings of the system and the needs of the user. For example, video data can be output to a display screen through interfaces such as HDMI, DP, VGA, or MIPI TX to display images. The output video data can be high-quality video after encoding and decoding processing, or different video data selected according to different resolutions and frame rates. In this way, the user can intuitively observe the processing effect of the multimedia chip on the video data, thereby evaluating and verifying the performance of the chip.

[0124] In summary, the FPGA-based multimedia chip prototype verification method realizes the verification of the function and performance of the multimedia chip. From configuring the system to receiving, processing, encoding and decoding, and outputting video data, the entire process covers the main functions required by the multimedia chip in actual application, providing an effective means for the development and optimization of the multimedia chip.

[0125] In one embodiment, as shown in Figure 6 The second FPGA further includes a second storage module and a data selection module, and the method further includes:

[0126] Step 201: The data selection module transmits second video data from the second data bus to the first storage module or the second storage module.

[0127] The data selection module receives second video data from the second data bus. The role of the data selection module is to determine whether to transmit the second video data to the first storage module or the second storage module according to the running state of the system and the characteristics of the video data. The storage path of the video data can be flexibly managed to improve the efficiency and performance of the system.

[0128] Step 202: When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module transmits the second video data to the first storage module through the first data bus, the encoding and decoding module reads the second video data in the first storage module through the first data bus, encodes and decodes the second video data to obtain third video data, and stores the third video data in the first storage module through the first data bus.

[0129] The reading and output module reads the third video data in the first storage module through the second data bus and the first data bus and outputs it.

[0130] When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, it means that the amount of video data is large at this time, and needs to be encoded and decoded to reduce the data bandwidth pressure. In this case, the third FPGA is opened. The data selection module transmits the second video data into the first storage module, so that the encoding and decoding module can perform encoding and decoding operations on it. The processing of the encoding and decoding module can effectively compress and decompress the video data, and improve the efficiency of data transmission and storage. In this way, even in the case of high resolution and high frame rate, the system can stably process and transmit video data.

[0131] Step 203: When the resolution of the video data is less than the first preset threshold and / or the frame rate of the video data is less than the second preset threshold, the third FPGA is closed, the data selection module transmits the second video data into the second storage module for storage, and the reading output module reads the second video data in the second storage module through the second data bus and outputs.

[0132] When the resolution of the video data is less than the first preset threshold and / or the frame rate of the video data is less than the second preset threshold, it indicates that the amount of video data is relatively small, and the pressure on the bandwidth is also small. At this time, in order to improve the performance of the device and the test efficiency, the third FPGA is closed to reduce unnecessary resource occupation. The data selection module transmits the second video data into the second storage module for storage. The reading output module directly reads the second video data in the second storage module through the second data bus and outputs, without the need to pass through the processing of the encoding and decoding module. In this way, the link of data processing can be reduced, and the transmission speed of data and the response time of the system can be improved. In this way, according to the different characteristics of the video data, the system can automatically adjust the working mode to realize the optimal allocation of resources and improve the overall performance.

[0133] In one embodiment, the second FPGA further comprises a calculation module, and no matter in step 202 or step 203, the method further comprises: the calculation module reads the third video data in the first storage module or reads the second video data in the second storage module, and performs precision calculation on the third video data or the second video data, and re-stores the third video data or the second video data after precision calculation into the first storage module or the second storage module. When further processing of the video data is needed, the calculation module can access the third video data after codec processing in the first storage module or access the second video data in the second storage module through the connection with the second data bus when the resolution and frame rate of the video data are low. The calculation module can improve the quality of the video data, improve the output effect, provide more accurate and clearer images for developers and testers, and help better evaluate the performance and function of the multimedia chip.

[0134] For the specific structure of the FPGA-based multimedia chip prototype verification device, refer to the above embodiments, which will not be repeated in this embodiment.

[0135] In the above embodiment, an FPGA-based multimedia chip prototype verification device is proposed, and in this embodiment, the prototype verification device will be further described in its actual use scenario. In one embodiment, taking the participation of a third FPGA in work as an example: Figure 2 As shown in the figure:

[0136] Step S1: After the hardware is ready, the platform is powered on.

[0137] Step S2: Confirm the dial code state, select the data path, and the select signals of the first MUX module and the second MUX module on the left and right of the ISP module are controlled by the dial code 0 on the FPGA platform, and by default, 0 is kept unchanged, that is, ISP0 is selected; the data selection module is controlled by the dial code 1 of the second FPGA platform, and when 0 is selected by default, the second storage module is selected; when the dial code is adjusted to 1, the first storage module in the first FPGA is selected.

[0138] Step S3: After the version is burned, the first storage module, the second storage module, the bus transmission unit, and the serial-parallel conversion unit are respectively initialized.

[0139] Step S4: The user can configure the drive by writing software, and solidify it into the read-only memory (ReadOnly Memory, abbreviated as ROM) space in the control module. After the system is powered on, the ROM instructions in the control module will be automatically read, and the control module will start at all levels (generally three levels, and finally enter the kernel).

[0140] Step S5: In the start-up process, the control module initializes the configuration low-speed module, the receiving module, the selection processing module, the calculation module, the read output module, and the codec module, etc. through the first configuration bus and the second configuration bus, respectively, and finally enters the kernel.

[0141] Step S6: After the control module is started and the initialization of other modules of the system is completed, the user can write software drivers again to call the work of each module under the kernel.

[0142] Step S7: The camera is started, the first video data is transmitted from the camera to the receiving module, and then output to the first MUX module. Since the code dial is set to 0, the second video data is selected by the ISP0 module, and then transmitted to the data selection module through the second mux module and the second data bus. Since the mux of the data selection module selects 1, the second video data is transmitted to C2C in the third interconnection module, and then to Aurora in the third interconnection module, and then to Aurora and C2C in the first interconnection module through the Serdes interface, and finally written to the first storage module through the first data bus.

[0143] Step S8: The calculation module reads and writes data. After the ISP0 processes the first video data, the calculation module needs to read the second video data (or the third video data after encoding, provided that the codec module works first and the calculation module works later, and the selection is based on different application scenarios) stored in the first storage module through the ISP0 and then through the second data bus and the first data bus. After the calculation module processes the second video data, it is written back to the first storage module through the second data bus and the first data bus.

[0144] Step S9: The codec module in the third FPGA reads the second video data from the first storage module through the fourth interconnection module and the second interconnection module, encodes the second video data, and then writes it back to the first storage module (where it needs to be noted that the addresses of different modules written to the first storage module are not the same and cannot conflict. The codec stage of the codec module is relatively independent, and it can operate as long as there is data in the first storage module. This step can also be operated after the ISP module writes and before the calculation module reads).

[0145] Step S10: The codec module reads the encoded second video data from the first storage module through the fourth interconnection module and the second interconnection module, decodes the second video data to obtain the third video data, and then writes the third video data back to the first storage module.

[0146] Step S11: the reading unit reads the decoded third video data in the first storage module (if the coding and decoding module works before the calculation module, the path of the reading unit to obtain the data is the same as the path of the calculation module to obtain the data, and the reading is performed from the storage unit to which the data is written by the calculation module), and the reading unit supports multiple sets of video interface format output.

[0147] Step S12: the reading unit outputs to different output units and displays on the corresponding format display screen through the cable.

[0148] When the third FPGA does not participate in the work, the overall process can also be simply deduced according to the above process, and only the mux selection of the data selection module in step S7 is changed from 1 to 0, and the second video data is directly given to the second storage module. Then the remaining operations are performed, and more specifically, reference is made to the above embodiment, and details are not repeated here.

[0149] For the specific structure of the FPGA-based multimedia chip prototype verification device, reference is made to the above embodiment, and details are not repeated here in this embodiment.

[0150] The above only describes the preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An FPGA-based multimedia chip prototyping device, comprising: The application relates to a video processing system, which comprises a first FPGA, a second FPGA and a third FPGA, wherein the first FPGA comprises a control module, a first bus and a first storage module; the second FPGA comprises a second bus, a receiving module, a selection processing module and a reading output module; and the third FPGA comprises a coding and decoding module. The control module is connected with the first bus, the first bus is connected with the second bus, the first storage module and the coding and decoding module respectively; The second bus is connected with the receiving module, the selection processing module and the reading output module respectively, and the receiving module is connected with the selection processing module; The control module is used for issuing a configuration signal, and the first bus and the second bus are used for configuring downstream modules according to the configuration signal; The receiving module is used for receiving first video data, the selection processing module is used for processing the first video data to obtain second video data, and the second video data is stored in the first storage module through the second bus and the first bus in sequence; The coding and decoding module is used for coding and decoding the second video data in the first storage module to obtain third video data, and the third video data is stored in the first storage module; The reading output module is used for reading the third video data in the first storage module and outputting the third video data through a corresponding display mode. The first bus comprises a first configuration bus and a first data bus, the second bus comprises a second configuration bus and a second data bus, the first configuration bus and the first data bus are connected with the control module respectively, the first configuration bus is further connected with the second configuration bus and the coding and decoding module respectively, and the first data bus is further connected with the second data bus, the first storage module and the coding and decoding module respectively; 2. The FPGA-based multimedia chip prototyping device of claim 1, wherein, The second configuration bus is connected with the receiving module, the selection processing module and the reading output module respectively, and the second data bus is connected with the selection processing module and the reading output module respectively; The first configuration bus and the second configuration bus are used for configuring downstream modules according to the configuration signal; The second data bus is used for receiving the second video data and transmitting the second video data to the first data bus, and the first data bus is used for transmitting the second video data to the first storage module for storage; The first data bus is further used for transmitting the second video data to the coding and decoding module for processing to obtain third video data, and transmitting the third video data to the first storage module; The reading output module is used for reading the third video data in the first storage module through the second data bus and the first data bus in sequence. The first FPGA comprises a first interconnection module and a second interconnection module, the second FPGA comprises a third interconnection module, and the third FPGA comprises a fourth interconnection module.

3. The FPGA-based multimedia chip prototyping device of claim 2, wherein, ​ Each interconnection module comprises a bus transmission unit and a serial-parallel conversion unit, the bus transmission unit is configured to receive a corresponding configuration signal or video data signal, and the serial-parallel conversion unit is configured to convert the configuration signal or the video data signal into a parallel signal and transmit the parallel signal to the serial-parallel conversion unit in another interconnection module; The serial-parallel conversion unit is further configured to convert the parallel signal into the configuration signal or the video data signal, and the bus transmission unit is further configured to transmit the configuration signal or the video data signal to the corresponding module; The bus transmission units in the first interconnection module and the second interconnection module are connected with the first configuration bus and the first data bus, respectively; The bus transmission unit in the third interconnection module is connected with the second configuration bus and the second data bus, respectively; and the bus transmission unit in the fourth interconnection module is connected with the codec module; The serial-parallel conversion unit in the first interconnection module is connected with the serial-parallel conversion unit in the third interconnection module; and the serial-parallel conversion unit in the second interconnection module is connected with the serial-parallel conversion unit in the fourth interconnection module.

4. The FPGA-based multimedia chip prototyping device of claim 2, wherein, The second FPGA further comprises a second storage module and a data selection module, the data selection module is connected with the second data bus at a common end, connected with the first data bus at a first branch end, and connected with the second storage module at a second branch end; The data selection module is configured to transmit second video data from the second data bus to the first storage module or the second storage module; When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module is configured to transmit the second video data to the first storage module through the first data bus, the codec module is configured to read the second video data in the first storage module through the first data bus, encode and decode the second video data to obtain third video data, and store the third video data in the first storage module through the first data bus; The reading and outputting module is configured to read the third video data in the first storage module through the second data bus and the first data bus and output the third video data; When the resolution of the video data is less than the first preset threshold and / or the frame rate of the video data is less than the second preset threshold, the third FPGA is turned off, the data selection module is configured to transmit the second video data to the second storage module for storage, and the reading and outputting module is configured to read the second video data in the second storage module through the second data bus and output the second video data.

5. The FPGA-based multimedia chip prototyping device of claim 4, wherein, The reading and outputting module comprises a reading unit and an output unit, the reading unit is connected with the second data bus, and the output unit is connected with the reading unit; The reading unit is configured to read the third video data in the first storage module or read the second video data in the second storage module. The output unit is configured to output the third video data or the second video data to a display screen of a corresponding format through a corresponding cable to output an image.

6. The FPGA-based multimedia chip prototyping device of claim 5, wherein, The second FPGA further comprises a calculation module connected with the second data bus; The calculation module is configured to read third video data in the first storage module or second video data in the second storage module, and perform precision calculation on the third video data or the second video data, and store the third video data or the second video data after the precision calculation back into the first storage module or the second storage module; When the reading unit reads the third video data from the first storage module or the second video data from the second storage module, if the image output by the output unit and the corresponding display screen is a distorted image, the calculation module reads the third video data from the first storage module or the second video data from the second storage module, and sends the corresponding video data back to the first storage module or the second storage module after processing by the calculation module. The reading unit reads again, and the output unit and the display screen display again, and the image displayed by the display screen is the image after correction of distortion.

7. A method for FPGA-based multimedia chip prototyping, characterized in that, The method is suitable for the FPGA-based multimedia chip prototype verification device as claimed in any one of claims 1 to 6, and comprises: The control module sends a configuration signal, and the first bus and the second bus respectively configure downstream modules according to the configuration signal; The receiving module receives first video data, the selection processing module processes the first video data to obtain second video data, and the second video data is sequentially stored in the first storage module through the second bus and the first bus; The encoding and decoding module encodes and decodes the second video data in the first storage module to obtain third video data, and stores the third video data in the first storage module; The reading and outputting module reads the third video data in the first storage module and outputs through a corresponding display mode.

8. The FPGA-based multimedia chip prototyping and verification method according to claim 7, wherein, The first bus comprises a first configuration bus and a first data bus, and the second bus comprises a second configuration bus and a second data bus; The reading and outputting module reads the third video data in the first storage module comprises: The first configuration bus and the second configuration bus respectively configure downstream modules according to the configuration signal; The second data bus receives the second video data and transmits the second video data to the first data bus, and the first data bus transmits the second video data to the first storage module for storage; The first data bus further transmits the second video data to the encoding and decoding module for processing to obtain third video data, and transmits the third video data to the first storage module; The reading output module reads third video data in the first storage module through the second data bus and the first data bus in turn.

9. The FPGA-based multimedia chip prototyping and verification method according to claim 8, wherein, The second FPGA further comprises a second storage module and a data selection module; the method further comprises: The data selection module transmits second video data from the second data bus to the first storage module or the second storage module; When the resolution of the video data is greater than or equal to a first preset threshold and / or the frame rate of the video data is greater than or equal to a second preset threshold, the third FPGA is turned on, the data selection module transmits the second video data to the first storage module through the first data bus, the encoding and decoding module reads second video data in the first storage module through the first data bus, encodes and decodes the second video data to obtain third video data, and stores the third video data in the first storage module through the first data bus; the reading output module reads third video data in the first storage module through the second data bus and the first data bus and outputs the third video data; When the resolution of the video data is less than the first preset threshold and / or the frame rate of the video data is less than the second preset threshold, the third FPGA is turned off, the data selection module transmits the second video data to the second storage module for storage, and the reading output module reads second video data in the second storage module through the second data bus and outputs the second video data.

10. The FPGA-based multimedia chip prototyping and verification method according to claim 9, wherein, The second FPGA further comprises a calculation module; the method further comprises: The calculation module reads third video data in the first storage module or second video data in the second storage module, performs precision calculation on the third video data or the second video data, and re-stores the third video data or the second video data after the precision calculation in the first storage module or the second storage module.

Images