Multi-channel audio and video synchronous acquisition board

Abstract

The utility model relates to the technical field of audio and video acquisition, in particular to a multichannel audio and video synchronous acquisition board, which comprises a multichannel audio access unit, a photoelectric conversion module, a field programmable gate array (FPGA), a processor and a power supply module, the output end of the multi-channel audio access unit and the output end of the photoelectric conversion module are both connected with the input end of the FPGA, the output end of the FPGA is connected with the processor, and the power module is used for supplying power to the multi-channel audio access unit, the photoelectric conversion module, the FPGA and the processor. And the requirement of multi-channel synchronous acquisition of different digital video signals and audio signals is met.

Description

Technical Field

[0001] The utility model relates to the technical field of audio and video acquisition, in particular to a multi-channel audio and video synchronous acquisition board. Background Art

[0002] Audio and video capture technology is primarily used in the security field, playing a vital role in public safety and security monitoring. With the continuous advancement of technology, audio and video capture technology is increasingly being used in other personal spaces or specialized areas, such as interior scene monitoring and acquisition in cars and specialized aircraft. These applications are typically limited in size, placing high demands on miniaturized and lightweight audio and video capture equipment. This requires achieving simultaneous multi-channel audio and video capture on a compact board.

[0003] Currently, existing audio and video capture modules or devices typically utilize specialized audio and video codec chips to connect directly to external devices to capture the audio and video signals input by these devices. The chips then process the encoded audio and video and output them to other external devices. Therefore, the number of supported audio and video channels and formats depends on the physical interface performance of the audio and video codec chips themselves. However, single-chip audio / video codec chips typically support a limited number of channels. To achieve multi-channel audio and video capture, a large number of integrated audio / video codec chips are required. This inevitably leads to technical issues with multi-channel audio and video capture devices, such as complex design, large board size, and bulky equipment. These issues make them unable to meet the needs of space-constrained applications requiring multi-channel, synchronous capture of different digital video and audio signals.

[0004] To this end, Chinese patent CN114900622A discloses a multi-channel audio and video capture system based on HiSilicon chips. The system uses the HiSilicon Hi3521D V200, a quad-core processor, as the main control chip. Multiple MIPI D-PHY interfaces are integrated within the chip to improve the video input performance of the chip's digital interface. However, even with the highly integrated chip, maintaining synchronization between audio and video signals across different channels remains a technical challenge, particularly when processing digital video and audio signals in different formats. Furthermore, the data transmission interfaces between the audio and video capture modules and the main control chip in this patent are all MIPI interfaces, which requires a short transmission distance. Increased transmission distances can easily lead to video data transmission errors, making this system unsuitable for applications where the front-end audio and video sources are far from the back-end main control chip. Furthermore, this integrated solution limits the scalability of the audio and video capture system. Future support for more channels or different signal formats may require a complete redesign of the entire capture system. Utility Model Content

[0005] In response to the above technical problems, the present invention proposes a multi-channel audio and video synchronous acquisition board, which aims to achieve miniaturization of the acquisition board while meeting the demand for multi-channel synchronous acquisition of different digital video signals and audio signals.

[0006] To this end, the present invention adopts the following technical solutions:

[0007] A multi-channel audio and video synchronization acquisition board includes a multi-channel audio access unit, a photoelectric conversion module, an FPGA, a processor, and a power module. The multi-channel audio access unit is used to receive audio signals. The input end of the photoelectric conversion module uses optical fiber to receive video signals. The output end of the multi-channel audio access unit and the output end of the photoelectric conversion module are both connected to the input end of the FPGA. The output end of the FPGA is connected to the processor. The power module is used to supply power to the multi-channel audio access unit, the photoelectric conversion module, the FPGA, and the processor.

[0008] Preferably, the photoelectric conversion module has a plurality of independent parallel optical fiber channels.

[0009] Preferably, the output end of the photoelectric conversion module is connected to the input end of the FPGA via a high-speed serial interface.

[0010] Preferably, the output end of the FPGA is connected to the processor using a multi-channel MIPI video transmission interface.

[0011] Preferably, each of the MIPI video transmission interfaces includes at least two Lanes.

[0012] Preferably, the multi-channel audio access unit includes several operational amplifier circuits and a multi-channel A / D converter, the input end of the operational amplifier circuit is used to receive audio signals, the output ends of the several operational amplifier circuits are connected to the input end of the multi-channel A / D converter, and the output end of the multi-channel A / D converter is connected to the input end of the FPGA.

[0013] Preferably, the output end of the FPGA is connected to the processor using a PCIE interface.

[0014] Preferably, the multi-channel audio and video synchronization acquisition board includes a dynamic memory, and the dynamic memory is connected to the FPGA.

[0015] Preferably, the processor uses the HiSilicon Hi3559A chip.

[0016] The beneficial technical effects of the present utility model include at least:

[0017] 1. A multi-channel audio and video synchronization acquisition board is used. Through the ingenious design of the multi-channel audio access unit, optoelectronic conversion module, FPGA and other structural components in the acquisition board, the integration of the acquisition board is greatly improved, overcoming the technical difficulty of miniaturized and lightweight audio and video multi-channel acquisition equipment in maintaining synchronization of audio and video signals of different channels. While meeting the demand for multi-channel synchronous acquisition of different digital video signals and audio signals, the scalability of the acquisition board is greatly improved. Optical fiber is used for video signal reception, and after optoelectronic conversion by the optoelectronic conversion module, it is connected to the FPGA, which performs parallel acquisition of the input multi-channel video. Compared with the MIPI interface connection method used in the existing technology, the signal attenuation is smaller, the anti-interference ability is stronger, and the transmission distance is greatly increased, thereby avoiding the technical problem of video data transmission errors caused by the long transmission distance between the front-end video source and the back-end acquisition board;

[0018] 2. The number of video access channels can be further expanded by connecting multiple photoelectric conversion modules to the FPGA. The number of audio access channels can be further expanded by connecting multiple multi-channel A / D converters to the FPGA. Combined with the operational amplifier circuit to convert the signal of each front-end audio source, the acquisition board can meet the application requirements of multi-channel synchronous acquisition of digital video signals and audio signals in different formats without changing the main control chip and basic hardware architecture.

[0019] Other features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The present invention will be further described below with reference to the accompanying drawings:

[0021] Figure 1 This is a structural diagram of a multi-channel audio and video synchronization acquisition board according to an embodiment of the present utility model.

[0022] Figure 2 This is a schematic structural diagram of the photoelectric conversion module according to an embodiment of the present utility model.

[0023] Figure 3 This is a structural diagram of a multi-channel audio access unit according to an embodiment of the present utility model.

[0024] Figure 4 This is a structural diagram of the FPGA output end of an embodiment of the present utility model. DETAILED DESCRIPTION

[0025] The following is an explanation and description of the technical solutions of the embodiments of the present invention in conjunction with the drawings of the embodiments of the present invention, but the following embodiments are only preferred embodiments of the present invention and are not exhaustive. Based on the embodiments in the embodiments, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention.

[0026] In the following description, terms such as "inside", "outside", "up", "down", "left", "right", etc. that indicate directions or positional relationships are only used to facilitate the description of the embodiments and simplify the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, they should not be understood as limiting the present invention.

[0027] Please see the attached Figure 1 , Figure 1 The figure shows a schematic structural diagram of a multi-channel audio and video synchronization acquisition board provided by an embodiment of this specification.

[0028] like Figure 1 As shown, the multi-channel audio and video synchronization acquisition board may include at least: a multi-channel audio access unit, a photoelectric conversion module, an FPGA, a processor and a power supply module, wherein:

[0029] The multi-channel audio access unit is used to receive audio signals. The input end of the optoelectronic conversion module uses optical fiber to receive video signals. The output end of the multi-channel audio access unit and the output end of the optoelectronic conversion module are both connected to the input end of the FPGA. The output end of the FPGA is connected to the processor. The power module is used to power the multi-channel audio access unit, optoelectronic conversion module, FPGA and processor.

[0030] Among them, the specific implementation device of the photoelectric conversion module can be various types of photoelectric conversion devices, such as electro-optical modulators, photodiodes, etc. The specific selection depends on the format of the video signal, transmission distance, transmission speed, cost and specific requirements of the system. This embodiment does not limit this.

[0031] Among them, FPGA (Field-Programmable Gate Array) is an integrated circuit that contains programmable logic blocks (PLBs). Users can define the functions of these logic blocks through hardware description language (HDL). The programmability of FPGA means that they can be reconfigured many times without remanufacturing, which makes FPGA very suitable for applications that require flexibility and scalability. At the same time, FPGA has multiple I / O ports that can be connected to external devices such as sensors, displays, network interfaces, etc.

[0032] It is understood that the front-end video source can be various visible light or infrared cameras, as well as other types of video sources. The video source can output the video signal to the optoelectronic conversion module via a 10 Gigabit Ethernet port or ARINC818 interface. In this embodiment, optical fiber is used to receive the video signal, which is then converted by the optoelectronic conversion module and connected to the FPGA. The FPGA then performs parallel acquisition of the input multi-channel video. Compared to the MIPI interface connection method used in the prior art, this method has less signal attenuation, stronger anti-interference capabilities, and significantly increases the transmission distance, thereby avoiding the technical problem of video data transmission errors caused by the long transmission distance between the front-end video source and the back-end acquisition board.

[0033] In this embodiment, the output end of the multi-channel audio access unit and the output end of the photoelectric conversion module are connected to the input end of the FPGA, and then the output end of the FPGA is connected to the processor (ie Figure 1 The Hi3559A audio and video encoding circuit in the FPGA is connected, and the processor encodes the audio and video signals and outputs the encoded audio and video signals to the external device. Therefore, the FPGA can be used to perform synchronization processing during the audio and video acquisition process. The FPGA uses logical processing methods such as audio and video data alignment and time stamping according to video frames to achieve audio and video acquisition synchronization. Among them, the implementation method of audio and video synchronization processing by FPGA is the existing technology, and the implementation method of encoding audio and video signals by the processor is the existing technology.

[0034] Preferably, the processor in this embodiment can adopt the HiSilicon Hi3559A chip.

[0035] The HiSilicon Hi3559A chip provides 8K30 / 4K120 broadcast-quality digital video recording, supports 8-channel sensor input, supports H.265 encoding output or film-grade RAW data output, and integrates high-performance ISP processing. It also uses an advanced 12nm low-power process and a miniaturized package, and supports DDR4 / LPDDR4, making the Hi3559AV100 capable of supporting product miniaturization. Therefore, this embodiment can use the Hi3559A to perform real-time encoding processing such as H.264 / H.265 on each received video signal and PCM encoding on each received audio data.

[0036] It is understood that please see the attached Figure 2Each optoelectronic conversion module can access multiple video channels. Therefore, connecting multiple optoelectronic conversion modules to an FPGA can further expand the number of video access channels. The FPGA can also perform format conversion or pre-processing on video signals. This allows the acquisition board to meet the application requirements of synchronously acquiring multiple digital video signals in different formats without changing the main control chip (i.e., processor) or basic hardware architecture. The implementation of format conversion or pre-processing of video signals using an FPGA is currently available.

[0037] Furthermore, in this embodiment, the multi-channel audio and video synchronous acquisition board includes a dynamic memory, and the dynamic memory is connected to the FPGA.

[0038] Among them, dynamic memory (i.e. Figure 1 The DDR3 in the embodiment is a volatile memory whose storage unit is based on capacitors and is used to temporarily cache intermediate processing data. It can be understood that the entire multi-channel audio and video synchronization acquisition board provided by this embodiment does not include a data storage module (such as a solid-state drive, hard disk drive, etc.). The system design is more concise, making the size and weight of the entire acquisition board smaller, making it more suitable for portable or space-constrained applications. In addition, the dynamic memory can be quickly reconfigured, which helps the FPGA to more flexibly process and cache intermediate processing data to adapt to different audio and video synchronization acquisition requirements.

[0039] It is understandable that the devices used in this embodiment are not limited to the device models listed herein, and devices with the same functions may be used as replacements.

[0040] Furthermore, in this embodiment, the photoelectric conversion module has a plurality of independent parallel optical fiber channels.

[0041] For example, the rate of each channel of the optoelectronic conversion module in this embodiment is 10.3125 Gbps, which can fully meet the bandwidth requirement of video signal transmission.

[0042] Furthermore, in this embodiment, the output end of the photoelectric conversion module and the input end of the FPGA are connected using a high-speed serial interface.

[0043] For example, in this embodiment, the photoelectric conversion module and the FPGA are connected using a GTH high-speed serial interface. Figure 2 As shown in FIG, FPGA simultaneously collects the video signals output by two photoelectric conversion modules through the GTH high-speed serial interface, and can realize real-time acquisition of all input video signals.

[0044] Further, in this embodiment, please refer to the attached Figure 4 ,The output end of FPGA is connected to the processor using a multi-channel MIPI video transmission interface.

[0045] Furthermore, in this embodiment, each MIPI video transmission interface includes at least two Lanes.

[0046] Among them, Lane is a differential signal line used to widen the data bandwidth. This configuration allows a higher data transmission rate because each Lane can independently transmit data at a high rate, and the two Lanes together can provide double the bandwidth.

[0047] It can be understood that in this embodiment, the output end of the FPGA is connected to the processor using a multi-channel MIPI video transmission interface, and each video signal is transmitted using at least two Lane MIPI interfaces, thereby realizing simultaneous processing of video streams with a large amount of data, and achieving high-speed data transmission while maintaining low power consumption.

[0048] On the other hand, in this embodiment, please refer to the attached Figure 3 The multi-channel audio access unit includes several operational amplifier circuits and a multi-channel A / D converter. The input end of the operational amplifier circuit is used to receive audio signals. The output ends of the several operational amplifier circuits are connected to the input end of the multi-channel A / D converter. The output end of the multi-channel A / D converter is connected to the input end of the FPGA.

[0049] It is understood that the analog audio signal input from the front-end audio source is typically a differential signal input. Therefore, in this embodiment, an op amp circuit is used to preprocess the input analog audio signal. The op amp circuit converts the differential signal into a single-ended signal before inputting it into a multi-channel A / D converter. Each multi-channel A / D converter is interconnected with the FPGA using an independent digital interface. The FPGA controls the multi-channel A / D converters to read the converted single-ended audio signals in parallel, thereby achieving simultaneous acquisition of all audio inputs. The method of using an FPGA to control the multi-channel A / D converters to read the audio signals in parallel is conventional technology.

[0050] In this embodiment, by providing an operational amplifier circuit in front of the multi-channel A / D converter to convert the differential signal into a single-ended signal, noise can be further reduced, the quality of the analog audio signal of the front-end audio source can be improved, and the signal-to-noise ratio of the A / D converter can be improved. The operational amplifier circuit adjusts the level of the differential signal to within the input range of the multi-channel A / D converter, ensuring that the signal does not exceed the maximum input voltage of the multi-channel A / D converter, thereby avoiding distortion. At the same time, since the multi-channel A / D converter requires a certain input drive current to ensure conversion accuracy, the operational amplifier circuit can provide sufficient drive capability to drive the input of the multi-channel A / D converter, ensuring that the signal is not distorted due to insufficient drive capability during the A / D conversion process.

[0051] It is understandable that each multi-channel A / D converter can access multiple channels of analog audio. Therefore, using multiple multi-channel A / D converters to connect to the FPGA can further expand the number of audio access channels. Combined with the operational amplifier circuit to convert the signal of each front-end audio source, the acquisition board can meet the application requirements of multi-channel synchronous acquisition of audio signals in different formats without changing the main control chip (i.e., processor) and basic hardware architecture.

[0052] Further, in this embodiment, please refer to the attached Figure 4 , the output end of FPGA is connected to the processor using a PCIE interface.

[0053] like Figure 4 As shown in the figure, a PCIE interface is used between the FPGA and Hi3559A to transmit audio data. Hi3559A performs PCM encoding and other processing on each channel of received audio data. The encoded video and audio streams can be packaged according to the pre-defined frame format in the processor and transmitted to other external devices through the Gigabit Ethernet port.

[0054] Understandably, since the bandwidth of audio signals is relatively narrow, typically ranging from a few kilohertz to tens of kilohertz, the bandwidth requirements for the transmission medium are also relatively low. Even if optical fiber is not used for long-distance transmission, the quality of the audio signal will not be significantly affected. PCIE (Peripheral Component Interconnect Express) is a high-speed computer bus standard. The bandwidth of the PCIE interface can reach tens of Gbps, which is far higher than the bandwidth requirement of audio signals. Therefore, even if a PCIE interface is used to connect to the processor, the transmission speed of the audio signal will not be affected.

[0055] To sum up, the multi-channel audio and video synchronization acquisition board provided in the embodiments of this specification greatly improves the integration of the acquisition board through the ingenious design of the multi-channel audio access unit, optoelectronic conversion module, FPGA and other structural components in the acquisition board, overcomes the technical difficulty of miniaturized and lightweight audio and video multi-channel acquisition equipment in maintaining the synchronization of audio and video signals of different channels, and while meeting the needs of multi-channel synchronous acquisition of different digital video signals and audio signals, greatly improves the scalability of the acquisition board.

[0056] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Those skilled in the art will understand that the present invention includes, but is not limited to, the contents described in the drawings and the above specific embodiments. Any modifications that do not deviate from the functional and structural principles of the present invention are intended to be included within the scope of the claims.

Claims

1. A multi-channel audio and video synchronization acquisition board, characterized in that: It includes a multi-channel audio access unit, a photoelectric conversion module, an FPGA, a processor and a power module. The multi-channel audio access unit is used to receive audio signals. The input end of the photoelectric conversion module uses optical fiber to receive video signals. The output end of the multi-channel audio access unit and the output end of the photoelectric conversion module are both connected to the input end of the FPGA, and the output end of the FPGA is connected to the processor. The power module is used to power the multi-channel audio access unit, the photoelectric conversion module, the FPGA and the processor.

2. A multi-channel audio and video synchronization acquisition board as claimed in claim 1, characterized in that: The photoelectric conversion module has a plurality of independent parallel optical fiber channels.

3. A multi-channel audio and video synchronization acquisition board as claimed in claim 2, characterized in that: The output end of the photoelectric conversion module is connected to the input end of the FPGA using a high-speed serial interface.

4. A multi-channel audio and video synchronization acquisition board as claimed in claim 3, characterized in that: The output end of the FPGA is connected to the processor using a multi-channel MIPI video transmission interface.

5. A multi-channel audio and video synchronization acquisition board as claimed in claim 4, characterized in that: Each of the MIPI video transmission interfaces includes at least two Lanes.

6. A multi-channel audio and video synchronization acquisition board as claimed in claim 1, characterized in that: The multi-channel audio access unit includes several operational amplifier circuits and a multi-channel A / D converter. The input end of the operational amplifier circuit is used to receive audio signals, and the output ends of the several operational amplifier circuits are connected to the input end of the multi-channel A / D converter. The output end of the multi-channel A / D converter is connected to the input end of the FPGA.

7. A multi-channel audio and video synchronization acquisition board as claimed in claim 6, characterized in that: The output end of the FPGA is connected to the processor using a PCIE interface.

8. The multi-channel audio and video synchronization acquisition board according to claim 1, characterized in that: The multi-channel audio and video synchronous acquisition board includes a dynamic memory, and the dynamic memory is connected to the FPGA.

9. The multi-channel audio and video synchronization acquisition board according to claim 1, characterized in that: The processor uses the HiSilicon Hi3559A chip.

Images