Multi-interface transmission camera
By introducing FPGA chip and multi-interface design into the camera, the delay problem when multiple domain controllers share camera data is solved, and multi-interface transmission is realized, reducing costs and improving real-time and adaptability are suitable for smart cars. Advanced assisted driving and autonomous driving applications.
Patent Information
- Application Number
- CN202420133033.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-19
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2034-01-19
AI Technical Summary
In the on-board application of existing cameras, when multiple domain controllers need to share camera data at the same location, it leads to delayed image data and real-time problems, affecting the planning and decision-making of advanced assisted driving and autonomous driving.
Design a camera with multi-interface transmission, using FPGA chip, lens, image sensor, master interface and slave interface, and implement multiple transmission channels through the main serializer and slave serializer, data conversion module and shunt module for data processing and Replication, use optical transceivers to achieve efficient data transmission, and adjust image sensor parameters through control interfaces to adapt to the needs of different domain controllers.
It realizes real-time transmission of images of multiple domain controllers at the same time by one camera, which reduces installation costs, reduces data delays, and improves the real-time and adaptability of image data.
Smart Images

Figure CN222073233U_ABST
Abstract
Description
Technical Field
[0001] The utility model relates to the field of cameras, in particular to a camera with multi-interface transmission. Background Art
[0002] With the rapid development of electronic technology, cameras have become an indispensable part of automotive parts and can be applied to vehicle assisted driving, 360-degree surround view, electronic rearview mirror, monitoring and recording, etc. In vehicle-mounted camera applications, usually one camera is connected to one control domain, which may result in multiple cameras at the same location, and multiple domains of smart cars may use camera data at a certain location. For example, the intelligent driving domain needs to perform intelligent driving algorithm calculations, the central control screen in the cockpit needs to display the image at that location, and the T-BOX needs to upload the image at that location to the cloud.
[0003] Existing plans include Figure 1 As shown in the figure, the camera is equipped with one interface, and the camera data is sent to domain 1 through the single interface, and then copied and forwarded to domain 2 and domain 3. Usually, frame buffering and MAC and PHY data parsing and forwarding are performed, which will inevitably cause image data delay, affect real-time performance, and affect the planning and decision-making of advanced driver assistance and autonomous driving vehicles. Therefore, the development of smart cars urgently needs to solve the low latency and real-time transmission of multiple different ECUs or domain controllers that need to collect the same camera data at the same time. Utility Model Content
[0004] Based on one of the defects in the prior art, the utility model provides a camera with multi-interface transmission, comprising at least: a lens, an image sensor, an FPGA chip, at least one master interface and at least one slave interface, wherein the image sensor is connected to the lens and the FPGA chip respectively, wherein at least one master serializer and at least one slave serializer are arranged on the FPGA chip;
[0005] The master serializer and the slave serializer are connected to the master interface and the slave interface respectively, and the master interface can receive downlink signals and be used to adjust parameters of the image sensor;
[0006] The uplink data is transmitted from the image sensor to the FPGA chip through the mipi interface.
[0007] A camera with multi-interface transmission, further preferably, the FPGA chip is configured as follows:
[0008] A data conversion module is used to convert and process the video data transmitted by the image transmitter according to a preset protocol;
[0009] The data distribution module is used to copy the video data transmitted by the data conversion module and output it to different channels;
[0010] At least two data transmission channels are used to transmit the video data forwarded by the data distribution module.
[0011] A camera with multi-interface transmission, further preferably, the data transmission channel at least includes:
[0012] A first cache module, used for caching received image data;
[0013] The encapsulation module is used to obtain data from the first cache module according to a preset rule and then encapsulate the data packet using a preset communication protocol.
[0014] A camera with multi-interface transmission, further preferably, a master serializer or a slave serializer is arranged inside the data transmission channel and connected to the encapsulation module, and the encapsulation module transmits the encapsulated data packet to the master serializer or the slave serializer;
[0015] The master serializer sends data packets to the master interface, and the slave serializer sends data packets to the slave interface.
[0016] A camera with multi-interface transmission, further preferably, the main interface is an optical transceiver;
[0017] The optical transceiver is in the uplink and can convert electrical signals into optical signals;
[0018] The optical transceiver can convert optical signals into electrical signals in the downlink.
[0019] A camera with multi-interface transmission, further preferably, the slave interface is an optical transmitter;
[0020] The optical transmitter can convert the uplink electrical signal into an optical signal.
[0021] A camera with multi-interface transmission, further preferably, the master serializer and the slave serializer are used to transmit the same video data.
[0022] A camera with multi-interface transmission, further preferably, the FPGA chip is configured as follows:
[0023] A first signal module, which receives a downlink signal transmitted by a main serializer and parses the downlink signal to obtain a synchronization signal and a control signal;
[0024] An image control module processes the synchronization signal and the control signal to form an image control signal;
[0025] The control interface receives the image control signal and transmits it to the image sensor through a control channel.
[0026] A camera with multi-interface transmission, further preferably, the main interface is used to receive a downlink signal sent by an external domain controller;
[0027] The downlink signal includes at least: a synchronization signal and a control signal.
[0028] A camera with multi-interface transmission, further preferably, the data conversion module is used to convert the video data of the MIPI protocol into single-channel video stream data.
[0029] Beneficial effects:
[0030] In the technical solution provided by the utility model, the utility model adopts FPGA chip to realize the adaptation of multiple transmission channels to multiple interfaces, so as to realize the simultaneous transmission of images from one camera to multiple domain controllers, thereby reducing the cost pressure caused by the need to install multiple cameras at the same location;
[0031] At the same time, the problem of a single camera needing to be forwarded to other domain controllers through domain controller copying is solved. Specifically, when one domain controller is transferred to other domain controllers, on the one hand, when the domain controller sends data packets, it needs to cache and package them again, and at the same time, it needs to wait for the current domain controller to be in the cache and package state before sending. Otherwise, if the domain controller used for forwarding has higher priority tasks to process, it must process the current higher priority tasks, which will cause a large delay problem and realize real-time transmission of image data. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] The following drawings are only used to illustrate and explain the present invention, and do not limit the scope of the present invention.
[0033] Figure 1 This is a schematic diagram of image data of multiple domains sharing the same camera in the prior art of the utility model.
[0034] Figure 2 It is a schematic diagram of a multi-interface camera according to an embodiment of the utility model.
[0035] Figure 3 This is a schematic diagram of a multi-interface camera transmitting image data to multiple domain controllers simultaneously through multiple interfaces according to an embodiment of the utility model.
[0036] Figure 4 This is a schematic diagram of the hardware structure of a multi-interface camera in one embodiment of the present utility model.
[0037] Figure 5 The figure is a schematic diagram of the hardware structure of another multi-interface camera in one embodiment of the present utility model. DETAILED DESCRIPTION
[0038] In order to have a clearer understanding of the technical features, purposes and effects of this invention, the specific implementation methods of the present invention are now described with reference to the accompanying drawings. The same reference numerals in each figure represent the same parts. In order to make the drawings concise, the parts related to the present invention are schematically shown in each figure, and do not represent the actual structure of the product. In addition, in order to make the drawings concise and easy to understand, in some figures, only one of the parts with the same structure or function is schematically drawn or marked.
[0039] In the present invention, "connection" may include direct connection, indirect connection, communication connection, and electrical connection, unless otherwise specified.
[0040] The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that when used in the specification, the terms "include" and / or "comprise" refer to the presence of stated features, values, steps, operations, elements and / or components, but do not exclude the presence or additional addition of one or more other features, values, steps, operations, elements, components and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the listed related items.
[0041] The utility model provides a camera with multi-interface transmission, which can be specifically applied to a variety of controllers with the same image requirements. The controller can be a domain controller, an ECU or a PC, a single-chip microcomputer or the like.
[0042] Specifically, a plurality of interfaces are provided on the camera, and each interface can send image data collected by the camera to the outside.
[0043] Specifically, the image data described in this application includes video data.
[0044] Specifically, see Figures 2 to 5 , the camera includes at least:
[0045] A lens, an image sensor, an FPGA chip, at least one master interface and at least one slave interface, wherein the image sensor is connected to the lens and the FPGA chip respectively, wherein at least one master serializer and at least one slave serializer are arranged on the FPGA chip;
[0046] The master serializer and the slave serializer are connected to the master interface and the slave interface respectively, and the master interface can receive downlink signals and be used to adjust parameters of the image sensor;
[0047] The uplink data is transmitted from the image sensor to the FPGA chip through the mipi interface.
[0048] Specifically, Figure 2 As shown, the camera is provided with a main interface M and two slave interfaces S, and the main interface M and the slave interface S are optical fiber interfaces;
[0049] Figure 3 In this embodiment, multiple domain controllers are connected simultaneously through multiple interfaces of the multi-interface camera. Specifically, the camera is set with a master interface and two slave interfaces. The master interface M is connected to the intelligent driving domain controller, one slave interface S is connected to the cockpit domain, and the other slave interface S is connected to the T-Box.
[0050] Specifically, Figure 4 As shown, in the FPGA chip settings:
[0051] A data conversion module is used to convert and process the video data transmitted by the image transmitter according to a preset protocol;
[0052] The data distribution module is used to copy the video data transmitted by the data conversion module and output it to different channels;
[0053] At least two data transmission channels are used to transmit the video data forwarded by the data distribution module.
[0054] Specifically, the video data of the MIPI protocol is converted into single-channel video stream data.
[0055] Specifically, the MIPI protocol uses multi-channel data transmission, usually four channels. For example, if 8 bytes of data are transmitted, they are divided into four channels and sent simultaneously. Each channel only needs to transmit 2 bytes, so the speed is faster. In existing cameras, data is not converted, but MIPI data is directly transmitted to the SoC in the network, which is parsed and processed by the SoC. In this embodiment, in order to reduce the hardware transmission cost and reduce the number of channels required for transmission in the FPGA, a data conversion module is designed to convert multi-channel data into single-channel data before subsequent replication and diversion processing, which can save hardware overhead and short-term cache space.
[0056] Specifically, in order to reduce latency, this embodiment designs a data diversion module, which uses hardware in the FPGA to automatically copy the same number of data as the number of data transmission channels according to the number of data channels that need to be transmitted, and then forwards it to the corresponding data transmission channels for processing at the same time.
[0057] For example, if a camera is designed with three interfaces and needs to transmit data to three domain controllers at the same time, the data needs to be copied three times at the same time. In this way, the three domain controllers can receive data at the same time at the sending end.
[0058] Specifically, it should be noted that the domain controller may also be replaced by a controller such as an ECU or a central computer platform.
[0059] Specifically, Figure 5 As shown, the data transmission channel is provided with a first cache module for caching the received image data;
[0060] Since the data collected by the image sensor is acquired frame by frame, but the amount of image data in each frame is large, the transmission using the Ethernet data protocol of this application cannot be transmitted in one data packet. In addition, each frame of data needs to be cached before being sent in packets. Therefore, this application designs a cache module in each channel to temporarily store the data forwarded by the data diversion module. Specifically, the data structure can be cached in a linked list or a first-in-first-out data format.
[0061] The data transmission channel is provided with an encapsulation module, which is used to obtain data from the first cache module according to a preset rule and then encapsulate the data packet using a preset communication protocol.
[0062] For example, the data packet may be encapsulated according to the automotive Ethernet data packet format and sent after being encapsulated by the IEEE AVTP protocol.
[0063] Specifically, the master serializer or the slave serializer is arranged inside the data transmission channel and connected to the encapsulation module, and the encapsulation module transmits the encapsulated data packet to the master serializer or the slave serializer;
[0064] The master serializer sends data packets to the master interface, and the slave serializer sends data packets to the slave interface.
[0065] Specifically, in order to transmit large amounts of data, the main interface is an optical transceiver;
[0066] The optical transceiver is in the upstream and can convert the electrical signal into an optical signal;
[0067] The optical transceiver is in the downstream and can convert the optical signal into an electrical signal.
[0068] Specifically, it should be noted that the master interface or the slave interface does not necessarily have to be an optical interface, and an electrical interface may also be used.
[0069] The advantage of using optical ports is that they can quickly transmit large amounts of data.
[0070] The slave interface is an optical transmitter, which can convert the uplink electrical signal into an optical signal.
[0071] Specifically, in this embodiment, in order to reduce the need for different domain controllers or ECUs to use image data of the same camera, the master serializer and the slave serializer are required to transmit the same video data.
[0072] Since the data required by the camera is also triggered by external devices, such as the required image resolution and transmission frame rate, if sent by the camera itself, it may not be compatible with external devices. In vehicle-mounted cameras, there are usually multiple domain controllers, but the intelligent driving domain controller has high real-time requirements. Therefore, if the output format, resolution and imaging parameters inside the camera image are directly adapted to the intelligent driving domain controller, the time for image processing by the intelligent driving domain controller will be saved, and real-time performance will be improved. In order to solve this problem, in this embodiment, a control channel loop is involved in the FPGA to receive the signal sent by the intelligent driving domain controller, and the image sensor is adjusted and adapted.
[0073] The specific plan is:
[0074] Design in FGPA chip:
[0075] A first signal module, which receives a downlink signal transmitted by a main serializer and parses the downlink signal to obtain a synchronization signal and a control signal;
[0076] The synchronization signal is sent as a data packet. During the conversion, the data packet involving the synchronization signal in the downlink signal needs to be converted into a pulse signal.
[0077] An image control module processes the synchronization signal and the control signal to form an image control signal;
[0078] The control interface receives the image control signal and transmits it to the image sensor through a control channel.
[0079] The control interface is mainly an I2C control chip, through which the synchronization signal and control signal are sent to the image sensor for image parameter configuration;
[0080] Image parameter configuration may be adjustments such as image format, resolution, frame rate, etc., and the specific modification method may be to modify the value of a register in the image sensor to achieve the modification.
[0081] Specifically, the main interface is used to receive a downlink signal sent by an external domain controller;
[0082] The downlink signal includes at least: a synchronization signal and a control signal.
[0083] For example, after receiving the downlink signal, the main interface sends the downlink signal to the main serializer, and the main serializer unpacks the signal and sends it to the first signal processing module for processing.
[0084] The above is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiment. It is clear to those skilled in the art that the form in the embodiment is not limited thereto, and the adjustable manner is not limited thereto. It is understood that other improvements and changes directly derived or associated with by those skilled in the art without departing from the basic concept of the present invention should be considered to be included in the protection scope of the present invention.
Claims
1. A camera with multi-interface transmission, characterized in that: At least: A lens, an image sensor, an FPGA chip, at least one master interface and at least one slave interface, wherein the image sensor is connected to the lens and the FPGA chip respectively, wherein at least one master serializer and at least one slave serializer are arranged on the FPGA chip; The master serializer and the slave serializer are connected to the master interface and the slave interface respectively, and the master interface can receive downlink signals and be used to adjust parameters of the image sensor; The uplink data is transmitted from the image sensor to the FPGA chip through the mipi interface.
2. The camera with multiple interfaces as claimed in claim 1, characterized in that: The FPGA chip settings: A data conversion module is used to convert and process the video data transmitted by the image sensor according to a preset protocol; The data distribution module is used to copy the video data transmitted by the data conversion module and output it to different channels; At least two data transmission channels are used to transmit the video data forwarded by the data splitting module.
3. The camera with multiple interfaces as claimed in claim 2, characterized in that: The data transmission channel at least includes: A first cache module, used for caching received image data; The encapsulation module is used to obtain data from the first cache module according to a preset rule and then encapsulate the data packet using a preset communication protocol.
4. The camera with multiple interfaces as claimed in claim 3, characterized in that: The master serializer or the slave serializer is arranged inside the data transmission channel and connected to the encapsulation module, and the encapsulation module transmits the encapsulated data packet to the master serializer or the slave serializer; The master serializer sends data packets to the master interface, and the slave serializer sends data packets to the slave interface.
5. The camera with multiple interfaces as claimed in claim 1, characterized in that: The main interface is an optical transceiver; The optical transceiver is in the uplink and can convert electrical signals into optical signals; The optical transceiver can convert optical signals into electrical signals in the downstream direction.
6. The camera with multiple interfaces as claimed in claim 1, characterized in that: The slave interface is an optical transmitter; The optical transmitter can convert the uplink electrical signal into an optical signal.
7. The camera with multiple interfaces as claimed in claim 1, characterized in that: The master serializer and the slave serializer are used to transmit the same video data.
8. The camera with multiple interfaces as claimed in claim 1, characterized in that: FPGA chip settings: A first signal module, which receives a downlink signal transmitted by a main serializer and parses the downlink signal to obtain a synchronization signal and a control signal; An image control module processes the synchronization signal and the control signal to form an image control signal; The control interface receives the image control signal and transmits it to the image sensor through a control channel.
9. The camera with multiple interfaces as claimed in claim 1, characterized in that: The main interface is used to receive a downlink signal sent by an external domain controller; The downlink signal includes at least: a synchronization signal and a control signal.
10. The camera with multiple interfaces as claimed in claim 2, characterized in that: The data conversion module is used to convert the video data of the MIPI protocol into single-channel video stream data.
