A camera system image data transmission method, camera system and computer program product

Abstract

The present application relates to camera system image data transmission method, camera system and computer program product, especially to the image data transmission method of single MIPI D-PHY connection multiple CMOS image sensor, the method of the present application is provided with 1 MIPI D-PHY chip and 1 IO of controlling camera image sensor CIS on camera SoC chip, and the MIPI D-PHY chip between SoC chip and n camera image sensor CIS1-CISn is directly connected through star bus;Or, 1 MIPI D-PHY chip and n IO of controlling camera image sensor CIS are provided on camera SoC chip, and the MIPI D-PHY chip between SoC chip and n camera image sensor CIS1-CISn is directly connected through star bus;The method can reduce the design and production cost of multi-view camera SoC due to only one MIPI D-PHY, so as to reduce the cost of camera mode implementation.

Description

Technical Field

[0001] This invention relates to image data transmission methods for camera systems, camera systems, and computer program products, and particularly to image data transmission methods for connecting multiple CMOS image sensors to a single MIPI D-PHY. Background Technology

[0002] MIPI stands for Mobile Industry Processor Interface. It's an open standard for mobile application processors initiated by the MIPI Alliance. MIPI can be divided into two main parts: the physical layer and the logic layer. Based on the physical layer, MIPI can be further divided into three types: D-PHY, C-PHY, and M-PHY. The logic layer of D-PHY is primarily designed for applications such as cameras (CSI) and displays (DSI). The "D" in D-PHY stands for the Roman numeral 500, and the initial version of D-PHY supported 500 Mbits / s. D-PHY uses differential signal transmission (not entirely differential; LP is single-ended transmission). Each lane consists of two signal lines, P and N. A clock lane is essential, and the number of data lanes can be selected based on the data transmission throughput, but at least one data lane is required.

[0003] Currently, multi-view cameras using multiple CIS (CMOS Image Sensors) are increasingly appearing on the market. In these camera implementation solutions, image data from multiple CISs needs to be transmitted to the SoC chip of a single camera, and solutions 1 and 2 are commonly used.

[0004] Option 1 is as follows Figure 1 As shown, the SoC chip has four MIPI D-PHYs, which receive data from CIS1 to CIS4 in parallel via four cables. The image data transmission control logic is as follows: After power-on, CIS1 to CIS4 continuously output image data according to the configuration. The SoC receives the image data from CIS1 to CIS4 through the four MIPI D-PHYs respectively and stores it in the corresponding memory buffer. Solution 1 requires multiple MIPI D-PHYs. Compared with a solution with only a single MIPI-PHY, the camera SoC chip designed and manufactured requires more die area. At the same time, multiple MIPI interfaces require more pins to be brought out during packaging. Both of these factors increase the cost of the camera SoC chip, thereby increasing the cost of implementing the camera solution.

[0005] Option 2 as follows Figure 2As shown, the SoC has only one MIPI D-PHY, which is connected to the MIPI Switch chip via one cable. The four CISs are also connected to the MIPI Switch via four cables. The SoC controls the MIPI Switch chip to switch between the four channels via the Switch Ctl signal, receiving data from CIS1 to CIS4 in a time-division multiplexing manner. The image data transmission control logic is as follows: After power-on, CIS1 to CIS4 continuously output image data according to the configuration; Step 1: The SoC first controls the MIPI switch to channel 1 via Switch Ctl. After the SoC receives a complete frame of image data from CIS1, it stores it in the corresponding memory buffer; Step 2: The SoC then controls the MIPI switch to channel 2 via Switch Ctl. After the SoC receives a complete frame of image data from CIS2, it stores it in the corresponding memory buffer; Step 3: The SoC then controls the MIPI switch to channel 3 via Switch Ctl. After the SoC receives a complete frame of image data from CIS3, it stores it in the corresponding memory buffer; Step 4: The SoC then controls the MIPI switch to channel 4 via Switch Ctl. After the SoC receives a complete frame of image data from CIS4, it stores it in the corresponding memory buffer; Steps 1-4 are executed in a loop, polling and receiving image data from CIS1 to CIS4, and storing it in the corresponding memory buffer. Option 2 requires the addition of a MIPI switch chip, which will increase the overall cost of implementing the camera solution. Summary of the Invention

[0006] To address the aforementioned technical problems, the present invention aims to provide an image data transmission method for connecting multiple CMOS image sensors with a single MIPI D-PHY. Since this method only requires one MIPI D-PHY, it can reduce the design and production costs of multi-camera SoCs, thereby lowering the cost of implementing camera-based solutions.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A method for image data transmission using a single MIPI D-PHY to connect multiple CMOS image sensors is disclosed. The camera SoC chip has at least one MIPI D-PHY chip and one I / O control for the camera image sensors (CIS). The SoC chip and the MIPI D-PHY chips of n camera image sensors (CIS1 to CISn) are directly connected via a star bus. The image data transmission control logic is as follows: Upon power-up, in the initial state, according to the configuration, the MIPI D-PHYs of CIS1 to CISn are all in a high-impedance state; Step 1: After receiving the Trigger, CIS1 starts outputting 1 frame of image data. The image data output by CIS1 contains VC mark information conforming to the MIPI Virtual Channel specification. After CIS1 finishes outputting, it restores the MIPI D-PHY to the high impedance state and, according to the pre-configured delay parameters, outputs the Trigger signal to the CIS2 Trigger in port through the Trigger out port after a delay. Step 2: After receiving the Trigger, CIS2 starts outputting 1 frame of image data. The image data output by CIS2 carries VC mark information conforming to the MIPI Virtual Channel specification. After CIS2 finishes outputting, it restores the MIPI D-PHY to the high impedance state and, according to the pre-configured delay parameters, outputs the Trigger signal to the CIS3 Trigger in port through the Trigger out port after a delay. And so on; Step n-1: After CISn-1 is completed and the Trigger is received, one frame of image data is output. The image data output by CISn-1 carries VC mark information conforming to the MIPI Virtual Channel specification. After CISn-1 is completed, the MIPID-PHY is restored to the high impedance state, and according to the pre-configured delay parameters, after the delay, the Trigger signal is output to the CISn Trigger in port through the Trigger out port. Step n: After CISn completes and receives the Trigger, it starts outputting 1 frame of image data. The image data output by CISn carries VC mark information conforming to the MIPI Virtual Channel specification. After CISn completes output, the MIPI D-PHY is restored to the high impedance state, and according to the pre-configured delay parameters, after the delay, the Trigger signal is output to the CIS1 Trigger in port through the Trigger out port. Step1 through Stepn are executed in a loop. The SoC’s MIPI D-PHY continuously receives image data and stores the image data from CIS1 to CISn in the corresponding memory buffers according to the VC tag information in the data.

[0008] Preferably, the VC tagging information allows multiple image data sources to transmit data through a single physical channel; the header of the data packet contains a Virtual Channel Identifier (VCID) to distinguish data from different virtual channels.

[0009] As a further preferred embodiment, the data packet consists of two parts: a short data packet and a long data packet, and both types of data packets contain a VCID field; Short data packet: includes data type, virtual channel ID, and 6-bit data field; Long data packets include data type, virtual channel ID, data length, a variable-length data payload, and a CRC checksum.

[0010] Preferably, in Step 1 to Step 2, when multiple CISs transmit data to the SoC through a physical channel, after each CIS finishes transmitting data, a delay parameter is set to delay before triggering the next CIS to transmit data. This delay amount is configured by the SoC to the CIS, which can control the amount of data transmitted per unit time to match the data processing capability of the SoC.

[0011] Preferably, n ≥ 2; more preferably 4-10.

[0012] As another implementation method, the present invention also provides another technical solution: A method for image data transmission using a single MIPI D-PHY to connect multiple CMOS image sensors is disclosed. The camera SoC chip has at least one MIPI D-PHY chip and n I / O pins for controlling the camera image sensors (CIS). The SoC chip and the MIPI D-PHY chips of the n camera image sensors (CIS1 to CISn) are directly connected via a star bus. The image data transmission control logic is as follows: Upon power-up, in the initial state, according to the configuration, the MIPI D-PHYs of CIS1 to CISn are all in a high-impedance state; Step 1: The SoC outputs a Trigger signal to the CIS1 Trigger in port through the Trigger out-1 port; after receiving the Trigger, the CIS1 starts outputting 1 frame of image data. After the CIS1 finishes outputting, it restores the MIPI D-PHY to the high impedance state. Step 2: The SoC outputs a Trigger signal to the CIS2 Trigger in port through the Trigger out-2 port; after receiving the Trigger, the CIS2 starts outputting 1 frame of image data. After the CIS2 finishes outputting, it restores the MIPI D-PHY to the high impedance state. And so on; Step n-1: The SoC outputs a Trigger signal from the Trigger out-n-1 port to the CISn-1 Trigger in port; after the CISn-1 outputs the signal to the Trigger, it starts outputting one frame of image data. After the CISn-1 output is complete, the MIPI D-PHY is restored to the high impedance state. Step n: The SoC outputs a Trigger signal to the CISn Trigger in port through the Trigger out-n port; after receiving the Trigger, CISn starts outputting 1 frame of image data. After CISn finishes outputting, it restores the MIPI D-PHY to the high impedance state. Step1-n is executed in a loop. The SoC's MIPI D-PHY continuously receives image data and stores the image data from CIS1 to CISn in the corresponding memory buffers according to the Trigger out status information.

[0013] Preferably, n≥2; more preferably 4-10.

[0014] Furthermore, the present invention also provides a surveillance camera system, which includes a camera SoC chip and n camera image sensors CIS1 to CISn; the camera SoC chip is provided with one MIPI D-PHY chip and one I / O for controlling the camera image sensors CIS, and the SoC chip and the n MIPI D-PHY chips of the camera image sensors CIS1 to CISn are directly connected via a star bus; or, the camera SoC chip is provided with one MIPI D-PHY chip and n I / O for controlling the camera image sensors CIS, and the SoC chip and the n MIPI D-PHY chips of the camera image sensors CIS1 to CISn are directly connected via a star bus; the image data transmission control logic adopts the above method.

[0015] Furthermore, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the method.

[0016] Furthermore, the present invention also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the method.

[0017] Because the present invention adopts the above-mentioned technical solution, and only requires one MIPI D-PHY, the design and production costs of multi-camera SoC can be reduced, thereby reducing the cost of implementing the camera method. Attached Figure Description

[0018] Figure 1 This is a product structure diagram for the existing Scheme 1.

[0019] Figure 2 This is a product structure diagram for the existing Scheme 2.

[0020] Figure 3 This is a product structure block diagram of Embodiment 1 of the present invention.

[0021] Figure 4 This is a product structure block diagram of Embodiment 2 of the present invention. Detailed Implementation

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0023] Example 1 like Figure 3 The image data transmission method shown is a method for connecting multiple CMOS image sensors via a single MIPI D-PHY. The camera SoC chip has one MIPI D-PHY chip and one I / O control for the camera image sensors (CIS). The SoC chip and the MIPI D-PHY chips of the four camera image sensors (CIS1 to CIS4) are directly connected via a star bus. The image data transmission control logic is as follows: Upon power-up, in the initial state, according to the configuration, the MIPI D-PHYs of CIS1 to CIS4 are all in a high-impedance state; The SoC outputs a trigger signal to the CIS1 Trigger in port via the Trigger out port; Step 1: After receiving the Trigger, CIS1 starts outputting 1 frame of image data. The image data output by CIS1 contains VC mark information conforming to the MIPI Virtual Channel specification. After CIS1 finishes outputting, it restores the MIPI D-PHY to the high impedance state and, according to the pre-configured delay parameters, outputs the Trigger signal to the CIS2 Trigger in port through the Trigger out port after a delay. Step 2: After receiving the Trigger, CIS2 starts outputting 1 frame of image data. The image data output by CIS2 carries VC mark information conforming to the MIPI Virtual Channel specification. After CIS2 finishes outputting, it restores the MIPI D-PHY to the high impedance state and, according to the pre-configured delay parameters, outputs the Trigger signal to the CIS3 Trigger in port through the Trigger out port after a delay. Step 3: After CIS3 completes and receives the Trigger, it starts outputting 1 frame of image data. The image data output by CIS3 carries VC mark information conforming to the MIPI Virtual Channel specification. After CIS3 output is complete, the MIPI D-PHY is restored to the high impedance state, and according to the pre-configured delay parameters, after the delay, the Trigger signal is output to the CIS4 Trigger in port through the Trigger out port. Step 4: After CIS4 completes and receives the Trigger, it starts outputting 1 frame of image data. The image data output by CIS3 carries VC mark information conforming to the MIPI Virtual Channel specification. After CIS3 completes output, the MIPI D-PHY is restored to the high impedance state, and according to the pre-configured delay parameters, after the delay, the Trigger signal is output to the CIS1 Trigger in port through the Trigger out port. Step-4 is executed repeatedly. The SoC's MIPI D-PHY continuously receives image data and stores the image data of CIS1 to CIS4 in the corresponding memory buffers according to the VC tag information in the data.

[0024] In this embodiment, the VC tag information in MIPI CSI refers to a Virtual Channel. Virtual channels were introduced to allow multiple image sensors to share a single physical channel. Each virtual channel has an associated Virtual Channel Identifier (VCID). This allows multiple image data sources (e.g., multiple sensors) to transmit data through a single physical channel (PHY). The primary purpose of virtual channels is to reuse the same physical channel to support multiple sensors or other data sources.

[0025] In the MIPI CSI-2 protocol, the header of a data packet contains a field called the Virtual Channel Identifier (VCID), which is used to distinguish data from different virtual channels. The VCID is a 2-bit field, so there can be 4 different virtual channels (0-3).

[0026] Packet Structure: MIPI CSI-2 packets consist of two parts – a short packet and a long packet. Both types of packets contain a VCID field.

[0027] Short data packet: includes data type (5 bits), virtual channel ID (2 bits), and 6-bit data field.

[0028] Long data packets include data type (5 bits), virtual channel ID (2 bits), data length (16 bits), then a variable-length data payload, and finally a CRC checksum.

[0029] Furthermore, in this embodiment, when multiple CISs transmit data to the SoC through a physical channel in Steps 1 to 4, after each CIS finishes transmitting data, there should be a slight delay before triggering the next CIS to transmit data. This delay can be configured by the SoC to control the amount of data transmitted per unit time to match the data processing capability of the SoC.

[0030] Example 2 like Figure 4 The image data transmission method shown is a method for connecting multiple CMOS image sensors via a single MIPI D-PHY. The camera SoC chip has one MIPI D-PHY chip and four I / O pins controlling the camera image sensors (CIS). The SoC chip and the MIPI D-PHY chips of the four camera image sensors (CIS1 to CIS4) are directly connected via a star bus. The image data transmission control logic is as follows: Upon power-up, in the initial state, according to the configuration, the MIPI D-PHYs of CIS1 to CIS4 are all in a high-impedance state; Step 1: The SoC outputs a Trigger signal to the CIS1 Trigger in port through the Trigger out-1 port; after receiving the Trigger, the CIS1 starts outputting 1 frame of image data. After the CIS1 finishes outputting, it restores the MIPI D-PHY to the high impedance state. Step 2: The SoC outputs a Trigger signal to the CIS2 Trigger in port through the Trigger out-2 port; after receiving the Trigger, the CIS2 starts outputting 1 frame of image data. After the CIS2 finishes outputting, it restores the MIPI D-PHY to the high impedance state. Step 3: The SoC outputs a Trigger signal to the CIS3 Trigger in port via the Trigger out-3 port; after receiving the Trigger, the CIS3 starts outputting one frame of image data. After the CIS3 finishes outputting, it restores the MIPI D-PHY to the high impedance state. Step 4: The SoC outputs a Trigger signal to the CIS4 Trigger in port through the Trigger out-4 port; after receiving the Trigger, the CIS4 starts outputting 1 frame of image data. After the CIS4 finishes outputting, it restores the MIPI D-PHY to the high impedance state. Steps 1-4 are executed in a loop. The SoC's MIPI D-PHY continuously receives image data and stores the image data of CIS1 to CIS4 in the corresponding memory buffers according to the Trigger out status information.

[0031] The resource requirements of Embodiments 1 and 2 of the present invention are compared with those of the existing solutions 1 and 2 as follows: plan CIS D-PHY to SoC D-PHY MIPI Switch SoC I / O usage CIS IO usage VC support MIPI Clock Option 1 1 to 1 unnecessary 0 0 unnecessary f Option 2 n to 1, where n is determined by the MIPI switch. need n is determined by the MIPI Switch. 0 unnecessary Approximately n*f Example 1 n to 1,n<4 unnecessary 1 2 need Approximately n*f Example 2 n to 1,n<4 unnecessary 4 1 unnecessary Approximately n*f Our solutions are Embodiment 1 and Embodiment 2. Compared to Embodiment 1, these two solutions reduce the design and manufacturing costs of the multi-camera SoC by requiring only one MIPI PHY, thereby lowering the cost of implementing the camera solution. Compared to Embodiment 2, the elimination of the need for a MIPI switch chip also reduces the cost of implementing the camera solution.

[0032] Compared to Example 2, Example 1 requires fewer I / O pins on the camera SoC chip. Although an additional trigger-out I / O pin is needed on the CIS side, the increased cost due to the common CSP packaging of CIS is very low or even negligible. Therefore, Example 1 is a superior solution compared to Example 2.

[0033] The foregoing description of embodiments of the present invention, through which those skilled in the art are able to implement or use the present invention, will be readily apparent to those skilled in the art. Various modifications to these embodiments will be readily apparent to those skilled in the art. The general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novelty disclosed herein.

Claims

1. A method for transmitting image data from a single MIPI D-PHY connected to multiple CMOS image sensors, characterized in that, The camera SoC chip has at least one MIPI D-PHY chip and one I / O for controlling the camera image sensor CIS. The SoC chip and the MIPI D-PHY chips of n camera image sensors CIS1 to CISn are directly connected via a star bus. The image data transmission control logic is as follows: Upon power-up, in the initial state, according to the configuration, the MIPI D-PHYs of CIS1 to CISn are all in a high-impedance state; Step 1: After receiving the Trigger, CIS1 starts outputting 1 frame of image data. The image data output by CIS1 contains VC mark information conforming to the MIPI Virtual Channel specification. After CIS1 finishes outputting, it restores the MIPI D-PHY to the high impedance state and, according to the pre-configured delay parameters, outputs the Trigger signal to the CIS2 Trigger in port through the Trigger out port after a delay. Step 2: After receiving the Trigger, CIS2 starts outputting 1 frame of image data. The image data output by CIS2 carries VC mark information conforming to the MIPI Virtual Channel specification. After CIS2 finishes outputting, it restores the MIPI D-PHY to the high impedance state and, according to the pre-configured delay parameters, outputs the Trigger signal to the CIS3 Trigger in port through the Trigger out port after a delay. And so on; Step n-1: After CISn-1 is completed and the Trigger is received, one frame of image data is output. The image data output by CISn-1 carries VC mark information conforming to the MIPI Virtual Channel specification. After CISn-1 is completed, the MIPI D-PHY is restored to the high impedance state, and according to the pre-configured delay parameters, after the delay, the Trigger signal is output to the CISn Trigger in port through the Trigger out port. Step n: After CISn completes and receives the Trigger, it starts outputting 1 frame of image data. The image data output by CISn carries VC mark information conforming to the MIPI Virtual Channel specification. After CISn completes output, the MIPI D-PHY is restored to the high impedance state, and according to the pre-configured delay parameters, after the delay, the Trigger signal is output to the CIS1 Trigger in port through the Trigger out port. Step1 through Stepn are executed in a loop. The SoC’s MIPI D-PHY continuously receives image data and stores the image data from CIS1 to CISn in the corresponding memory buffers according to the VC tag information in the data.

2. The image data transmission method according to claim 1, characterized in that, VC tagging information allows multiple image data sources to transmit data through a single physical channel; the header of the data packet contains a Virtual Channel Identifier (VCID) used to distinguish data from different virtual channels.

3. The image data transmission method according to claim 2, characterized in that, The data packet consists of two parts: a short data packet and a long data packet. Both types of data packets contain a VCID field. Short data packet: includes data type, virtual channel ID, and 6-bit data field; Long data packets include data type, virtual channel ID, data length, a variable-length data payload, and a CRC checksum.

4. The image data transmission method according to claim 1, characterized in that, In Steps 1 through 2, when multiple CISs transmit data to the SoC through a single physical channel, after each CIS finishes transmitting data, a delay parameter is set to delay the transmission before triggering the next CIS to transmit data. This delay is configured by the SoC to control the amount of data transmitted per unit time, so as to match the data processing capabilities of the SoC.

5. The image data transmission method according to claim 1, characterized in that, n≥2。 6. The image data transmission method according to claim 1, characterized in that, n is 4-10.

7. A method for transmitting image data from a single MIPI D-PHY connected to multiple CMOS image sensors, characterized in that, The camera SoC chip has at least one MIPI D-PHY chip and n I / O pins for controlling the camera image sensor CIS. The SoC chip and the n MIPI D-PHY chips of the camera image sensors CIS1 to CISn are directly connected via a star bus. The image data transmission control logic is as follows: Upon power-up, in the initial state, according to the configuration, the MIPI D-PHYs of CIS1 to CISn are all in a high-impedance state; Step 1: The SoC outputs a Trigger signal to the CIS1 Trigger in port through the Trigger out-1 port; after receiving the Trigger, the CIS1 starts outputting 1 frame of image data. After the CIS1 finishes outputting, it restores the MIPI D-PHY to the high impedance state. Step 2: The SoC outputs a Trigger signal to the CIS2 Trigger in port through the Trigger out-2 port; after receiving the Trigger, the CIS2 starts outputting 1 frame of image data. After the CIS2 finishes outputting, it restores the MIPI D-PHY to the high impedance state. And so on; Step n-1: The SoC outputs a Trigger signal from the Trigger out-n-1 port to the CISn-1 Trigger in port; after the CISn-1 outputs the signal to the Trigger, it starts outputting one frame of image data. After the CISn-1 output is complete, the MIPI D-PHY is restored to the high impedance state. Step n: The SoC outputs a Trigger signal to the CISn Trigger in port through the Trigger out-n port; after receiving the Trigger, CISn starts outputting 1 frame of image data. After CISn finishes outputting, it restores the MIPI D-PHY to the high impedance state. Step1-n is executed in a loop. The SoC's MIPI D-PHY continuously receives image data and stores the image data from CIS1 to CISn in the corresponding memory buffers according to the Trigger out status information.

8. The image data transmission method according to claim 7, characterized in that, n≥2。 9. The image data transmission method according to claim 7, characterized in that, n is 4-10.

10. A surveillance camera system, characterized in that, The system includes a camera SoC chip and n camera image sensors CIS1 to CISn; the camera SoC chip has one MIPI D-PHY chip and one I / O for controlling the camera image sensors CIS, and the SoC chip and the n MIPI D-PHY chips of the camera image sensors CIS1 to CISn are directly connected via a star bus; the image data transmission control logic is as described in any one of claims 1-4; or, the camera SoC chip has one MIPI D-PHY chip and n I / Os for controlling the camera image sensors CIS, and the SoC chip and the n MIPI D-PHY chips of the camera image sensors CIS1 to CISn are directly connected via a star bus; the image data transmission control logic is as described in any one of claims 7-9.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1-9.

12. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the method described in any one of claims 1-9.

Images