An image acquisition and processing chip

Abstract

The present invention discloses an image acquisition and processing chip, which includes an image sensor pixel array, a digital control circuit, a peripheral analog circuit, and a data output interface. The effective pixel array size of the image sensor pixel array is 30720x34048. The image sensor pixel array includes eight pixel sub-arrays, and the eight pixel sub-arrays respectively correspond to eight MIPI interfaces. The image acquisition and processing chip of the present invention divides the pixel array of the image sensor into eight pixel sub-arrays, and is a high-quality and high-performance image acquisition and processing chip that can achieve 1 billion pixels and output high-definition images of 48 Gbps through the mipi interface.

Description

Technical Field

[0001] The present invention relates to an image acquisition and processing chip. Background Art

[0002] VPS4114A is a high-quality and high-performance image acquisition and processing chip. The technologies behind this chip involve the following aspects:

[0003] 1. High-pixel sensor technology: VPS4114A adopts advanced 1 billion pixel sensor technology, which can capture extremely rich image details and output them with high bandwidth through the MIPI interface.

[0004] 2. MIPI interface technology: The MIPI interface is a high-speed serial interface technology specifically for mobile devices and embedded systems. VPS4114A adopts the MIPI interface technology, which can output high-definition images in a high-speed and stable manner.

[0005] 3. Image processing technology: VPS4114A integrates a variety of advanced image processing technologies internally, including noise filtering, color correction, sharpening, white balance, intelligent fill light, etc. These technologies can help users process and analyze image data quickly and accurately.

[0006] 4. High-speed data transmission technology: The output rate of VPS4114A is as high as 48 Gbps, and high-speed and stable data transmission technology is required to ensure that data can be accurately transmitted to the target device in a short time.

[0007] Combining these technologies, VPS4114A can achieve high-quality and high-performance image acquisition and processing, and is widely used in fields such as high-end cameras, industrial vision, and medical imaging.

[0008] However, the existing VPS4114A chip can only be imported, with a high price and a risk of information leakage.

[0009] Therefore, a new image acquisition and processing chip is needed to solve the above problems. Summary of the Invention

[0010] The object of the present invention is to provide an image acquisition and processing chip to overcome the problems existing in the prior art.

[0011] The image acquisition and processing chip of the present invention includes an image sensor pixel array, a digital control circuit, a peripheral analog circuit, and a data output interface. On both sides of the image sensor pixel array, there are 240 columns of dummy pixels. The effective pixel array size of the image sensor pixel array is 30720x34048. The image sensor pixel array includes pixel sub-arrays Subarray_0, Subarray_1, Subarray_2, Subarray_3, Subarray_4, Subarray_5, Subarray_6, and Subarray_7. Each of the pixel sub-arrays Subarray_0, Subarray_1, Subarray_2, Subarray_3, Subarray_4, Subarray_5, Subarray_6, and Subarray_7 includes 7800x17024 pixels. The data output interface includes MIPI interfaces MIPI0, MIPI1, MIPI2, MIPI3, MIPI4, MIPI5, MIPI6, and MIPI7. The MIPI interface MIPI0 corresponds to the pixel sub-array Subarray_0, the MIPI interface MIPI1 corresponds to the pixel sub-array Subarray_1, the MIPI interface MIPI2 corresponds to the pixel sub-array Subarray_2, the MIPI interface MIPI3 corresponds to the pixel sub-array Subarray_3, the MIPI interface MIPI4 corresponds to the pixel sub-array Subarray_4, the MIPI interface MIPI5 corresponds to the pixel sub-array Subarray_5, the MIPI interface MIPI6 corresponds to the pixel sub-array Subarray_6, and the MIPI interface MIPI7 corresponds to the pixel sub-array Subarray_7. The working modes of the chip include a full-sampling mode, a decimated-sampling mode, and a windowing mode.

[0012] Furthermore, the digital control circuit includes a timing control module, a data reading module, a data transmission module, a data processing module, a data storage module, and a communication interface.

[0013] Furthermore, the analog circuit includes a pixel measurement and data generation module, a switch physical implementation module, a temperature detection module, a voltage conversion module, and a clock generation module.

[0014] Further, the pixel sub-array Subarray_2 and the pixel sub-array Subarray_0 are vertically mirror-symmetrical, the pixel sub-array Subarray_3 and the pixel sub-array Subarray_1 are vertically mirror-symmetrical, the pixel sub-array Subarray_6 and the pixel sub-array Subarray_4 are vertically mirror-symmetrical, and the pixel sub-array Subarray_7 and the pixel sub-array Subarray_5 are vertically mirror-symmetrical.

[0015] Further, when the working mode is the full sampling mode, the pixel reading order of each row of the pixel sub-arrays Subarray_1, Subarray_4, and Subarray_5 is the same as that of the pixel sub-array Subarray_0, and the pixel reading order of each row of the pixel sub-arrays Subarray_3, Subarray_6, and Subarray_7 is the same as that of the pixel sub-array Subarray_2.

[0016] Further, when the working mode is the full sampling mode, the pixel sub-arrays Subarray_0, Subarray_1, Subarray_4, and Subarray_5 all read pixels from row 0 to row 17023, and the pixel sub-arrays Subarray_2, Subarray_3, Subarray_6, and Subarray_7 all read pixels from row 34047 to row 17024.

[0017] Further, when the working mode is the decimated sampling mode, decimated sampling is performed on the rows and columns of the entire image sensor pixel array, and only the decimated pixels are output.

[0018] Further, when the working mode is the windowing mode, 1 to 16 windows are randomly opened in the entire image sensor pixel array, and the size of each window is 512x512 pixels.

[0019] Further, the data output by each pixel is 10 bits.

[0020] Further, the data bit widths of the MIPI interfaces MIPI0, MIPI1, MIPI2, and MIPI3 are 64 bits, the bit width of the internal data word of the chip is 256 bits, and the internal data word of the chip is converted into 4 64-bit serials and sent through the MIPI interface MIPI0, MIPI interface MIPI1, MIPI interface MIPI2, or MIPI interface MIPI3.

[0021] Beneficial effects: The image acquisition and processing chip of the present invention divides the pixel array of the image sensor into eight pixel sub-arrays, and can achieve a high-quality and high-performance image acquisition and processing chip with 1 billion pixels and output high-definition images at 48 Gbps through the mipi interface. Description of the Drawings

[0022] Figure 1 It is the chip functional block diagram of the image acquisition and processing chip;

[0023] Figure 2 It is the structural diagram of the image sensor pixel array;

[0024] Figure 3 It is the schematic diagram of the effective pixel array and the dummy pixel column;

[0025] Figure 4 It is the schematic diagram of data word parallel-to-serial conversion;

[0026] Figure 5 It is the schematic diagram of the data word containing 25 pixel data;

[0027] Figure 6 It is the schematic diagram of the data word containing less than 25 pixel data;

[0028] Figure 7 It is the schematic diagram of the frame structure - full sampling mode;

[0029] Figure 8 It is the schematic diagram of the frame structure - 2x decimation sampling;

[0030] Figure 9 It is the schematic diagram of windowing;

[0031] Figure 10 Schematic diagram of windowing data distribution;

[0032] Figure 11 It is the mipi0 output data frame format in windowing mode;

[0033] Figure 12 It is the mipi1 output data frame format in windowing mode

[0034] Figure 13 It is the mipi2 output data frame format in windowing mode;

[0035] Figure 14 It is the mipi3 output data frame format in windowing mode;

[0036] Figure 15 It is the mipi4 output data frame format in windowing mode;

[0037] Figure 16 It is the mipi5 output data frame format in windowing mode;

[0038] Figure 17 It is the mipi6 output data frame format in the windowing mode;

[0039] Figure 18 It is the mipi7 output data frame format in the windowing mode. Detailed implementation mode

[0040] Now, the present invention will be further described in detail with reference to the accompanying drawings.

[0041] Embodiment 1:

[0042] Please refer to Figure 1 As shown, the image acquisition and processing chip of the present invention includes an image sensor pixel array, a peripheral analog circuit, a digital control circuit, and a data output interface.

[0043] The digital control circuit includes a timing control module, a data reading module, a data transmission module, a data processing module, a data storage module, and a communication interface.

[0044] The analog circuit includes a pixel measurement and data generation module, a switch physical implementation module, a temperature detection module, a voltage conversion module, and a clock generation module.

[0045] Please refer to Figure 1 As shown, the chip functional block diagram: The chip mainly includes three parts: an image sensor pixel array, a peripheral analog circuit, a digital control circuit, and a data output interface. The digital circuit is responsible for timing control, data reading, data transmission, data processing, data storage, interface communication, etc.; the analog circuit is responsible for pixel measurement and data generation, switch physical implementation, temperature detection, voltage conversion, clock generation, etc.

[0046] Image sensor pixel array structure:

[0047] Please refer to Figure 2 As shown, the pixel array consists of 34048 rows, each row has 31200 pixels, and the size of the entire array is 31200x34048. The entire pixel array is divided into eight sub-arrays (Subarray), and the size of each sub-array is 7800x17024. In the figure, p_xx represents the xxth pixel of each row. As follows Figure 2 As shown.

[0048] Please refer to Figure 3 As shown, the data output by each pixel is 10bit. The data of the entire pixel array is output through 8 MIPI interfaces, and each MIPI interface corresponds to a pixel sub-array. For example, MIPI0 corresponds to Subarray_0, MIPI1 corresponds to Subarray_1, and so on. There are 240 columns of dummy pixels on both the left and right sides of the pixel array, such as Figure 2-3As shown. They are not real image data and need to be discarded from the final picture. So the final effective pixel array size is 30720x34048.

[0049] Please participate Figure 4 As shown, since the data bit width of the MIPI interface inside the chip is 64 bits, while the bit width of the internal data word (WORD) of the chip is 256 bits. Therefore, the data word needs to be converted from 256 bits into 4 64-bit serials to be sent through MIPI. Among them, the highest 64 bits are sent first, and the lowest 64 bits are sent last.

[0050] Please refer to Figure 5 - Figure 6 As shown, each frame is composed of several data words (WORDs), and each data word is 256 bits. Each data word can store up to 25 pixels at most, each pixel is 10 bits, a total of 250 bits, and they are stored in the lower 250 bits of the data word. If there are less than 25 pixels, then each pixel is arranged in sequence starting from the lowest bit of the data word, and the high bits are filled with 0.

[0051] The high 6 bits of each data word are used to store some control information, and the control word is defined as follows:

[0052] uBit[252:250]: Frame ID (Frame_ID). When this data word is the first data word of this frame, these three bits of data are the Frame_ID of this frame. Otherwise, it is all 0. The frame ID starts from 0, and increments by 1 for each additional frame, and loops continuously.

[0053] uBit[254:253]: Reserved bits, all 0.

[0054] uBit

[255] : Data full flag (Full_flag), indicating whether this data word contains 25 pixel data.

[0055] 0: This data word contains 25 pixel data ( Figure 5 ).

[0056] 1: This data word contains less than 25 pixel data ( Figure 6 ).

[0057] If Full_flag is 1, then Bit [249:245] of this data word represents how many pixel values are in this data word.

[0058] In different working modes, the data volume of each image varies due to different sampling and windowing, so the size of the data frames is also different. In the full-sampling mode and the decimated-sampling mode, 8 sub-arrays work simultaneously and output 8 data frames, which are output to the outside of the chip through 8 MIPI interfaces at the same time. The size and format of each data frame are the same, and each image is composed of these 8 frames. In the windowing mode, since the position of the window is random, the data volume corresponding to each sub-array is not fixed, resulting in different sizes of the frames of each MIPI.

[0059] Full-sampling mode frame structure

[0060] When the chip works in the full-sampling mode, the chip samples all pixels of the entire array and outputs all pixels. Taking the first sub-array (Subarray_0) as an example, the format of the data frame is as Figure 7 shown.

[0061] The frame length is 5,311,488 data words (WORDs), which contains 17,024 rows of pixels. Each row of pixels is composed of 312 data words (WORDs) and contains 7,800 pixels. Since the entire array is symmetrically mirrored up and down, the frame output by the third sub-array (Subarray_2) is similar to the above figure, but the 34,047th row is output first, then the 34,046th row, until the 17,024th row. The pixel data output order of Subarray_1, Subarray_4, and Subarray_5 is the same as that of Subarray_0, and the pixel data output order of Subarray_2, Subarray_6, and Subarray_7 is the same as that of Subarray_2.

[0062] Decimated-sampling mode frame structure

[0063] In the decimated-sampling mode, the chip performs decimated sampling on the rows and columns of the entire array and only outputs the decimated-sampled pixels. The format of the data frame is similar to the full-sampling working mode, except that the size of the image data part is different. Taking 2x decimated sampling as an example, 2x decimated sampling performs 2x decimated sampling on the rows and columns respectively, and the length of the image data is one-fourth of that in the full-sampling working mode. The frame structure of the first sub-array is as follows Figure 8 shown. Image data: The length is 1,327,872 data words (WORDs), which contains 8,512 rows of pixels. Each row of pixels is composed of 156 data words (WORDs) and contains 3,900 pixels. The sizes of the frames in different decimated-sampling modes are as Figure 8 shown.

[0064] Windowing mode frame structure

[0065] In the windowing mode, the host computer sends instructions to send the windowing positions and the number of windows to the chip. The chip supports randomly opening 1 to 16 windows in the entire pixel array. The size of each window is 512x512 pixels, and the windows can overlap. When taking a photo, the coordinates are updated before each shot; when shooting a video, the number of windows opened between frames cannot change, but the windowing positions can be updated. The chip will use the new windowing coordinates at the start of the next frame.

[0066] As Figure 9 shown, after obtaining the windowing coordinates for each frame, the blue part is the effective pixel area of the target window, and the yellow part is the invalid pixel area. MIPI will output the blue effective data and the yellow invalid data together.

[0067] The chip data is divided into 8 parts. As Figure 10 shown, among them, the left side of the upper half is output to MIPI0 and MIPI1 respectively, and the right half is output to MIPI4 and MIPI5; the left half of the lower half is output to MIPI2 and MIPI3, and the right half is output to MIPI6 and MIPI7. The reading order of the upper half is from top to bottom, and the reading order of the lower half is from bottom to top. After obtaining new coordinates at the start of each frame, the number of rows read by the four MIPIs in the upper half and the lower half is the same respectively, but the amount of data output by each MIPI is determined by the windowing position: ① If there is Figure 9 blue or yellow data in the pixel area corresponding to the MIPI, then in the reading order, every 25 pixels in each row form a group and are output in sequence from left to right. If there are invalid pixels in the last group, the highest bit is set to 1, and bit[249:245] outputs the number of effective pixels. Each row is output in the same way until the last row is output; ② If there is no blue or yellow data in the pixel area corresponding to the MIPI, then the MIPI is in the idle state and does not output data.

[0068] Taking four windows as an example, as Figure 10 shown, the starting coordinates of window 1 are (100, 7299), the starting coordinates of window 2 are (17124, 15089), the starting coordinates of window 3 are (300, 22889), and the starting coordinates of window 4 are (611, 22914). Window 3 and window 4 overlap. The value range of the windowing coordinates is: X coordinate: [0:33536], Y coordinate: [240:30448]. In the windowing mode, the chip output is Figure 10All the blue and yellow pixel data, where the blue part is the valid data of the target window and the yellow part is the invalid data outside the target window. The pixel array is divided into 8 sub-arrays. The upper half is output to mipi0, mipi1, mipi4, and mipi5 respectively, and the lower half is output to mipi2, mipi3, mipi6, and mipi7 respectively. The upper and lower parts are symmetrically output, as Figure 10 shown by the arrows in

[0069] Among them,

[0070] The output data frame structure of mipi0 is as Figure 11 shown

[0071] The output data frame structure of mipi1 is as Figure 12 shown

[0072] The output data frame structure of mipi2 is as Figure 13 shown

[0073] The output data frame structure of mipi3 is as Figure 14 shown

[0074] The output data frame structure of mipi4 is as Figure 15 shown

[0075] The output data frame structure of mipi5 is as Figure 16 shown

[0076] The output data frame structure of mipi6 is as Figure 17 shown

[0077] The output data frame structure of mipi7 is as Figure 18 shown

[0078] The MIPI (Mobile Industry Processor Interface) Alliance is an alliance founded in 2003 by companies such as ARM, Nokia, ST, and TI. MIPI (Mobile Industry Processor Interface) is an open standard initiated by the MIPI Alliance for mobile application processors. The purpose is to standardize the interfaces inside the mobile phone, such as the camera, display interface, radio frequency / baseband interface, etc., thereby reducing the complexity of mobile phone design and increasing design flexibility. MIPI is not a single interface or protocol, but rather a set of protocols and standards to meet the unique requirements of various subsystems. Common protocols include the camera interface CSI, the display interface DSI, the radio frequency interface DigRF, the microphone / speaker interface SLIMbus, etc. Currently, relatively mature interface applications are DSI (Display Serial Interface) and CSI (Camera Serial Interface). DSI defines a high-speed serial interface between the processor and the display module; CSI defines a high-speed serial interface between the processor and the camera module.

[0079] The image acquisition and processing chip of the present invention divides the pixel array of the image sensor into eight pixel sub-arrays, and is a high-quality and high-performance image acquisition and processing chip with 1 billion pixels and outputs high-definition images at 48 Gbps through the mipi interface. The chip has an automatic power-on calibration function. After calibration, the shutter modes include global exposure and normal rolling exposure. The sampling modes include full sampling, decimated sampling, and windowing mode. The chip will adopt different processing methods for the acquired data according to different working modes.

Claims

1. An image acquisition and processing chip, comprising an image sensor pixel array, a digital control circuit, a peripheral analog circuit, and a data output interface, characterized in that On both sides of the image sensor pixel array, there are 240 columns of dummy pixels. The effective pixel array size of the image sensor pixel array is 30720x34048. The image sensor pixel array includes pixel sub-arrays Subarray_0, Subarray_1, Subarray_2, Subarray_3, Subarray_4, Subarray_5, Subarray_6, and Subarray_7. Each of the pixel sub-arrays Subarray_0, Subarray_1, Subarray_2, Subarray_3, Subarray_4, Subarray_5, Subarray_6, and Subarray_7 includes 7800x17024 pixels. The data output interface includes MIPI interfaces MIPI0, MIPI1, MIPI2, MIPI3, MIPI4, MIPI5, MIPI6, and MIPI7. The MIPI interface MIPI0 corresponds to the pixel sub-array Subarray_0, the MIPI interface MIPI1 corresponds to the pixel sub-array Subarray_1, the MIPI interface MIPI2 corresponds to the pixel sub-array Subarray_2, the MIPI interface MIPI3 corresponds to the pixel sub-array Subarray_3, the MIPI interface MIPI4 corresponds to the pixel sub-array Subarray_4, the MIPI interface MIPI5 corresponds to the pixel sub-array Subarray_5, the MIPI interface MIPI6 corresponds to the pixel sub-array Subarray_6, and the MIPI interface MIPI7 corresponds to the pixel sub-array Subarray_7. The working modes of the chip include full sampling mode, subsampling mode, and windowing mode.

2. The image acquisition and processing chip according to claim 1, characterized in that It includes a timing control module, a data reading module, a data transmission module, a data processing module, a data storage module, and a communication interface.

3. The image acquisition and processing chip according to claim 1, characterized in that, The analog circuit includes a pixel measurement and data generation module, a switch physical implementation module, a temperature detection module, a voltage conversion module, and a clock generation module.

4. The image acquisition and processing chip according to claim 1, wherein, The pixel sub-array Subarray_2 and the pixel sub-array Subarray_0 are vertically mirror-symmetrical. The pixel sub-array Subarray_3 and the pixel sub-array Subarray_1 are vertically mirror-symmetrical. The pixel sub-array Subarray_6 and the pixel sub-array Subarray_4 are vertically mirror-symmetrical. The pixel sub-array Subarray_7 and the pixel sub-array Subarray_5 are vertically mirror-symmetrical.

5. The image acquisition and processing chip according to claim 1, characterized in that When the working mode is the full sampling mode, the pixel read order of each row of the pixel sub-arrays Subarray_1, Subarray_4, and Subarray_5 is the same as that of the pixel sub-array Subarray_0, and the pixel read order of each row of the pixel sub-arrays Subarray_3, Subarray_6, and Subarray_7 is the same as that of the pixel sub-array Subarray_2.

6. The image acquisition and processing chip according to claim 1, characterized in that When the working mode is the full sampling mode, the pixel sub-arrays Subarray_0, Subarray_1, Subarray_4, and Subarray_5 all read pixels from row 0 to row 17023, and the pixel sub-arrays Subarray_2, Subarray_3, Subarray_6, and Subarray_7 all read pixels from row 34047 to row 17024.

7. The image acquisition and processing chip according to claim 1, characterized in that, When the working mode is the decimated sampling mode, decimated sampling is performed on the rows and columns of the entire image sensor pixel array, and only the decimated pixels are output.

8. The image acquisition and processing chip according to claim 1, characterized in that, When the working mode is the windowing mode, 1 to 16 windows are randomly opened in the entire image sensor pixel array, and the size of each window is 512x512 pixels.

9. The image acquisition and processing chip according to claim 1, characterized in that, The data output by each pixel is 10 bits.

Images