Image data reading system of image sensor and method for reading and organizing image data

By designing a segmented parallel processing image data reading system in the image sensor, and using mask and address control modules to realize dynamic window reading, the circuit of high-frame rate X million pixel image sensor when dynamic window reading is solved is solved. Scale and frame rate issues, achieving high frequency switching and compatibility.

CN112954241BActive Publication Date: 2024-11-15NANJING VPS SEMICONDUCTOR TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110191735.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-02-20
Publication Date
2024-11-15
Estimated Expiration
2041-02-20

AI Technical Summary

Technical Problem

When high-frame rate X million pixel image sensors are read dynamically, the prior art is difficult to be compatible with full pixel, subsampling and dynamic windowing, resulting in increased circuit size or reduced frame rate, and unable to efficiently track targets.

Method used

An image data reading system for an image sensor is designed, including a pixel array, a pixel array control unit, a system control module, a mask generation module and a data recombination module. By splitting pixel data in parallel processing, a pixel mask is generated using a mask generation module. code, control column and row address selection, realize dynamic window reading, and output the reorganized data through the data reorganization module.

Benefits of technology

Dynamic window reading with high frequency switching between image frames is realized, reducing the circuit scale and back-end wiring pressure, and is compatible with full-pixel, multiple sampling and dynamic window reading.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112954241B_ABST
    Figure CN112954241B_ABST
Patent Text Reader

Abstract

The present invention relates to an image data reading system of an image sensor and a method for reading and organizing. The system includes a pixel array, a pixel array control unit, a system control module, a mask generation module and a data reorganization module. The method includes: updating a pixel mask M at the beginning of each frame; waiting for a row of exposure to end, obtaining a row of pixel data D, and mapping and dividing the pixel data D into n parts; selecting a cache register R for each part of the pixel data after division; judging whether the pixel mask value is 1; putting the pixel data corresponding to the pixel mask value into the cache register R, splicing the valid pixel data; judging whether the cache register R is full; when all the pixel data are processed, writing the pixel data of the n parts to the data in the data queue at the same time and reading them out in sequence. The present invention can reduce the circuit scale, reduce the pressure of the back-end engineers, and is compatible with full pixel, multiple sampling and dynamic windowing reading.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an image data reading system of an image sensor, and also relates to an image data reading and organizing method of an image sensor, belonging to the technical field of image sensors. Background Art

[0002] Image sensor is an optoelectronic imaging device widely used in machine vision, target reconnaissance, security monitoring and medical testing. Due to its advantages of low cost and high integration, image sensor chips can integrate more camera system functions, thereby reducing the camera's peripheral circuits and achieving miniaturization of camera size and diversification of functions.

[0003] In target reconnaissance, in order to achieve rapid imaging and monitoring and tracking of key targets, the camera first performs sub-sampling mode, and then enters window scanning and tracking mode after the target is found. It only scans the row and column pixels within the set area in the pixel array to capture an accurate image of the target.

[0004] With the development of technology, the number of pixels contained in image sensors can now reach tens of millions of pixels and billions of pixels. When such large-pixel image sensors are used for target reconnaissance, the window position needs to be quickly and dynamically adjusted according to the changes in the target position to capture the target. Sometimes, multiple targets need to be captured. In this case, it is necessary to dynamically obtain multiple column pixels within a selected range in each frame of the image. However, there is currently no dynamic windowing solution for high-frame-rate billions-pixel image sensors. When the existing methods are applied to high-frame-rate billions-pixel sensors, if the column pixel data is intercepted in parallel, the circuit scale will increase and the back-end wiring resources will be tight. If the column pixel data is intercepted serially, the frame rate will be reduced, and it will not be possible to dynamically track the target at a high frame rate. Summary of the invention

[0005] The primary purpose of the present invention is to overcome the problems existing in the prior art and provide an image data reading system for an image sensor, which can reduce the circuit scale, alleviate the pressure on back-end engineers and be compatible with full-pixel, sub-sampling and dynamic windowing reading.

[0006] In order to solve the above problems, the present invention provides an image data reading system of an image sensor, comprising a pixel array, a pixel array control unit, a system control module, a mask generation module and a data reorganization module, wherein the pixel array control unit comprises a column address control module, a row address control module and a pixel data reading module; the system control module receives instructions through an SPI port, an output end of the system control module is connected to an input end of the pixel array control unit, an output end of the column address control module is connected to a column selection circuit in the pixel array, an output end of the row address control module is connected to a row selection circuit in the pixel array, pixel data in the pixel array is connected to an input end of the data reorganization module through the pixel data reading module; an output end of the system control module is connected to an input end of the data reorganization module through the mask generation module, and the data reorganization module outputs reorganized data through an interface.

[0007] The beneficial effects of the present invention are: 1. The column pixel range to be read can be switched at a high frequency between frames of an image;

[0008] 2. Divide the image into n parts for parallel processing, which reduces the circuit scale, reduces the space for back-end wiring, and alleviates the pressure on back-end wiring;

[0009] 3. Compatible with full pixel, multiple sampling and dynamic window reading.

[0010] Another object of the present invention is to overcome the problems existing in the prior art and provide a method for reading and organizing image data of an image sensor, which can reduce the circuit scale, alleviate the pressure on back-end engineers and be compatible with full-pixel, sub-sampling and dynamic windowing reading.

[0011] In order to solve the above problems, the present invention also provides a method for reading and organizing image data of an image sensor, which comprises the following steps in sequence:

[0012] Step 1: Update the pixel mask M at the beginning of each frame. Each bit of the pixel mask corresponds to a pixel data in a row. Divide the pixel mask into n parts according to the requirements. Each part has M / n bit pixel masks, where n is an integer greater than 1.

[0013] Step 2: Wait for one row of exposure to be completed, obtain one row of pixel data D, map and divide the pixel data D into n parts, corresponding to the pixel mask in step 1, and the n parts of the pixel data D are processed in parallel with the same process;

[0014] Step 3: For each portion of pixel data after segmentation, a buffer register R that can accommodate multiple pixels is selected as a buffer;

[0015] Step 4: Determine whether the pixel mask value is 1. If the pixel mask value is 0, discard the pixel data corresponding to the pixel mask value; if the pixel mask value is 1, proceed to step 5;

[0016] Step 5: Put the pixel data corresponding to the pixel mask value described in step 4 into the buffer register R, and splice the valid pixel data;

[0017] Step 6: Determine whether the buffer register R is full. If the buffer register R is not full, continue to process the next pixel data; if the buffer register R is full, that is, the pixel data has been spliced ​​into a fixed bit width, write the spliced ​​valid pixel data into the data queue;

[0018] When all pixel data are processed, regardless of whether the buffer register is full or not, the spliced ​​valid pixel data is written into the data queue;

[0019] Step 7: When all pixel data are processed, the pixel data of n parts are simultaneously written into the data queue and the data are read out in sequence;

[0020] Step 8: Repeat steps 2-7 until one frame is processed;

[0021] Step 9: Repeat steps 1-8 until the image stops being exposed.

[0022] Furthermore, the data queue is a fifo data queue.

[0023] The beneficial effects of the present invention are: 1. The column pixel range to be read can be switched at a high frequency between frames of an image;

[0024] 2. Divide the image into n parts for parallel processing, which reduces the circuit scale, reduces the space for back-end wiring, and alleviates the pressure on back-end wiring;

[0025] 3. Compatible with full pixel, multiple sampling and dynamic window reading. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of an image data reading system of an image sensor of the present invention;

[0027] Figure 2 for Figure 1 Schematic diagram of pixel data reading, mask generation and buffer register;

[0028] Figure 3 A flow chart of a method for reading and organizing image data from an image sensor;

[0029] Figure 4 This is a schematic diagram of sequential output after data processing is completed in the present invention. DETAILED DESCRIPTION

[0030] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0031] like Figure 1 As shown, the image data reading system of the image sensor of the present invention includes a pixel array, a pixel array control unit, a system control module, a mask generation module and a data reorganization module, the pixel array control unit includes a column address control module, a row address control module and a pixel data reading module; the system control module receives instructions through an SPI port, the output end of the system control module is connected to the input end of the pixel array control unit, the output end of the column address control module is connected to the column selection circuit in the pixel array, the output end of the row address control module is connected to the row selection circuit in the pixel array, the pixel data in the pixel array is connected to the input end of the data reorganization module through the pixel data reading module; the output end of the system control module is connected to the input end of the data reorganization module through the mask generation module, and the data reorganization module outputs the reorganized data through an interface.

[0032] Engineers send image processing instructions to the system control module through the SPI port. The system control module controls the mask generation module to generate a pixel mask. At the same time, the system control module controls the column selection circuit of the pixel array through the column address control module in the pixel array control unit, and controls the row selection circuit of the pixel array through the row address control module in the pixel array unit. The column selection circuit receives the instructions issued by the column address control module, selects the pixel data corresponding to the column address and outputs it; the row selection circuit receives the instructions issued by the row address control module, selects the pixel data corresponding to the row address and outputs it. The pixel data output by the pixel array is read out by the pixel data reading module and recombined with the pixel mask. The recombined pixel data is output through the interface and transmitted to the engineer for the next step of processing.

[0033] like Figures 2 to 4 As shown, the method for reading and organizing image data of an image sensor of the present invention comprises the following steps in sequence:

[0034] Step 1: Update the pixel mask M at the beginning of each frame. Each bit of the pixel mask corresponds to a pixel data in a row. Divide the pixel mask into n parts according to the requirements. Each part has M / n bit pixel masks, where n is an integer greater than 1.

[0035] Step 2: Wait for one row of exposure to be completed, obtain one row of pixel data D, map and divide the pixel data D into n parts, corresponding to the pixel mask in step 1, and the n parts of the pixel data D are processed in parallel with the same process;

[0036] Step 3: For each portion of pixel data after segmentation, select a buffer register R that can accommodate multiple pixels as a buffer; Figure 2 As shown, when processing an image of 18 pixels in a row, the pixel mask is evenly divided into 2 parts, that is, each part has a 9-bit pixel mask, and a row of pixel data D is divided into 2 parts, each part selects a buffer register R, each buffer register R can accommodate 3 pixel data, and the pixel mask value is a binary value of 9'h193;

[0037] Step 4: Determine whether the pixel mask value is 1. If the pixel mask value is 0, discard the pixel data corresponding to the pixel mask value; if the pixel mask value is 1, proceed to step 5;

[0038] Step 5: Put the pixel data corresponding to the pixel mask value described in step 4 into the buffer register R, and splice the valid pixel data;

[0039] Step 6: Determine whether the buffer register R is full. If the buffer register R is not full, continue to process the next pixel data; if the buffer register R is full, that is, the pixel data has been spliced ​​into a fixed bit width, write the spliced ​​valid pixel data into the data queue;

[0040] When all pixel data are processed, regardless of whether the buffer register is full or not, the spliced ​​valid pixel data is written into the data queue;

[0041] Step 7: When all pixel data are processed, the pixel data of n parts are simultaneously written into the data queue and the data are read out in sequence; Figure 4 As shown, if the mask value corresponding to pixel data P0 is 1, it is put into the cache register, and if the mask value corresponding to pixel data P3 is 0, it is discarded. When pixel data P0, P1, and P4 fill the cache register, the cache register is written into the fifo queue and the cache register is cleared. When pixel data P7 and P8 are put into the cache register, although the cache register is not full, all pixel data have been processed, and the cache register at this time is written into the fifo queue. While the first part is working, the second part works in parallel according to the same process;

[0042] Step 8: Repeat steps 2-7 until one frame is processed;

[0043] Step 9: Repeat steps 1-8 until the image stops being exposed.

[0044] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A method for reading and organizing image data of an image sensor, It is characterized in that The following steps are involved: Step 1: Update the pixel mask M at the beginning of each frame. Each bit of the pixel mask corresponds to a pixel data in a row. Divide the pixel mask into n parts according to the requirements. Each part has M / n bit pixel masks, where n is an integer greater than 1. Step 2: Wait for one row of exposure to be completed, obtain one row of pixel data D, map and divide the pixel data D into n parts, corresponding to the pixel mask in step 1, and the n parts of the pixel data D are processed in parallel with the same process, where n is an integer greater than 1; Step 3: For each portion of pixel data after segmentation, a buffer register R that can accommodate multiple pixels is selected as a buffer; Step 4: Determine whether the pixel mask value is 1. If the pixel mask value is 0, discard the pixel data corresponding to the pixel mask value; if the pixel mask value is 1, proceed to step 5; Step 5: Put the pixel data corresponding to the pixel mask value described in step 4 into the buffer register R, and splice the valid pixel data; Step 6: Determine whether the buffer register R is full. If the buffer register R is not full, continue to process the next pixel data; if the buffer register R is full, that is, the pixel data has been spliced ​​into a fixed bit width, write the spliced ​​valid pixel data into the data queue; When all pixel data are processed, regardless of whether the buffer register is full or not, the spliced ​​valid pixel data is written into the data queue; Step 7: When all pixel data are processed, the pixel data of n parts are simultaneously written into the data queue and the data are read out in sequence; Step 8: Repeat steps 2-7 until one frame is processed; Step 9: Repeat steps 1-8 until the image stops being exposed.

2. The method for reading and organizing image data of an image sensor according to claim 1, characterized in that: The data queue is a fifo data queue.