Image sensing system and image sensing data processing method
By designing an asymmetric register layout for the storage device in the image sensing system and using SIPO circuitry, the number of registers to be started was reduced, the high power consumption problem was solved, and a significant reduction in power consumption was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PIXART IMAGING INC
- Filing Date
- 2022-03-29
- Publication Date
- 2026-07-14
AI Technical Summary
In existing image processing architectures, the flip-flops of storage devices and filters are triggered simultaneously on each clock pulse, leading to high power consumption.
The design employs a storage device with more registers in the first direction than in the second direction, and couples the storage device and the filter through a SIPO circuit. The registers of the storage device are read in series and written to the registers of the filter according to the address.
By reducing the number of registers started, the power consumption of the image sensing system is significantly reduced.
Smart Images
Figure CN116156338B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an image sensing system and an image sensing data processing method, and more particularly to an image sensing system and an image sensing data processing method that can reduce power consumption during image sensing data processing. Background Technology
[0002] Existing image processing architectures typically include filters for processing image sensing data and a storage device (or pixel storage device) formed by an array of shift registers (e.g., flip-flops) for storing the image sensing data. However, in such an architecture, the shift registers of the storage device and the flip-flops of the filters are triggered simultaneously on each clock pulse to shift the received image sensing data, resulting in high power consumption.
[0003] For example, if the filter is a 4×4 filter that receives 8 bits of image sensing data, then the filter includes 4×4×8 = 128 shift registers. Alternatively, if the storage device includes 3 rows, each containing 32 shift registers and receiving 8 bits of image sensing data, then the storage device includes 3×32×8 = 768 shift registers. In this case, a total of 896 flip-flops are triggered simultaneously on each clock pulse, resulting in high power consumption. Summary of the Invention
[0004] One objective of this invention is to disclose an image sensing system that can reduce power consumption.
[0005] Another objective of this invention is to disclose an image sensing data processing method that can reduce power consumption.
[0006] An embodiment of the present invention discloses an image sensing system, characterized in that it includes: a storage device including a first register, wherein the number of the first registers in a first direction of the storage device is greater than the number of the first registers in a second direction of the storage device; a filter including a second register; and a SIPO (Serial-In Parallel-Out) circuit coupled between the storage device and the filter.
[0007] Another embodiment of the present invention discloses an image sensing data processing method used in a storage device including a first register and a filter including a second register, characterized in that it includes: serially reading image sensing data of the first register in a second direction of the storage device, wherein the number of the first registers in a first direction of the storage device is greater than the number of the first registers in the second direction of the storage device; and writing the image sensing data into the second register according to the address of the serially read first register.
[0008] According to the aforementioned embodiments, since only a small number of temporary registers need to be activated when reading and writing data, power consumption can be greatly reduced. Attached Figure Description
[0009] Figure 1 A block diagram of an image sensing system according to an embodiment of the present invention is shown.
[0010] Figure 2 The illustration depicts an embodiment of the present invention. Figure 1 A schematic diagram showing the detailed structure of the filter and storage device.
[0011] Figure 3 The illustration depicts an embodiment of the present invention. Figure 2 The diagram shows the operation of the filter and storage device.
[0012] Figure 4 The illustration depicts an embodiment of the present invention. Figure 1 A schematic diagram of the pixel array shown.
[0013] Figures 5 to 6 The illustration depicts an embodiment of the present invention. Figure 1 The image sensing system shown is in operation.
[0014] Figure 7 A flowchart illustrating an image sensing data processing method according to an embodiment of the present invention is shown.
[0015] The reference numerals in the attached figures are explained as follows:
[0016] 100 Image Sensing System
[0017] 101 pixel array
[0018] 103 Reading Circuit
[0019] 105 ADC107 storage device
[0020] 109 filter
[0021] 111 SIPO Circuit
[0022] Columns C21, C22, C23, and C24
[0023] FF1, FF2, FF3 Shifters
[0024] Rows R11, R12, R13, R21, R22, R23, R24 Detailed Implementation
[0025] The present invention will be described below with reference to several embodiments. It should be noted that the elements in each embodiment can be implemented by hardware (e.g., a device or circuit) or firmware (e.g., at least one program written in a microprocessor). Furthermore, the terms "first," "second," and similar descriptions in the following description are only used to define different elements, parameters, data, signals, or steps, and are not intended to limit their order. For example, the first device and the second device can be devices with the same structure but different from each other.
[0026] Figure 1 A block diagram of an image sensing system according to an embodiment of the present invention is shown. Figure 1 As shown, the image sensing system 100 includes a pixel array 101, a readout circuit 103, an ADC (Analog to Digital Converter) 105, a storage device 107, a filter 109, and a SIPO (Serial-In Parallel-Out) circuit 111.
[0027] Pixel array 101 includes at least one pixel for generating image sensing data (e.g., image sensing charge). Readout circuit 103 reads the image sensing data from pixel array 101. ADC 105 converts the image sensing data into digital image sensing values. Storage device 107 includes at least two first registers (e.g., flip-flops). The number of first registers in a first direction of storage device 107 is greater than the number of first registers in a second direction of storage device 107. In one embodiment, first registers in the first direction refer to first registers in a row of storage device 107, while first registers in the second direction refer to first registers in a column of storage device 107. In other words, the number of first registers in a row of storage device 107 is greater than the number of first registers in a column of storage device 107. Storage device 107 may be, for example, a memory device or any other electronic device capable of temporarily storing data.
[0028] Filter 109 includes at least two second registers. The first and second registers can be shift registers. SIPO circuitry 111 is coupled between storage device 107 and filter 109. Filter 109 and storage device 107 each receive at least a portion of the digital image sensing values from ADC 105. Furthermore, SIPO 111 is used to read data from the first register in the second direction of data storage device 107 and to write data into the second register in filter 109 according to the address of the read first register. In one embodiment, filter 109 is used to filter noise from the digital image sensing values. The detailed operation of storage device 107, filter 109, and SIPO 111 will be described in more detail later.
[0029] Figure 2 An illustration is provided according to an embodiment of the present invention. Figure 1 A schematic diagram showing the detailed structure of the filter and storage device. (See attached diagram.) Figure 2 As shown, the storage device 107 includes 36×3 first registers, which are represented by blocks. Furthermore, the filter 109 includes 4×4 second registers, also represented by blocks. Moreover, the SIPO circuit 111 is used for column-oriented ( Figure 2 (In the X direction shown) reads the data stored in the first register of the storage device 107, and moves the read data into the second register of the filter 109 according to the column address of the first register being read.
[0030] Figure 3 An illustration is provided according to an embodiment of the present invention. Figure 2 A schematic diagram illustrating the operation of the filter and storage device. Figure 3 In the embodiment described, the SIPO circuit 111 includes multiple flip-flops FF1, FF2, and FF3 for shifting data from the storage device 107 to the filter 109. Data in a first register in a row of the storage device 107 is shifted by the SIPO circuit 111 to the corresponding row of a second register, even if the first register is read in column direction. For example, data in row R11 of the storage device 107 is shifted by the SIPO circuit 111 to row R22 of the second register. As another example, data in row R12 of the storage device 107 is shifted by the SIPO circuit 111 to row R23 of the second register. Specific operation will be described in the following embodiments.
[0031] Figure 4 An illustration is provided according to an embodiment of the present invention. Figure 1 A schematic diagram of the pixel array is shown. (As shown) Figure 4As shown, the pixel array has 36×6 pixels, corresponding to digital image sensing values 0-215. Please note that 0-215 here only represents the pixel sequence corresponding to the digital image sensing values, and does not represent the actual values of the digital image sensing values.
[0032] Figures 5 to 6 An illustration is provided according to an embodiment of the present invention. Figure 1 The image sensing system shown is in operation. Please also note that... Figure 5 and Figure 6 In the embodiment, rows R11 and R21 represent the first row of the first temporary register in the storage device 107 and the first row of the second temporary register in the filter 109, respectively. However, any row of the first temporary register in the storage device 107 and any row of the second temporary register in the filter 109 can be set as the first row.
[0033] exist Figure 5 In this embodiment, digital image sensing values are shifted to a first register and correspondingly copied to a second register until the first row of the first register is filled with digital image sensing values. For example, digital image sensing values 0-35 are input to the first register in row R11 and copied to the second register in row R21. Therefore, after digital image sensing values 0-35 are input to the first register in row R11, the second register in row R21 stores digital image sensing values 32-35. Then, digital image sensing value 36 is input to the second register in row R21, column C21, and the first register in row R11, column C11. In this case, the digital image sensing value 0, originally stored in the first register in row R11, column C11, is shifted to the first register in the same column (column C11) of the next row (row R12) and copied to the second register in row R22, column C21. In this case, the digital image sensing values 35-33 in row R21 will be shifted.
[0034] Then, the digital image sensing value 37 is input into the second register located in row R21 and column C21, and the first register located in row R11 and column C12. The digital image sensing value 1, originally stored in the first register in row R11 and column C12, is shifted to the first register in row R12 (the next row) and copied to the second register in row R22 and column C21. In this case, the digital image sensing value 0, stored in the second register in row R22 and column C21, is shifted to the second register in row R22 and column C22. Furthermore, the digital image sensing values 36-34 in the second register in row R21 are shifted.
[0035] Figure 5The operation can be summarized as follows: After the first row of the first register is filled with digital image sensing values and a new digital image sensing value (e.g., 36) is written from the ADC 105 to the first row of the second sensor, the new digital image sensing value is written to the last second register in the first row of the second sensor (e.g., the second register in row R11, column C11). The digital image sensing value temporarily stored in the Nth (e.g., N=1) first register in the first row of the first register is shifted to the Nth first register in the second row of the first register, and the digital image sensing value shifted to the Nth first register in the second row of the first register is copied to the second registers in the second row and first column of the second register (e.g., the second registers in row R22 and column C21). N represents the number of times a new digital image sensing value is written from the ADC to the first row of the second register. In this embodiment, when N=2, the new digital image sensing value is 37, and the digital image sensing value 1 is copied to the second register located in row R22 and column C21 in the same manner.
[0036] Additionally, when N is greater than 1, the digital image sensing value copied to the second register in the second row and first column is shifted to the second register in the second row and M column, where M = N. For example, when N = 2, the digital image sensing value 37 is input to the storage device 107, and the digital image sensing value 1 is shifted to the first register in row R12 and column C12. In this case, the digital image sensing value 0 stored in the second register in row R21 and column C21 is shifted to the second register in row R21 and column C22.
[0037] Can be repeated Figure 5 The steps described herein, such as Figure 6 As shown, all image sensing data generated from a single sensing operation of pixel array 109 has been processed. Furthermore, filter 109 performs pixel conversion calculations (i.e., noise filtering) after all second registers are filled with digital image sensing values. Note also that in... Figure 6In this embodiment, after the second row of the first register is filled with digital image sensing values, a new digital image sensing value (e.g., 72) is written from the ADC to the first row of the second register. This new digital image sensing value is then written to the last second register (e.g., the second register in row R21, column C21). The digital image sensing value (e.g., 0) temporarily stored in the Kth (in this embodiment, K=1) first register in the second row of the first register is shifted to the Kth first register in the third row of the first register (e.g., the first register in row R13, column C11). The digital image sensing value shifted to the Kth first register in the third row of the first register is copied to the second register in the third row and the second register in the first column (e.g., the second register in row R23, column C21). Here, K represents the number of times a new digital image sensing value is written from the ADC to the first row of the second register after the second row of the first register is filled with digital image sensing values.
[0038] In the above embodiment, when reading data, only the temporary registers in the rows of the storage device 107 need to be activated, instead of all the temporary registers in the storage device 107, thus greatly reducing power consumption.
[0039] Based on the foregoing description, an image sensing data processing method can be obtained, which is applied to a storage device (e.g., storage device 107) including at least two first registers and a filter (e.g., filter 109) including at least two second registers. Figure 7 This is a flowchart of an image sensing data processing method disclosed in an embodiment of the present invention, including:
[0040] Step 701
[0041] Image sensing data of the first register in the second direction of the storage device is read in a serial manner, wherein the number of the first registers in the first direction of the storage device is greater than the number of the first registers in the second direction of the storage device.
[0042] In one embodiment, the first register in the first direction refers to the first register in a row of the storage device, while the first register in the second direction refers to the first register in a column of the storage device.
[0043] Step 703
[0044] Image sensing data is written to the second register based on the address of the first register that is read in serial order.
[0045] Read and write operations can be performed by SIPO circuits, for example Figure 1 The SIPO circuit shown.
[0046] According to the aforementioned embodiments, since only a small number of temporary registers need to be activated when reading and writing data, power consumption can be greatly reduced.
[0047] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. An image sensing system, characterized in that, include: A storage device includes a first register, wherein the number of the first registers in a first direction of the storage device is greater than the number of the first registers in a second direction of the storage device. The filter includes a second temporary register; and A SIPO (Serial-In Parallel-Out) circuit is coupled between the storage device and the filter; The image sensing system further includes: A pixel array, comprising pixels, is used to generate image sensing data; ADC is used to convert the image sensing data into digital image sensing values; The filter and the storage device respectively receive the digital image sensing values; The digital image sensing value is shifted to the first register and correspondingly copied to the second register until the first row of the first register is filled with the digital image sensing value. After the first row of the first register is filled with the digital image sensing value and a new digital image sensing value is written from the ADC to the first row of the second register, the new digital image sensing value is written to the last second register of the first row of the second register. The digital image sensing value temporarily stored in the Nth first register of the first row of the first register is shifted to the Nth first register of the second row of the first register, and the digital image sensing value shifted to the Nth first register of the second row of the first register is copied to the second register located in the first column of the second row of the second register. Where N represents the number of times the new digital image sensing value is written from the ADC to the first row of the second register; The digital image sensing value copied to the second register located in the second row and first column of the second register is shifted to the second register located in the second row and M column of the second register, where N is greater than 1 and M = N.
2. The image sensing system as described in claim 1, characterized in that, The SIPO circuit is used to read data from the first register in the second direction of the storage device and to write the data to the second register according to the address of the first register that has been read.
3. The image sensing system as described in claim 1, characterized in that, This filter is used to remove noise from the digital image sensing values.
4. The image sensing system as described in claim 1, characterized in that, The first register in the first direction of the storage device is the first register on the row of the storage device, and the first register in the second direction of the storage device is the first register on the column of the storage device.
5. The image sensing system as described in claim 1, characterized in that, After all the second registers are filled with the digital image sensing values, the filter performs pixel conversion calculations.
6. The image sensing system as described in claim 1, characterized in that, After the second row of the first register is filled with the digital image sensing value and a new digital image sensing value is written from the ADC to the first row of the second register, the new digital image sensing value is written to the last second register. The digital image sensing value temporarily stored in the Kth first register of the second row of the first register is shifted to the Kth first register of the third row of the first register, and the digital image sensing value shifted to the Kth first register of the third row of the first register is copied to the third row of the second register and the second register in the first column. Where K represents the number of times the new digital image sensing value is written from the ADC to the first row of the second register after the second row of the first register is filled with the digital image sensing value.
7. An image sensing data processing method, used in a storage device including a first register and a filter including a second register, characterized in that, include: Image sensing data of the first register in a serial manner is read from the second direction of the storage device, wherein the number of the first registers in the first direction of the storage device is greater than the number of the first registers in the second direction of the storage device; and The image sensing data is written into the second temporary register according to the address of the first temporary register that is read serially; The storage device and the filter are included in an image sensing system, which further includes: A pixel array, comprising pixels, is used to generate image sensing data; ADC is used to convert the image sensing data into digital image sensing values; The filter and the storage device respectively receive the digital image sensing values; The digital image sensing value is shifted to the first register and correspondingly copied to the second register until the first row of the first register is filled with the digital image sensing value. After the first row of the first register is filled with the digital image sensing value and a new digital image sensing value is written from the ADC to the first row of the second register, the new digital image sensing value is written to the last second register of the first row of the second register. The digital image sensing value temporarily stored in the Nth first register of the first row of the first register is shifted to the Nth first register of the second row of the first register, and the digital image sensing value shifted to the Nth first register of the second row of the first register is copied to the second register located in the first column of the second row of the second register. Where N represents the number of times the new digital image sensing value is written from the ADC to the first row of the second register; The digital image sensing value copied to the second register located in the second row and first column of the second register is shifted to the second register located in the second row and M column of the second register, where N is greater than 1 and M = N.
8. The image sensing data processing method as described in claim 7, characterized in that, This filter is used to remove noise from the digital image sensing values.
9. The image sensing data processing method as described in claim 7, characterized in that, The first register in the first direction of the storage device is the first register on the row of the storage device, and the first register in the second direction of the storage device is the first register on the column of the storage device.
10. The image sensing data processing method as described in claim 7, characterized in that, After all the second registers are filled with the digital image sensing values, the filter performs pixel conversion calculations.
11. The image sensing data processing method as described in claim 7, characterized in that, After the second row of the first register is filled with the digital image sensing value and a new digital image sensing value is written from the ADC to the first row of the second register, the new digital image sensing value is written to the last second register. The digital image sensing value temporarily stored in the Kth first register of the second row of the first register is shifted to the Kth first register of the third row of the first register, and the digital image sensing value shifted to the Kth first register of the third row of the first register is copied to the third row of the second register and the second register in the first column. Where K represents the number of times the new digital image sensing value is written from the ADC to the first row of the second register after the second row of the first register is filled with the digital image sensing value.