An image storage method and apparatus for a delta pixel array
By preprocessing and segmenting the data stream of the triangular pixel array, the problems of image geometric distortion and data discontinuity are solved, achieving efficient image storage and reducing resource consumption and system costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG SHUANGYUAN TECH CO LTD
- Filing Date
- 2026-03-25
- Publication Date
- 2026-06-26
AI Technical Summary
The triangular pixel array introduces problems such as image geometric distortion, data discontinuity, and data redundancy during data storage, and existing technologies cannot effectively solve these problems.
By preprocessing the triangular pixel array data stream output by the line scan camera, a standard data packet is generated and divided into multiple sub-data packets according to the bit order. The storage space is divided according to the data interface bit width of the memory, the storage length and boundary point of each sub-data packet are calculated, and the data packets are segmented and cached. Finally, the data packets are written into the storage space to achieve data correction and splicing.
It corrects image geometric distortion, ensures data packet continuity, reduces storage resource usage, lowers system cost and power consumption, and achieves theoretically limit-limit memory bandwidth efficiency.
Smart Images
Figure CN121924384B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and particularly to an image storage method and device for a pin-shaped pixel array. Background Art
[0002] As an important industrial imaging device, a line scan camera realizes high-resolution and high-real-time imaging of a target by continuously scanning a moving object at high speed with a single row of pixels. Most line scan cameras have a pixel array distributed in a "one" shape, and there are also some special pixel arrays, such as a "pin" shape distribution. Referring to Figure 1 , a pin-shaped pixel array means that a row of pixels is divided into several pixel blocks, each pixel block includes a plurality of pixels arranged linearly, and longitudinally, the pixel blocks are staggered up and down in sequence, forming two rows as a whole. Horizontally, there are some overlapping pixels between adjacent pixel blocks. Due to this special structure, the following difficulties are introduced in data storage:
[0003] 1. Pixel longitudinal staggering causes image geometric distortion: A line scan camera generates an image row by row depending on the relative movement between the object and the camera. In a traditional regular pixel array, the longitudinal positions of a row of pixels are the same, so the captured row image strictly corresponds to the object features and can be directly stored row by row. However, the longitudinal staggered structure of the pin-shaped pixel array causes the captured row images not to be geometrically aligned during the continuous movement of the object. When scanning and storing row by row to DDR, it will cause image geometric distortion;
[0004] 2. The non-standardization of the data volume size of pixel blocks leads to discontinuous image data: The output data stream of a traditional pixel array is usually carefully designed and can be relatively easily organized into data packets aligned with the memory bit width. For a pin-shaped pixel array, the amount of valid pixel data contained in each pixel block is often a non-standard value, while the data interface bit widths of memories such as DDR (such as 128 bits, 256 bits) are fixed and standardized. When the data volume of a pixel block is not an integer multiple of the memory bit width, if the data of a pixel block is simply stored in a storage unit, in order to fill the fixed bit width, padding must be performed before and after the data, which will lead to discontinuous image data;
[0005] 3. Pixel horizontal overlap causes image duplication: The horizontal overlap structure of the pin-shaped pixel array causes the same object feature to be captured by multiple pixels. If not processed, it will generate data redundancy, significantly increasing the invalid storage and transmission load.
[0006] Patent CN109660774A discloses an FPGA-based method for composite color edge removal in linear array misaligned images. It sets up two shift registers to buffer pixels in the R and G rows of color arrays, respectively; three RAMs to store RGB correction parameters for correcting pixel data captured by a linear array camera; a 3r-bit register to combine three r-bit single-channel color values into a single 3r-bit data value; a background removal module to perform background discrimination on the entire 3r-bit data; a shift register to store the background-removed pixels, and a T*T-bit register to perform edge removal; the result is stored in register di and output. This method achieves color misalignment correction for three-channel pixel buffering, but it is not suitable for single-channel triangular pixel arrays. Summary of the Invention
[0007] This invention provides an image storage method and apparatus for a triangular pixel array to solve the problem of misalignment in the storage of triangular pixel arrays in line scan cameras.
[0008] An image storage method for a triangular pixel array includes:
[0009] The pixel block data stream of the triangular pixel array output by the line scan camera is acquired and preprocessed to generate a standard data packet;
[0010] Each pixel block in each row of the image in the standard data packet is divided according to its positional order to obtain multiple first sub-data packets, second sub-data packets, and third sub-data packets;
[0011] The storage space is divided according to the data interface bit width of the memory and the standard data packet;
[0012] Calculate the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet;
[0013] The third sub-data packet is segmented and cached based on the aforementioned boundary point;
[0014] Based on the pre-set row offset of the second sub-data packet and the storage length, the first and second sub-data packets are written into the storage space, and the segmented third sub-data packet is concatenated with the corresponding first sub-data packet and then written into the storage space.
[0015] Furthermore, the pixel block data stream is preprocessed to generate standard data packets, including:
[0016] Identify and remove duplicate pixels in each pixel block of the pixel block data stream to obtain valid pixels;
[0017] Each pixel block containing only valid pixels is spliced together according to the data interface width of the memory to obtain multiple standard data transmission units, which together form the standard data packet.
[0018] Furthermore, for each row of the image, each pixel block is divided according to its positional order, specifically including:
[0019] Each row of the image is divided into a first sub-data packet by dividing the pixel blocks from the odd-numbered order into a first sub-data packet, thus obtaining multiple first sub-data packets;
[0020] Each row of the image is divided into even-numbered pixel blocks into a second sub-data packet, resulting in multiple second sub-data packets;
[0021] The pixels from two adjacent pixel blocks in each row of the image are combined and divided into a third sub-data packet to obtain multiple third sub-data packets.
[0022] Furthermore, the storage space is divided according to the data interface bit width of the memory and the standard data packet, including:
[0023] Calculate the storage unit size for storing one row of an image based on the total number of effective pixels in a row, the data bit width of the effective pixels, and the data interface bit width of the memory.
[0024] Based on the size of the storage unit that stores one row of images, determine the row starting address of each storage unit in the memory's storage space.
[0025] Furthermore, the storage lengths of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet, are calculated separately, including:
[0026] Based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the data interface bit width of the memory, and the boundary point of the relevant third sub-data packet, calculate the storage length of each first sub-data packet and the second sub-data packet.
[0027] The boundary point of the current third sub-data packet is calculated based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the storage length of the first sub-data packet / second sub-data packet, the data interface bit width of the memory, and the boundary point of the previous third sub-data packet.
[0028] Furthermore, the third sub-data packet is segmented and cached based on the aforementioned boundary point, including:
[0029] Based on the dividing point, the third sub-data packet is segmented to obtain a first data segment belonging to an odd-order pixel block and a second data segment belonging to an even-order pixel block, and the first data segment and the second data segment are temporarily stored in an independent cache queue.
[0030] Further, according to the pre-set row offset of the second sub-data packet and the storage length, the first and second sub-data packets are written into the storage space, including:
[0031] The storage unit is divided according to the storage length of the first sub-data packet and the second sub-data packet;
[0032] Based on the row number of the current row image, determine the starting address of the row in the storage space where the first sub-data packet is stored;
[0033] Based on the row start address and storage length of the first sub-data packet, each complete first sub-data packet is written to the corresponding position of the storage unit of the corresponding row;
[0034] Determine the starting address of the second sub-data packet's row in the storage space based on the row offset of the second sub-data packet;
[0035] Based on the row start address and storage length of the second sub-data packet, each complete second sub-data packet is written to the corresponding position in the storage unit of the corresponding row.
[0036] Furthermore, the segmented third sub-data packet is concatenated with the corresponding first sub-data packet and then written into the storage space, including:
[0037] Based on the row number of the current row image and the row offset of the second sub-data packet, determine the row number to which the second data segment to be read belongs, and read the (j-1)th second data segment from the buffer queue;
[0038] The (j-1)th second data segment is concatenated with the start or end data segment of the jth first sub-data packet to obtain the concatenated data packet;
[0039] Based on the row start address of the current first sub-data packet, the concatenated data packet is written to the corresponding position of the storage unit of the corresponding row;
[0040] Where j is the number of the first sub-data packet / second sub-data packet.
[0041] Furthermore, if the second data segment is the starting data segment of the second sub-data packet, it is concatenated with the ending data segment of the first sub-data packet; if the second data segment is the ending data segment of the second sub-data packet, it is concatenated with the starting data segment of the first sub-data packet.
[0042] An image storage device for a triangular pixel array, comprising:
[0043] The image acquisition module is used to acquire the pixel block data stream of the triangular pixel array output by the line scan camera, perform preprocessing, and generate standard data packets;
[0044] The data partitioning module is used to partition each pixel block in each row of the image in the standard data packet according to the position order to obtain multiple first sub-data packets, second sub-data packets and third sub-data packets;
[0045] A space partitioning module is used to partition the storage space according to the data interface bit width of the memory and the standard data packet;
[0046] The calculation module is used to calculate the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet.
[0047] The segmentation module segments and caches the third sub-data packet based on the boundary point;
[0048] The data writing module is used to write the first and second sub-data packets into the storage space according to the pre-set row offset of the second sub-data packet and the storage length, and to concatenate the segmented third sub-data packet with the corresponding first sub-data packet before writing it into the storage space.
[0049] Furthermore, the image acquisition module preprocesses the pixel block data stream to generate a standard data packet, including:
[0050] Identify and remove duplicate pixels in each pixel block of the pixel block data stream to obtain valid pixels;
[0051] Each pixel block containing only valid pixels is spliced together according to the data interface width of the memory to obtain multiple standard data transmission units, which together form the standard data packet.
[0052] Furthermore, for each row of the image, the data segmentation module divides each pixel block according to its positional order, specifically including:
[0053] Each row of the image is divided into a first sub-data packet by dividing the pixel blocks from the odd-numbered order into a first sub-data packet, thus obtaining multiple first sub-data packets;
[0054] Each row of the image is divided into even-numbered pixel blocks into a second sub-data packet, resulting in multiple second sub-data packets;
[0055] The pixels from two adjacent pixel blocks in each row of the image are combined and divided into a third sub-data packet to obtain multiple third sub-data packets.
[0056] Furthermore, the space partitioning module partitions the storage space according to the data interface bit width of the memory and the standard data packet, including:
[0057] Calculate the storage unit size for storing one row of an image based on the total number of effective pixels in a row, the data bit width of the effective pixels, and the data interface bit width of the memory.
[0058] Based on the size of the storage unit that stores one row of images, determine the row starting address of each storage unit in the memory's storage space.
[0059] Furthermore, the calculation module calculates the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet, including:
[0060] Based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the data interface bit width of the memory, and the boundary point of the relevant third sub-data packet, calculate the storage length of each first sub-data packet and the second sub-data packet.
[0061] The boundary point of the current third sub-data packet is calculated based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the storage length of the first sub-data packet / second sub-data packet, the data interface bit width of the memory, and the boundary point of the previous third sub-data packet.
[0062] Furthermore, the segmentation module segments and caches the third sub-data packet based on the boundary point, including:
[0063] Based on the dividing point, the third sub-data packet is segmented to obtain a first data segment belonging to an odd-order pixel block and a second data segment belonging to an even-order pixel block, and the first data segment and the second data segment are temporarily stored in an independent cache queue.
[0064] Further, the data writing module writes the first and second sub-data packets into the storage space according to the pre-set row offset of the second sub-data packet and the storage length, including:
[0065] The storage unit is divided according to the storage length of the first sub-data packet and the second sub-data packet;
[0066] Based on the row number of the current row image, determine the starting address of the row in the storage space where the first sub-data packet is stored;
[0067] Based on the row start address and storage length of the first sub-data packet, each complete first sub-data packet is written to the corresponding position of the storage unit of the corresponding row;
[0068] Determine the starting address of the second sub-data packet's row in the storage space based on the row offset of the second sub-data packet;
[0069] Based on the row start address and storage length of the second sub-data packet, each complete second sub-data packet is written to the corresponding position in the storage unit of the corresponding row.
[0070] Furthermore, the data writing module concatenates the segmented third sub-data packet with the corresponding first sub-data packet and writes it into the storage space, including:
[0071] Based on the row number of the current row image and the row offset of the second sub-data packet, determine the row number to which the second data segment to be read belongs, and read the (j-1)th second data segment from the buffer queue;
[0072] The (j-1)th second data segment is concatenated with the start or end data segment of the jth first sub-data packet to obtain the concatenated data packet;
[0073] Based on the row start address of the current first sub-data packet, the concatenated data packet is written to the corresponding position of the storage unit of the corresponding row;
[0074] Where j is the number of the first sub-data packet / second sub-data packet.
[0075] Furthermore, if the second data segment is the starting data segment of the second sub-data packet, it is concatenated with the ending data segment of the first sub-data packet; if the second data segment is the ending data segment of the second sub-data packet, it is concatenated with the starting data segment of the first sub-data packet.
[0076] The image storage method and apparatus for a triangular pixel array provided by the present invention have at least the following beneficial effects:
[0077] (1) Correcting image geometric distortion problem: By introducing row offset, the spatially interleaved pixel block data is stored in independent rows with a definite positional relationship at the storage level, thus eliminating image geometric distortion from the source and solving the misalignment problem of the triangular pixel array storage;
[0078] (2) Solve the problem of discontinuity caused by data packet boundaries: The data boundary segmentation and splicing mechanism ensures that even if the data packet crosses the pixel block boundary, the data in each row that is finally stored in the memory is a continuous and complete physical scan row. This simplifies the subsequent reading logic and ensures the inherent continuity of the image rows.
[0079] (3) Reduce the use of additional storage resources: The traditional method of processing the vertical interlacing of the PIN array usually involves first performing vertical image offset and then storing it in memory. However, vertical image offset often relies on a large-capacity on-chip cache. When the offset is large, the required cache capacity increases sharply, becoming the main part of system resource consumption. This invention caches a very small amount of image data and directly completes the vertical image offset correction during the storage stage, which greatly saves valuable on-chip storage resources, simplifies hardware design, and reduces system cost and power consumption. It is particularly suitable for high-performance systems that are sensitive to resources.
[0080] (4) Achieving theoretically limit-limited memory bandwidth efficiency: A method is provided that requires only one write operation to store the data in its final correct location. This minimizes the number of memory accesses and data transfers, thereby achieving the theoretical minimum memory read / write bandwidth usage. This not only reduces the risk of bus congestion but also makes it possible for the system to support higher frame rate image acquisition under limited bandwidth. Attached Figure Description
[0081] Figure 1 This is a schematic diagram of one embodiment of a ternary pixel array.
[0082] Figure 2 This is a flowchart of one embodiment of the image storage method for a triangular pixel array provided by the present invention.
[0083] Figure 3 This is a schematic diagram illustrating the writing of a row of data in the image storage method for a triangular pixel array provided by the present invention.
[0084] Figure 4 This is a schematic diagram illustrating the writing of multiple rows of data in the image storage method for a triangular pixel array provided by the present invention.
[0085] Figure 5 This is a schematic diagram illustrating data writing in one application scenario of the image storage method for a triangular pixel array provided by the present invention.
[0086] Figure 6 This is a schematic diagram illustrating data writing in another application scenario of the image storage method for a triangular pixel array provided by the present invention.
[0087] Figure 7 This is a flowchart of one embodiment of the image storage device for a triangular pixel array provided by the present invention. Detailed Implementation
[0088] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.
[0089] refer to Figure 2 In some embodiments, an image storage method for a triangular pixel array is provided, comprising:
[0090] S1. Obtain the pixel block data stream of the triangular pixel array output by the line scan camera and preprocess it to generate a standard data packet;
[0091] S2. Divide each pixel block in each row of the standard data packet according to the position order to obtain multiple first sub-data packets, second sub-data packets and third sub-data packets;
[0092] S3. Divide the storage space according to the data interface width of the memory and the standard data packet;
[0093] S4. Calculate the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet.
[0094] S5. Segment and cache the third sub-data packet based on the boundary point;
[0095] S6. Based on the pre-set row offset of the second sub-data packet and the storage length, write the first sub-data packet and the second sub-data packet into the storage space, and then concatenate the segmented third sub-data packet with the corresponding first sub-data packet and write it into the storage space.
[0096] Specifically, in step S1, the line scan camera outputs a pixel block data stream. The image is arranged in a triangular pixel pattern, with each row of the image divided into several pixel blocks. Each pixel block includes multiple pixels arranged linearly. Vertically, the pixel blocks are staggered vertically, forming two rows. Horizontally, some pixels overlap between adjacent pixel blocks, such as... Figure 1 As shown.
[0097] The pixel block data stream is preprocessed to generate standard data packets, including:
[0098] S11. Identify and delete duplicate pixels in each pixel block of the pixel block data stream to obtain valid pixels;
[0099] S12. Each pixel block containing only valid pixels is spliced together according to the data interface width of the memory to obtain multiple standard data transmission units, which together form the standard data packet.
[0100] Specifically, in step S11, the pixel blocks arranged in an alternating manner include pixel block 1, pixel block 2, pixel block 3... pixel block N. If the last p pixels of pixel block 1 are the same as the first p pixels of pixel block 2, then the p pixels of the same pixel block 2 are deleted. If the last s pixels of pixel block 2 are the same as the first s pixels of pixel block 3, then the first s pixels of pixel block 3 are deleted, and so on.
[0101] After removing duplicate pixels from a pixel block, the remaining pixel blocks are stitched together until the total accumulated data reaches the data interface bit width of the memory. This data is then encapsulated into a complete data transmission unit conforming to the memory interface standard, resulting in a standard data packet. This step aims to reorganize non-standard-sized pixel block data streams into data transmission units strictly aligned with the memory data interface bit width, without introducing invalid padding data, thus ensuring the continuity of the data stream.
[0102] Furthermore, in step S2, the standard data packet may contain multiple rows of images. For each row of images, each pixel block is divided according to its positional order, specifically including:
[0103] S21. Divide the pixel blocks from odd-numbered order in each row of the image into a first sub-data packet to obtain multiple first sub-data packets;
[0104] S22. Divide the pixel blocks from even-numbered order in each row of the image into a second sub-data packet to obtain multiple second sub-data packets;
[0105] S23. Combine some pixels from two adjacent pixel blocks in each row of the image and divide them into a third sub-data packet to obtain multiple third sub-data packets.
[0106] Specifically, in step S21, pixel blocks originating from odd-numbered sequences (e.g., the 1st, 3rd, 5th...) in a row of images are assigned to the first sub-data packet A. Further, based on the pixel block sequence, the first data packet is A1, A3, A5...A j-1 .
[0107] In step S22, pixel blocks from an even-numbered row of the image (e.g., the 2nd, 4th, 6th...) are assigned to the second sub-data packet B. Further, based on the sequence of the pixel blocks, the second sub-data packets are B2, B4, B6...B j .
[0108] In step S23, data originating from two adjacent pixel blocks (such as the latter half of an odd-order pixel block combined with the first half of an even-order pixel block) are assigned to the third sub-data packet C. Further, based on the sequence of the two pixel blocks from which the data originates, the third sub-data packet is C. 1_2 C 2_3 C 3_4 ...C j-1_j .
[0109] Further, in step S3, the storage space is divided according to the data interface bit width of the memory and the standard data packet, including:
[0110] S31. Calculate the storage unit size for storing a row of images based on the total number of effective pixels in a row, the data bit width of the effective pixels, and the data interface bit width of the memory.
[0111] S32. Determine the starting address of each row of storage units in the memory space based on the size of the storage unit that stores one row of images.
[0112] Specifically, in step S31, the size of the storage unit for storing one row of images is calculated using the following formula:
[0113] L = P * Sensor_Width / DDR_Width; (1)
[0114] Where L is the size of the storage unit for storing a row of images, P is the total number of valid pixels in a row of images, Sensor_Width is the data bit width of a single valid pixel, and DDR_Width is the data interface bit width of the memory.
[0115] In step S32, the starting address Start_Addr of the memory cell in the i-th row is calculated using the following formula:
[0116] Start_Addr(i) = Base_Addr + i×L; (2)
[0117] Where i is the row number of the storage unit, Start_Addr(i) is the starting address of the storage unit in the i-th row, Base_Addr is the starting address of the storage space, and L is the size of the storage unit storing one row of images.
[0118] In step S4, the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet, are calculated, including:
[0119] S41. Calculate the storage length of each first sub-data packet and second sub-data packet based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the data interface bit width of the memory, and the boundary point of the relevant third sub-data packet.
[0120] S42. Calculate the boundary point of the current third sub-data packet based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the storage length of the first sub-data packet / second sub-data packet, the data interface bit width of the memory, and the boundary point of the previous third sub-data packet.
[0121] Further, in step S41, for the first first sub-data packet and the first second sub-data packet, or the first sub-data packet and the second sub-data packet whose boundary point of the previous third sub-data packet is 0, the corresponding storage length is calculated based on the effective pixel count of the corresponding pixel block, the data bit width of the effective pixels, and the data interface bit width of the memory. The calculation formula is as follows:
[0122] (3)
[0123] Among them, L j L represents the storage length of the j-th first sub-data packet / second sub-data packet. When j is odd, L j This represents the storage length of the corresponding first sub-data packet; when j is even, L... j P represents the storage length of the corresponding second sub-data packet.j This represents the number of valid pixels in the j-th pixel block, Sensor_Width represents the data bit width of the valid pixels, and DDR_Width is the data interface bit width of the memory.
[0124] For the first and second sub-data packets where the boundary point of the previous third sub-data packet is not 0, the corresponding storage length is calculated based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the data interface bit width of the memory, and the boundary point of the previous third sub-data packet. The calculation formula is as follows:
[0125] (4)
[0126] Among them, L j L represents the storage length of the j-th first sub-data packet / second sub-data packet. When j is odd, L j This represents the storage length of the corresponding first sub-data packet; when j is even, L... j P represents the storage length of the corresponding second sub-data packet. j This indicates the number of valid pixels in the j-th pixel block, Sensor_Width represents the data bit width of the valid pixels, DDR_Width is the data interface bit width of the memory, and Q... j-1 This is the boundary point of the previous third sub-data packet.
[0127] Further, in step S42, for the first third sub-data packet or the current third sub-data packet whose boundary point is 0, the boundary point of the current third sub-data packet is calculated based on the storage length of the first sub-data packet / second sub-data packet, the number of effective pixels in the corresponding pixel block, the data bit width of the effective pixels, and the data interface bit width of the memory. The calculation formula is as follows:
[0128] Q j =P j ×Sensor_Width-L j ×DDR_Width; Q j-1 =0 || j=1;(5)
[0129] Among them, Q j L is the boundary point of the j-th third sub-data packet. j L represents the storage length of the j-th first sub-data packet / second sub-data packet. When j is odd, L j This represents the storage length of the corresponding first sub-data packet; when j is even, L... j P represents the storage length of the corresponding second sub-data packet. jThis indicates the number of effective pixels in the j-th pixel block, where Sensor_Width is the data bit width of the effective pixels, DDR_Width is the data interface bit width of the memory, and Q... j-1 This is the boundary point of the previous third sub-data packet.
[0130] For the current third sub-data packet whose boundary point of the previous third sub-data packet is not 0, the boundary point of the current third sub-data packet is calculated based on the storage length of the first sub-data packet / second sub-data packet, the number of effective pixels in the pixel block, the data bit width of the effective pixels, the data interface bit width of the memory, and the boundary point of the previous third sub-data packet. The calculation formula is as follows:
[0131] Q j =P j ×Sensor_Width-L j ×DDR_Width-(DDR_Width-Q j-1 ); Q j-1 ≠0; (6)
[0132] Among them, Q j-1 Q is the boundary point of the previous third sub-data packet. j L is the boundary point of the j-th third sub-data packet. j L represents the storage length of the j-th first sub-data packet / second sub-data packet. When j is odd, L j This represents the storage length of the corresponding first sub-data packet; when j is even, L... j P represents the storage length of the corresponding second sub-data packet. j This indicates the number of valid pixels in the j-th pixel block, where Sensor_Width is the data bit width of the valid pixels and DDR_Width is the data interface bit width of the memory.
[0133] Specifically, after the data packet segmentation is completed, the above steps calculate the actual storage length of the first sub-data packet A and the second sub-data packet B in the memory, and perform precise delimitation point calculation for the third sub-data packet C, providing accurate address parameters for subsequent writing and splicing operations.
[0134] The storage length of the third sub-data packet C is one DDR_Width.
[0135] Further, in step S5, the third sub-data packet is segmented and cached based on the demarcation point, including:
[0136] Based on the dividing point, the third sub-data packet is segmented to obtain a first data segment belonging to an odd-order pixel block and a second data segment belonging to an even-order pixel block, and the first data segment and the second data segment are temporarily stored in an independent cache queue.
[0137] Specifically, the third sub-data packet C contains mixed data from two different pixel blocks. Based on the boundary point Q obtained in step S4... j The third sub-data packet C is precisely segmented, resulting in a first data segment belonging to odd-pixel blocks and a second data segment belonging to odd-pixel blocks. The size of each segmented data piece is smaller than the size of a single standard packet, and they are temporarily stored in a separate buffer queue. Using C... j-1_j（Aj） Indicates the third sub-data packet C j-1_j The first sub-data packet A in j The first segment; C j-1_j（Bj） Indicates the third sub-data packet C j-1_j The second segment belonging to the second sub-data packet Bj.
[0138] Further, in step S6, the first and second sub-data packets are written into the storage space according to the preset row offset of the second sub-data packet and the storage length, including:
[0139] S61. The storage unit is divided according to the storage length of the first sub-data packet and the second sub-data packet;
[0140] S62. Determine the starting address of the row where the first sub-data packet is stored in the storage space based on the row number of the current row image;
[0141] S63. Based on the row start address and storage length of the first sub-data packet, write each complete first sub-data packet into the corresponding position of the storage unit of the corresponding row;
[0142] S64. Determine the starting address of the row of the second sub-data packet in the storage space based on the row offset of the second sub-data packet;
[0143] S65. Based on the row start address and storage length of the second sub-data packet, write each complete second sub-data packet into the corresponding position of the storage unit of the corresponding row.
[0144] Furthermore, in step S61, between the storage locations of the odd-order first sub-data packet and the even-order sub-data packet in a row of storage units, there is also a storage location for storing spliced data packets, with a size of one DDR_Width.
[0145] In step S62, the row starting address of the first sub-data packet is determined according to formula (2), wherein the row number of the current row image is consistent with the row number of the storage unit.
[0146] In step S63, when the first sub-data packet is processed, its complete content is written to the corresponding position of the storage unit corresponding to the current row number i.
[0147] In step S64, taking the offset of the second sub-data packet as an example, the row offset of the second sub-data packet is directly related to the physical design of the triangular pixel array, specifically corresponding to the number of rows converted from the vertical positional deviation of adjacent pixel blocks. This can be obtained by examining the image and calculating the number of vertical row deviations.
[0148] The offset of the second sub-data packet described in this embodiment is related to the direction of object movement. Assuming that the line scan camera first acquires the pixel block of the second sub-data packet, and then acquires the pixel block of the first sub-pixel packet after n rows, the second sub-data packet is considered to be offset, where n is the row offset of the second sub-data packet, and vice versa.
[0149] The starting address of the second sub-data packet is Base_Addr + (i+n)×L; where n is the row offset, Base_Addr is the starting address of the storage space, L is the size of the storage unit for storing one row of images, and i is the row number of the current row of images, which is consistent with the row number of the storage unit.
[0150] In step S65, the complete second sub-data packet is written to the corresponding location of the storage unit with the row starting address Base_Addr + (i+n)×L.
[0151] Further, in step S6, the segmented third sub-data packet is concatenated with the corresponding first sub-data packet and then written into the storage space, including:
[0152] S66. Based on the row number of the current row image and the row offset of the second sub-data packet, determine the row number to which the second data segment to be read belongs, and read the (j-1)th second data segment from the buffer queue.
[0153] S67. The (j-1)th second data segment read is concatenated with the start or end data segment of the jth first sub-data packet to obtain a concatenated data packet;
[0154] S68. Based on the row start address of the current first sub-data packet, write the spliced data packet into the corresponding position of the storage unit of the corresponding row.
[0155] Specifically, the third sub-data packet C, which is temporarily stored because it crosses the data packet boundary, needs to be concatenated and then written.
[0156] In step S66, if the row number of the current image is i and the row offset of the second sub-data packet is n, then the row number of the second data segment to be read is in; the second data segment C belonging to row number in is retrieved from the buffer queue. j-1_j（Bj） .
[0157] In step S67, if the second data segment C j-1_j（Bj）If the second data segment is the starting data segment of the second sub-data packet, then it is concatenated with the ending data segment of the first sub-data packet. If the second data segment C... j-1_j（Bj） If it is the last data segment of the second sub-data packet, then it is concatenated with the first data segment of the first sub-data packet.
[0158] The starting and ending data segments of the first sub-data packet are determined based on the boundary point of the corresponding third sub-data packet, and the length of the ending data segment is Q. j The length of its data segment is DDR_Width-Q. j .
[0159] refer to Figure 3 Assume an image row consists of four pixel blocks, divided into two first sub-data packets, two second sub-data packets, and three third sub-data packets. The storage length of the first sub-data packet is L1, the storage length of the second sub-data packet is L2, the storage length of the third sub-data packet is L3, and the storage length of the fourth sub-data packet is L4. A DDR_Width is reserved between L1 and L2, between L2 and L3, and between L3 and L4. The row offset of the second sub-data packet is n.
[0160] For the image data in the i-th row, determine the row starting address of the first sub-data packet as Base_Addr + i×L, and then... Write the third data packet to position L1 of the i-th row. Write to the L3 position of the i-th row.
[0161] For the two second sub-data packets, write the second data packet to position L2 of row i+n, and write the fourth data packet to position L4 of row i+n.
[0162] The last data segment of the first sub-data packet is concatenated with the second data segment of the first third sub-data packet in the ni-th row, and written to position l after L1 in the i-th row.
[0163] After continuously writing row data, the final storage data structure in memory is as follows: Figure 4 As shown, it can be seen that within a memory row storage space, there is a sub-data packet A of the current row i and a sub-data packet B of the in-th row, which realizes the row offset of sub-data packet B and cleverly solves the problem of vertical misalignment of images in the ternary pixel array image sensor.
[0164] The methods provided in the above embodiments will be further explained below with specific application scenarios.
[0165] Parameters: Total effective pixels of a single row image P=8192, 4 pixel blocks in total, effective pixel count P1=2024, P2=2048, P3=2004, P4=2116, row offset n = 2, pixel data width Sensor_Width=8bit, memory data interface width DDR_Width=128bit, Base_Addr=0.
[0166] Calculate the row storage length L:
[0167] L = P * Sensor_Width / DDR_Width=8192 * 8 / 128 = 512;
[0168] Calculate the storage length L of the first sub-data packet A / the second sub-data packet B. j The boundary point Q between the third sub-data packet C and the third sub-data packet C j :
[0169] ;
[0170] Q1= P1 * Sensor_Width - L1 * DDR_Width= 2024 * 8 - 126 * 128 = 64;
[0171] ;
[0172] Q2 = P2 * Sensor_Width - L2 * DDR_Width - (DDR_Width - Q1)= 2024 * 8 -127 * 128 - (128 - 64) = 64;
[0173] ;
[0174] Q3 = P3* Sensor_Width - L3 * DDR_Width - (DDR_Width - Q2)= 2004 * 8 -124 * 128 - (128 - 64)= 96;
[0175] ;
[0176] Q4 = P4 * Sensor_Width - L4 * DDR_Width - (DDR_Width - Q3)= 2116 * 8 -132 * 128 - (128 - 96) = 0;
[0177] Based on the above parameters, each sub-data packet is written into the storage space, such as... Figure 5 As shown.
[0178] The total number of effective pixels in a single-line image is P=4096, with a total of 4 pixel blocks. The number of effective pixels is P1=1018, P2=1030, P3=1024, P4=1024, the line offset is n=3, the pixel data width is Sensor_Width=8bit, the DDR data interface width is DDR_Width=128bit, and Base_Addr=0.
[0179] Calculate the row storage length L:
[0180] L = P * Sensor_Width / DDR_Width=4096 * 8 / 128 = 256
[0181] Calculate the storage length L of the first sub-data packet A and the second sub-data packet / B. j The boundary point Q between the third sub-data packet C and the third sub-data packet C j :
[0182] ;
[0183] Q1 = P1 * Sensor_Width - L1 * DDR_Width= 1018 * 8 - 63 * 128 = 80;
[0184] ;
[0185] Q2 = P2 * Sensor_Width - L2 * DDR_Width - (DDR_Width - Q1)= 1030 * 8 -128 * 128 - (128 - 80) = 0;
[0186] ;
[0187] Q3 = P3 * Sensor_Width - L3 * DDR_Width= 1024 * 8 - 64 * 128 = 0;
[0188] ;
[0189] Q4 = P4 * Sensor_Width - L4 * DDR_Width= 1024 * 8 - 64 * 128 = 0;
[0190] Based on the above parameters, each sub-data packet is written into the storage space, such as... Figure 6 As shown.
[0191] refer to Figure 7 In some embodiments, an image storage device for a triangular pixel array is provided, comprising:
[0192] The image acquisition module 201 is used to acquire the pixel block data stream of the triangular pixel array output by the line scan camera and perform preprocessing to generate a standard data packet;
[0193] The data partitioning module 202 is used to partition each pixel block in each row of the image in the standard data packet according to the position order to obtain multiple first sub-data packets, second sub-data packets and third sub-data packets;
[0194] Space partitioning module 203 is used to partition storage space according to the data interface bit width of the memory and the standard data packet;
[0195] The calculation module 204 is used to calculate the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet.
[0196] The segmentation module 205 segments and caches the third sub-data packet based on the boundary point;
[0197] The data writing module 206 is used to write the first sub-data packet and the second sub-data packet into the storage space according to the preset row offset of the second sub-data packet and the storage length, and to write the segmented third sub-data packet into the storage space after splicing it with the corresponding first sub-data packet.
[0198] Furthermore, the image acquisition module 201 preprocesses the pixel block data stream to generate a standard data packet, including:
[0199] Identify and remove duplicate pixels in each pixel block of the pixel block data stream to obtain valid pixels;
[0200] Each pixel block containing only valid pixels is spliced together according to the data interface width of the memory to obtain multiple standard data transmission units, which together form the standard data packet.
[0201] Furthermore, for each row of the image, the data segmentation module 202 divides each pixel block according to its positional order, specifically including:
[0202] Each row of the image is divided into a first sub-data packet by dividing the pixel blocks from the odd-numbered order into a first sub-data packet, thus obtaining multiple first sub-data packets;
[0203] Each row of the image is divided into even-numbered pixel blocks into a second sub-data packet, resulting in multiple second sub-data packets;
[0204] The pixels from two adjacent pixel blocks in each row of the image are combined and divided into a third sub-data packet to obtain multiple third sub-data packets.
[0205] Furthermore, the space partitioning module 203 partitions the storage space according to the data interface bit width of the memory and the standard data packet, including:
[0206] Calculate the storage unit size for storing one row of an image based on the total number of effective pixels in a row, the data bit width of the effective pixels, and the data interface bit width of the memory.
[0207] Based on the size of the storage unit that stores one row of images, determine the row starting address of each storage unit in the memory's storage space.
[0208] Furthermore, the calculation module 204 calculates the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet, including:
[0209] Based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the data interface bit width of the memory, and the boundary point of the relevant third sub-data packet, calculate the storage length of each first sub-data packet and the second sub-data packet.
[0210] The boundary point of the current third sub-data packet is calculated based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the storage length of the first sub-data packet / second sub-data packet, the data interface bit width of the memory, and the boundary point of the previous third sub-data packet.
[0211] Furthermore, the segmentation module 205 segments and caches the third sub-data packet based on the boundary point, including:
[0212] Based on the dividing point, the third sub-data packet is segmented to obtain a first data segment belonging to an odd-order pixel block and a second data segment belonging to an even-order pixel block, and the first data segment and the second data segment are temporarily stored in an independent cache queue.
[0213] Further, the data writing module 206 writes the first and second sub-data packets into the storage space according to the preset row offset of the second sub-data packet and the storage length, including:
[0214] The storage unit is divided according to the storage length of the first sub-data packet and the second sub-data packet;
[0215] Based on the row number of the current row image, determine the starting address of the row in the storage space where the first sub-data packet is stored;
[0216] Based on the row start address and storage length of the first sub-data packet, each complete first sub-data packet is written to the corresponding position of the storage unit of the corresponding row;
[0217] Determine the starting address of the second sub-data packet's row in the storage space based on the row offset of the second sub-data packet;
[0218] Based on the row start address and storage length of the second sub-data packet, each complete second sub-data packet is written to the corresponding position in the storage unit of the corresponding row.
[0219] Furthermore, the data writing module 206 concatenates the segmented third sub-data packet with the corresponding first sub-data packet and writes it into the storage space, including:
[0220] Based on the row number of the current row image and the row offset of the second sub-data packet, determine the row number to which the second data segment to be read belongs, and read the (j-1)th second data segment from the buffer queue;
[0221] The (j-1)th second data segment is concatenated with the start or end data segment of the jth first sub-data packet to obtain the concatenated data packet;
[0222] Based on the row start address of the current first sub-data packet, the concatenated data packet is written to the corresponding position of the storage unit of the corresponding row;
[0223] Where j is the number of the first sub-data packet / second sub-data packet.
[0224] Furthermore, if the second data segment is the starting data segment of the second sub-data packet, it is concatenated with the ending data segment of the first sub-data packet; if the second data segment is the ending data segment of the second sub-data packet, it is concatenated with the starting data segment of the first sub-data packet.
[0225] The image storage method and apparatus for a triangular pixel array provided in the above embodiments have at least the following beneficial effects:
[0226] (1) Correcting image geometric distortion problem: By introducing row offset, spatially staggered pixel blocks are stored in independent rows with a defined positional relationship at the storage level, thus eliminating image geometric distortion from the source;
[0227] (2) Solve the problem of discontinuity caused by data packet boundaries: The data boundary segmentation and splicing mechanism ensures that even if the data packet crosses the pixel block boundary, the data in each row that is finally stored in the memory is a continuous and complete physical scan row. This simplifies the subsequent reading logic and ensures the inherent continuity of the image rows.
[0228] (3) Reduce the use of additional storage resources: The traditional method of processing the vertical interlacing of the PIN array usually involves first performing vertical image offset and then storing it in memory. However, vertical image offset often relies on a large-capacity on-chip cache. When the offset is large, the required cache capacity increases sharply, becoming the main part of system resource consumption. This invention caches a very small amount of image data and directly completes the vertical image offset correction during the storage stage, which greatly saves valuable on-chip storage resources, simplifies hardware design, and reduces system cost and power consumption. It is particularly suitable for high-performance systems that are sensitive to resources.
[0229] (4) Achieving theoretically limit-limited memory bandwidth efficiency: A method is provided that requires only one write operation to store the data in its final correct location. This minimizes the number of memory accesses and data transfers, thereby achieving the theoretical minimum memory read / write bandwidth usage. This not only reduces the risk of bus congestion but also makes it possible for the system to support higher frame rate image acquisition under limited bandwidth.
[0230] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.
Claims
1. An image storage method for a triangular pixel array, characterized in that, include: The pixel block data stream of the triangular pixel array output by the line scan camera is acquired and preprocessed to generate a standard data packet; Each pixel block in each row of the standard data packet is divided according to its positional order to obtain multiple first sub-data packets, second sub-data packets, and third sub-data packets: pixel blocks from odd-numbered positions in each row of the image are assigned to a first sub-data packet to obtain multiple first sub-data packets; pixel blocks from even-numbered positions in each row of the image are assigned to a second sub-data packet to obtain multiple second sub-data packets; and a portion of pixels from two adjacent pixel blocks in each row of the image are combined and assigned to a third sub-data packet to obtain multiple third sub-data packets. The storage space is divided according to the data interface bit width of the memory and the standard data packet; Calculate the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet; The third sub-data packet is segmented and cached based on the boundary point: the third sub-data packet is segmented based on the boundary point to obtain a first data segment belonging to an odd-order pixel block and a second data segment belonging to an even-order pixel block, and the first data segment and the second data segment are temporarily stored in an independent cache queue. Based on the pre-set row offset of the second sub-data packet and the storage length, the first and second sub-data packets are written into the storage space, and the segmented third sub-data packet is concatenated with the corresponding first sub-data packet and then written into the storage space.
2. The method according to claim 1, characterized in that, The pixel block data stream is preprocessed to generate standard data packets, including: Identify and remove duplicate pixels in each pixel block of the pixel block data stream to obtain valid pixels; Each pixel block containing only valid pixels is spliced together according to the data interface width of the memory to obtain multiple standard data transmission units, which together form the standard data packet.
3. The method according to claim 1, characterized in that, The storage space is divided according to the data interface bit width of the memory and the standard data packet, including: Calculate the storage unit size for storing one row of an image based on the total number of effective pixels in a row, the data bit width of the effective pixels, and the data interface bit width of the memory. Based on the size of the storage unit that stores one row of images, determine the row starting address of each storage unit in the memory's storage space.
4. The method according to claim 3, characterized in that, Calculate the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet, including: Based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the data interface bit width of the memory, and the boundary point of the relevant third sub-data packet, calculate the storage length of each first sub-data packet and the second sub-data packet. The boundary point of the current third sub-data packet is calculated based on the number of effective pixels in the pixel block, the data bit width of the effective pixels, the storage length of the first sub-data packet / second sub-data packet, the data interface bit width of the memory, and the boundary point of the previous third sub-data packet.
5. The method according to claim 4, characterized in that, Based on the pre-set row offset of the second sub-data packet and the storage length, the first and second sub-data packets are written into the storage space, including: The storage unit is divided according to the storage length of the first sub-data packet and the second sub-data packet; Based on the row number of the current row image, determine the starting address of the row in the storage space where the first sub-data packet is stored; Based on the row start address and storage length of the first sub-data packet, each complete first sub-data packet is written to the corresponding position of the storage unit of the corresponding row; Determine the starting address of the second sub-data packet's row in the storage space based on the row offset of the second sub-data packet; Based on the row start address and storage length of the second sub-data packet, each complete second sub-data packet is written to the corresponding position in the storage unit of the corresponding row.
6. The method according to claim 4, characterized in that, The segmented third sub-data packet is concatenated with the corresponding first sub-data packet and then written into the storage space, including: Based on the row number of the current row image and the row offset of the second sub-data packet, determine the row number to which the second data segment to be read belongs, and read the (j-1)th second data segment from the buffer queue; The (j-1)th second data segment is concatenated with the start or end data segment of the jth first sub-data packet to obtain the concatenated data packet; Based on the row start address of the current first sub-data packet, the concatenated data packet is written to the corresponding position of the storage unit of the corresponding row; Where j is the number of the first sub-data packet / second sub-data packet.
7. The method according to claim 6, characterized in that, If the second data segment is the starting data segment of the second sub-data packet, then it is concatenated with the ending data segment of the first sub-data packet; if the second data segment is the ending data segment of the second sub-data packet, then it is concatenated with the starting data segment of the first sub-data packet.
8. An image storage device for a triangular pixel array, characterized in that, include: The image acquisition module is used to acquire the pixel block data stream of the triangular pixel array output by the line scan camera, perform preprocessing, and generate standard data packets; The data partitioning module is used to partition each pixel block in each row of the standard data packet according to its positional order to obtain multiple first sub-data packets, second sub-data packets, and third sub-data packets: pixel blocks from odd-numbered positions in each row of the image are partitioned into a first sub-data packet to obtain multiple first sub-data packets; pixel blocks from even-numbered positions in each row of the image are partitioned into a second sub-data packet to obtain multiple second sub-data packets; and a portion of pixels from two adjacent pixel blocks in each row of the image are combined and partitioned into a third sub-data packet to obtain multiple third sub-data packets. A space partitioning module is used to partition the storage space according to the data interface bit width of the memory and the standard data packet; The calculation module is used to calculate the storage length of each first sub-data packet and the second data packet, as well as the boundary point of the third sub-data packet. The segmentation module segments and caches the third sub-data packet based on the dividing point: based on the dividing point, the third sub-data packet is segmented to obtain a first data segment belonging to an odd-order pixel block and a second data segment belonging to an even-order pixel block, and the first data segment and the second data segment are temporarily stored in an independent cache queue. The data writing module is used to write the first and second sub-data packets into the storage space according to the pre-set row offset of the second sub-data packet and the storage length, and to concatenate the segmented third sub-data packet with the corresponding first sub-data packet before writing it into the storage space.
Citation Information
Patent Citations
CN109660774A
CN119211757A
CN120612224A