Data processing method, processing circuit, circuit control method and processing system

Abstract

This invention discloses a data processing method, processing circuit, circuit control method, and processing system. The method replicates the value of each pixel on the focal plane into multiple processing units, and simultaneously performs different convolution calculations using multiple identical input values ​​and different weight values ​​to obtain multiple convolution results. The data processing method provided by this invention enables multiple convolution operations on the focal plane to be performed concurrently, improving computational efficiency, parallelism, and running speed.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data processing method, processing circuit, circuit control method, and processing system. Background Technology

[0002] In existing data processing methods, when performing convolution operations on data, the image sensor reads image data row by row, reading one row of pixel values ​​at a time. The read row data is then stored in memory units to prepare for convolution calculations. Alternatively, each element of the convolution kernel is mapped to the corresponding pixel position in the image sensor array, and convolution multiplication is performed. After each convolution operation, the kernel moves one step to the right. When all convolution kernels return to their initial positions in the same row, they move one step down. These steps are repeated until all convolution kernels have traversed the entire image, and all convolution results are summed to obtain the complete convolutional image. Finally, the image returns to its initial state through several rightward and downward movements.

[0003] However, convolution operations in existing data processing methods are inefficient. When reading a row of pixel values ​​and storing them in memory for convolution calculations, the computation speed is compromised due to limitations in bus bandwidth, data transmission, or local memory performance. Alternatively, when the convolution kernel maps to the corresponding pixel position on the image sensor array and moves to the right and down by the corresponding stride to traverse the entire image for convolution operations, a large number of invalid convolution operations are performed, wasting capacity and time. Summary of the Invention

[0004] This invention provides a data processing method, processing circuit, circuit control method, and processing system to enable the use of multiple processing units with the same value and different weight values ​​to perform different convolution calculation operations, so that multiple convolution operations on the focal plane can be performed simultaneously, thereby improving computational efficiency.

[0005] According to a first aspect of the present invention, a data processing method is provided, the method comprising:

[0006] Copy each pixel on the focal plane There are 1 processing unit; wherein the width and height of the convolution kernel are both n, and n≥1; the convolution stride is s; wherein s≥1;

[0007] In a single convolution operation, each processing unit in a pixel is multiplied by the weight of the corresponding convolution kernel to obtain a first convolution result; the first convolution results of the processing units corresponding to the same convolution kernel in the same row are summed to obtain a second convolution result.

[0008] The results of the second convolution between rows are summed to output the final convolution result.

[0009] According to a second aspect of the present invention, a data processing circuit is provided, the data processing circuit comprising: a focal plane module and a pulse width generator module;

[0010] The focal plane module and the pulse width generator module are connected;

[0011] The pulse width generator module is used to output a pulse width modulation signal representing the weights of different convolution kernels to the focal plane;

[0012] The focal plane module includes multiple pixel circuits;

[0013] The multiple pixel circuits of the focal plane module are used to multiply n identical input values ​​with the corresponding pulse width modulation signal to obtain the first convolution result;

[0014] The first convolution result in the pixel circuits corresponding to the same convolution kernel in the same row is accumulated to obtain the second convolution result;

[0015] The results of the second convolution between rows are summed to output the final convolution result.

[0016] According to a third aspect of the present invention, a data processing circuit control method is provided, wherein the data processing circuit includes a controller module; the pulse width generator module includes a weighted register unit and a pulse generation unit;

[0017] The data processing circuit control method includes:

[0018] The controller module controls the adjacent Each of the aforementioned pixel circuits acquires the same value for a single pixel; where n is the size of the convolution kernel and s is the stride of the convolution.

[0019] In a single convolution operation, the controller module controls the weighting register unit to generate weight values ​​based on the convolution kernel and send them to the pulse generation unit; the controller module controls the pulse generation unit to generate pulse width modulation signals based on the received weight values ​​and simultaneously send them to the pixel circuit of the focal plane.

[0020] The controller module controls the accumulation of the output results of the pixel circuits in the focal plane according to the time when each pixel circuit in the focal plane receives the pulse width modulation signal;

[0021] After the output of the pixel circuit is accumulated, the controller module controls the output of the accumulated result.

[0022] According to a fourth aspect of the present invention, a data processing system is provided, the data processing system comprising:

[0023] The data processing circuit and analog-to-digital conversion module provided in any embodiment of the present invention;

[0024] The analog-to-digital conversion module is connected to the data processing circuit, and the analog-to-digital conversion module is used to convert the signal output by the data processing circuit into an analog-to-digital signal.

[0025] This invention provides a data processing method, processing circuit, circuit control method, and processing system. In the technical solution provided by this invention, pixels in the focal plane are copied. For each processing unit, based on its position, it is multiplied by the weight values ​​in different convolution kernels to obtain the final convolution result. Specifically, the product of processing units in the same row of each convolution kernel with their weight values ​​is the first convolution result. The first convolution results between the same rows of the same convolution kernel are added to obtain the second convolution result. The second convolution results between different rows are added to obtain the final convolution result. This achieves multiple convolution results by using multiple identical input values ​​with different weight values ​​to perform different convolution calculations, improving parallelism and running speed.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of a data processing method provided in related technologies;

[0029] Figure 2 This is a schematic diagram of another data processing method provided in related technologies;

[0030] Figure 3 This is a flowchart of a data processing method provided in an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of a data processing method provided in an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of another data processing method provided in an embodiment of the present invention;

[0033] Figure 6This is a schematic diagram of a data processing circuit structure provided in an embodiment of the present invention;

[0034] Figure 7 A pixel circuit diagram of a focal plane module provided in an embodiment of the present invention;

[0035] Figure 8 This is a schematic diagram of another data processing circuit structure provided in an embodiment of the present invention;

[0036] Figure 9 This is a flowchart of a data processing circuit control method provided in an embodiment of the present invention;

[0037] Figure 10 This is a schematic diagram of the structure of a data processing system provided in an embodiment of the present invention. Detailed Implementation

[0038] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0039] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0040] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0041] Figure 1 This is a schematic diagram of a data processing method provided in related technologies, such as... Figure 1As shown, control module 010 controls row control module 020 to read data from the image sensor and sends each row of data to data storage module 030 for storage, followed by convolution calculation. When the convolution kernel is 3×3, control module 010 first controls row control module 020 to read the first row of data in the focal plane and send it to data storage module 030 for storage. Then, control module 010 sequentially controls row control module 020 to read the second and third rows of data in the focal plane and send them to data storage module 030 for storage. After reading and storing the first, second, and third rows of data, a convolution operation is performed on the data stored in data storage module 030. After convolution is completed, the next convolution is performed, that is, the second, third, and fourth rows of data are read, stored, and convolved in the same way. This process continues until all data read by the image sensor in the entire focal plane has been processed.

[0042] Figure 1 In the data processing method shown, on the one hand, the control module and the row control module need to read and store data line by line, and then perform the corresponding convolution after storage. Because only one line of data can be read at a time, the parallelism is small, which limits the calculation speed and makes it impossible to achieve the required performance level in applications requiring high throughput. On the other hand, frequent data transfer is required when reading and storing data line by line, which leads to increased bus bandwidth and data transfer latency, and also increases the system's demand for local memory, especially in high-resolution image processing, where the capacity requirement for the row buffer is large.

[0043] Figure 2 This is a schematic diagram of another data processing method provided in related technologies, such as... Figure 2As shown, 3×3 convolution kernels are tiled onto the focal plane, resulting in 9 regions. These include 3×3 regions W1, W2, W4, and W5; 3×1 regions W3 and W6; 1×3 regions W7 and W8; and 1×1 region W9. Regions W3, W6, W7, W8, and W9 are incompatible with the kernel size, making it impossible to place a complete kernel and rendering their calculations invalid. Only regions W1, W2, W4, and W5 are actually capable of convolution. During convolution, the corresponding regions W1, W2, W4, and W5 are convolved, and after each convolution operation, the kernel is moved one step to the right. When all kernels return to their initial positions in the same row, the kernel is moved one step down. Repeat the above steps until all convolutional kernels have traversed the entire image, then summarize all convolutional results to obtain the complete convolutional image. Finally, return to the initial state by moving to the right and down several times.

[0044] Figure 2 In the data processing method shown, while this technique can guarantee a high degree of computational parallelism in convolution calculations, it is also intuitively apparent that due to the mismatch between the convolution kernel and the image size, the distribution of the convolution kernel on the focal plane will be irregular. This irregular distribution means that a convolution kernel cannot be completely placed in the edge regions of the image, thus rendering the calculations at these locations invalid. Therefore, some redundant convolution operations are introduced, wasting energy and time.

[0045] Figure 3 This is a flowchart of a data processing method provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the method includes:

[0046] S101, replicates each pixel on the focal plane. There are 1 processing unit; wherein the width and height of the convolution kernel are both n, and n≥1; the convolution stride is s; wherein s≥1;

[0047] Specifically, n represents the width and height of the convolution kernel. During convolution on the focal plane, multi-channel convolution is performed to utilize multiple identical input values ​​(i.e., the values ​​of pixels on the focal plane) with different weight values ​​(i.e., the corresponding weight values ​​in the convolution kernel) to perform different convolution calculations, resulting in multiple convolution results. When the width and height of the convolution kernel are 3 and the convolution stride is 1, a pixel on the focal plane is multiplied by a maximum of 3 different weight values ​​in one convolution operation. Therefore, in step S101, for each pixel on the focal plane, a copy is performed... Each processing unit is assigned a different weight value.

[0048] Figure 4 This is a schematic diagram of a data processing method provided in an embodiment of the present invention, as shown below. Figure 4 As shown, data processing is performed on a focal plane 201 with 7 pixel columns, where the size of the convolution kernel 202 is 3×3 and the convolution stride is 1. Three processing units 203 are copied to each pixel.

[0049] For example, three processing units are copied for pixel 02 in the first row and third column, including: PE02-1, PE02-2, and PE02-3, which are the first processing unit for pixel 02 in the first row and third column.

[0050] S102, In a single convolution operation, each processing unit in a pixel is multiplied by the weight of the corresponding convolution kernel to obtain the first convolution result;

[0051] Specifically, it copies the data from the pixels. After each processing unit, the weights of each processing unit and the corresponding convolution kernel are multiplied to obtain the first convolution result.

[0052] For example, such as Figure 4 As mentioned above, for the pixels in the first two columns and the last two columns of the focal plane 201, since they are not multiplied with all the weight values ​​of the convolution kernel during the convolution operation, these pixels are not multiplied with all the weight values ​​of the corresponding row of the convolution kernel.

[0053] For pixel 00 in the first row and first column of focal plane 201, it is multiplied only by the weight value w00 in the first row and first column of convolution kernel 202. Therefore, the first processing unit PE00-1 of pixel 00 in the first row and first column only needs to be multiplied by the weight value w00 in the first row and first column of convolution kernel 202. For pixel 05 in the first row and sixth column of focal plane 201, it is multiplied only by the weight values ​​w01 and w02 in the first row and third column of convolution kernel 202. Therefore, the second processing unit PE05-2 of pixel 05 in the first row and sixth column multiplies by the weight value w01 in the first row and second column; the third processing unit PE05-3 of pixel 05 in the first row and sixth column multiplies by the weight value w02 in the first row and third column.

[0054] When the column number j3 of the pixel on the focal plane is n ≤ j3 ≤ j - n + 1, the pixel is located between the first n columns and the last n columns of the focal plane. Then, the pixel is multiplied by the three weight values ​​of the corresponding row of the convolution kernel. For example, as shown... Figure 4Pixel 12 in the 2nd row and 3rd column on the focal plane 201: Pixel 12 in the 2nd row and 3rd column is multiplied by the weight values ​​w10, w11, and w12 in the corresponding row of the convolution kernel. Therefore, the first processing unit PE12-1 of pixel 12 in the 2nd row and 3rd column is multiplied by the weight value w10 in the 2nd row and 1st column, the second processing unit PE12-2 of pixel 12 in the 2nd row and 3rd column is multiplied by the weight value w11 in the 2nd row and 2nd column, and the third processing unit PE12-3 of pixel 12 in the 2nd row and 3rd column is multiplied by the weight value w12 in the 2nd row and 3rd column. The corresponding product is the first convolution result.

[0055] S103, accumulate the first convolution results of the processing units corresponding to the same convolution kernel on the same row to obtain the second convolution result;

[0056] like Figure 4 As shown, for the processing unit PE00-1 in the first row and first column of the processing unit group 204, the weight value w00 in the first row and first column is multiplied; the weight value w01-2 in the first row and second column of the processing unit PE01-2 is multiplied with the weight value w01 in the first row and second column of the processing unit PE02-3 in the first row and third column of the processing unit group 204 is multiplied with the weight value w02 in the first row and third column of the processing unit group 204. Therefore, the three first convolution results are obtained by performing convolution operations on the processing units corresponding to the same convolution kernel in the same row. The three first convolution results are accumulated to obtain the second convolution result. Similarly, the first processing unit PE10-1 of the second row and first column of the pixel is multiplied by the weight value w10 of the second row and first column, the second processing unit PE11-2 of the second row and second column of the pixel is multiplied by the weight value w11 of the second row and second column of the pixel, and the third processing unit PE12-3 of the second row and third column of the pixel is multiplied by the weight value w12 of the second row and third column of the pixel. These three first convolution results are obtained by performing convolution operations on the processing units corresponding to the same convolution kernel in the same row. The second convolution result is obtained by accumulating these three first convolution results.

[0057] S104, sum the second convolution results between rows and output the final convolution result;

[0058] Continue to refer to Figure 4Within the same processing unit group 204, the first processing unit PE00-1, the second processing unit PE01-2, and the third processing unit PE02-3 of the first row and first column of pixels undergo convolution to obtain a second convolution result. Similarly, the first processing unit PE10-1, the second processing unit PE11-2, and the third processing unit PE12-3 of the second row and third column of pixels undergo convolution to obtain another second convolution result. Finally, the first processing unit PE20-1, the second processing unit PE21-2, and the third processing unit PE22-3 of the third row and third column of pixels undergo convolution to obtain yet another second convolution result. The pixel column numbers corresponding to these three second convolution results are all composed of columns 1, 2, and 3. These three second convolution results are then summed to output the final convolution result.

[0059] Figure 5 This is a schematic diagram of another data processing method provided in an embodiment of the present invention, such as... Figure 5 As shown, in conjunction with the above embodiment, when the number of columns of the focal plane 201 is 7, the convolution kernel 202 is 3×3, and the convolution stride is 1, during the first convolution operation, rows 1 to 3 are the first convolution region 211, and rows 4 to 6 are the second convolution region 212. Corresponding convolution operations are performed on the first convolution region 211 and the second convolution region 212. When the convolution kernel size is 3×3 and the convolution stride is 1, the first convolution region 211 has 5 pixel groups for convolution calculation; the second convolution region 212 also obtains 5 pixel groups for convolution calculation.

[0060] After the first convolution operation is completed, the convolution kernel moves down by one stride, and the second convolution operation begins. In the second convolution operation, the focal plane 201 can be divided into two convolution regions: rows 2 to 4 form the third convolution region 213, and rows 5 to 7 form the fourth convolution region 214. Convolution operations are performed in the third and fourth convolution regions 214.

[0061] After the second convolution operation, the convolution kernel moves down by one stride, and the third convolution operation begins. In the third convolution operation, the focal plane can be divided into three parts: rows 1-2 form the first part, rows 3-5 form the second part, and rows 6-7 form the third part. The convolution operation is performed in the second part.

[0062] After three convolution operations are completed, all the convolution results on the focal plane can be obtained.

[0063] Specifically, taking the first convolution operation as an example, in a data processing method provided in this embodiment of the invention, during the convolution operation, convolution operations can be performed simultaneously on each part that needs to be convolved. In this convolution operation, the pixels in the first and last columns are assigned values ​​to the convolution kernel; that is, the pixels in the first column are multiplied by the weight values ​​in the first column of the convolution kernel, and the pixels in the last column are multiplied by the weight values ​​in the third column of the convolution kernel. For example, pixel 00 in the first row and first column is multiplied only by the weight value w00 in the first row and first column of the convolution kernel in the first pixel group 221.

[0064] The pixels in the second and penultimate columns are assigned two values ​​to the convolution kernel. Specifically, the pixels in the second column are multiplied twice by the weight values ​​in the first and second columns of the convolution kernel, and the pixels in the penultimate column are multiplied twice by the weight values ​​in the second and third columns of the convolution kernel. For example, pixel 01 in the first row and second column is multiplied by the weight value in the first row and second column of the convolution kernel w01 in the first pixel group 221, and multiplied by the weight value in the first row and first column of the convolution kernel w00 in the second pixel group 222.

[0065] The pixels in the remaining columns of the focal plane are multiplied by the weights of all columns of the convolution kernel, i.e., three multiplication operations are performed. For example, pixel 02 in the 1st row and 3rd column is multiplied by the weight w02 in the 1st row and 3rd column of the convolution kernel in the first pixel group 221, multiplied by the weight w01 in the 1st row and 2nd column of the convolution kernel in the second pixel group 222, and multiplied by the weight w00 in the 1st row and 1st column of the convolution kernel in the third pixel group 223.

[0066] In summary, as Figure 5As shown, when performing a convolution operation with a stride of 1 and a kernel size of 3×3 on the focal plane 201, three processing units are copied for each pixel on the focal plane 201. In one convolution operation, each processing unit in the pixel is multiplied by the weight of the corresponding convolution kernel to obtain the first convolution result. Specifically, the pixels in the first and last columns are multiplied by one weight value of the convolution kernel; the pixels in the second and second-to-last columns are multiplied by two weight values ​​of the convolution kernel; and the pixels in the remaining columns are multiplied by three weight values ​​of the convolution kernel. The first convolution results of the processing units corresponding to the same convolution kernel in the same row are summed to obtain the second convolution result. When three first convolution results are obtained from convolution operations performed on processing units corresponding to the same convolution kernel in the same row, these three first convolution results are summed to obtain the second convolution result. The second convolution results between rows are summed to output the final convolution result. In the second convolution results between different rows, if the number of columns of the three pixels corresponding to the three second convolution results of a pixel is the same, these three second convolution results are accumulated to obtain the final convolution result. This ultimately achieves data processing for the focal plane 201.

[0067] In the data processing method provided by this invention, pixels in the focal plane are copied. For each processing unit, based on its position, it is multiplied by the weight values ​​in different convolution kernels to obtain the final convolution result. Specifically, the product of processing units in the same row of each convolution kernel with their weight values ​​is the first convolution result; the first convolution results between the same rows of the same convolution kernel are summed to obtain the second convolution result; and the second convolution results between different rows are summed to obtain the final convolution result. This achieves multiple convolution results by using multiple identical input values ​​with different weight values ​​to perform different convolution calculations, thus improving parallelism and running speed.

[0068] In conjunction with the above embodiments, after one convolution calculation, the convolution kernel region is moved down by one step for the next convolution operation until the final convolution result on the entire focal plane is obtained; wherein, the height of the convolution region is equal to the height of the convolution kernel, the width of the convolution region is equal to the width of the focal plane, and adjacent convolution regions are adjacent along the height direction of the focal plane.

[0069] Specifically, such as Figure 5 As shown, during the first convolution operation, the focal plane has two convolution regions: the first convolution region 211 is in rows 1 to 3 of the focal plane, and the second convolution region 212 is in rows 4 to 6 of the focal plane.

[0070] After the convolution operation is completed in the two convolution regions, the convolution kernel moves down by one stride. At this time, the second convolution operation is performed. Rows 2 to 4 of the focal plane are the third convolution region 213, and rows 5 to 7 of the focal plane are the fourth convolution region 214.

[0071] In conjunction with the above embodiments, such as Figure 3 The step S102 shown includes: when the focal plane is column j and s=1, in each convolution region, for the pixel in the j1th column and n1th row, the h1th bit processing unit of the pixel is multiplied by the weight in the n1th row and h1th column of the convolution kernel, where j1<n, h1≤j1; where n1 is the row number of the pixel in the convolution region, n1≤n.

[0072] For details, please refer to [link / reference]. Figure 4 Taking a convolution kernel size of 3×3, a convolution stride of 1, and a focal plane column number of 7 as an example, i.e., n=3 and j=7, j1<3 and h1≤j1. At this time, j1=1 or 2; when j1=1, h1=1, and when j1=2, h1=1 or 2.

[0073] When j1=1 and n1=1, h1=1. The first processing unit of pixel 00 in the first row and first column is multiplied by the weight in the first row and first column of the convolution kernel, that is, the first processing unit of the pixel is multiplied by the weight value w00 in the first row and first column of the convolution kernel to obtain the first convolution result.

[0074] When j1=2 and n1=2, h1=1 or 2. Then, the second processing unit of pixel 11 is multiplied by the weight in the first row and second column of the convolution kernel, that is, the first processing unit of the pixel is multiplied by the weight value w01 in the first row and second column of the convolution kernel to obtain the first convolution result. Alternatively, the first processing unit of the pixel is multiplied by the weight in the first row and first column of the convolution kernel, that is, the first processing unit of the pixel is multiplied by the weight value w00 in the first row and first column of the convolution kernel to obtain the first convolution result.

[0075] The method for obtaining the first convolution result for the remaining pixels in column j1 and row n1 is the same as in the above embodiment, and will not be repeated here.

[0076] In each convolutional region, for the pixel in the j2th column and n1th row, the weights in the n1st row and h2th column of the h2th bit processing unit convolutional kernel of the pixel are multiplied together, where j - n + 1 < j2 ≤ j, nj - j2 ≤ h2 ≤ n;

[0077] For details, please refer to [link / reference]. Figure 4 That is, n=3 and j=7. At this time, 5 < j2 ≤ 7. When j2=6, h2=2 or 3; when j2=7, h2=3.

[0078] When j2=6 and n1=2, h2=2 or 3. When h2=2, the second processing unit of pixel 15 in the second row and sixth column is multiplied by the weight in the second row and second column of the convolution kernel, that is, the second processing unit of pixel 15 in the second row and sixth column is multiplied by the weight value w11 in the second row and second column of the convolution kernel to obtain the first convolution result. When h2=3, the third processing unit of pixel 15 in the second row and sixth column is multiplied by the weight in the second row and third column of the convolution kernel, that is, the third processing unit of pixel 15 in the second row and sixth column is multiplied by the weight value w12 in the second row and third column of the convolution kernel to obtain the first convolution result.

[0079] When j2=7 and n1=1, h2=3. Multiply the 3rd processing unit of pixel 06 in the 1st row and 7th column by the weight in the 1st row and 3rd column of the convolution kernel, that is, multiply the 3rd processing unit of the pixel by the weight value w02 in the 1st row and 3rd column of the convolution kernel to obtain the first convolution result.

[0080] The method for obtaining the first convolution result for the remaining pixels in the j2th column and n1st row is the same as in the above embodiment, and will not be repeated here.

[0081] In each convolution region, for the pixel in the j3rd column and n1st row, the h3rd bit processing unit of the pixel is multiplied by the weight in the n1st row and h3rd column of the convolution kernel to obtain the first convolution result; where n ≤ j3 ≤ j - n + 1, h3 ≤ n;

[0082] For details, please refer to [link / reference]. Figure 4 That is, n=3, j=7. At this time, 3 ≤ j3 ≤5, h3≤3.

[0083] When j3=3 and n1=2, h2=1, 2, or 3.

[0084] When h2=1, the first processing unit of pixel 12 in the second row and third column is multiplied by the weight in the second row and first column of the convolution kernel, that is, the first processing unit of pixel 12 in the second row and third column is multiplied by the weight value w10 in the second row and first column of the convolution kernel to obtain the first convolution result.

[0085] When h2=2, the second processing unit of pixel 12 in the second row and third column is multiplied by the weight in the second row and second column of the convolution kernel, that is, the second processing unit of pixel 12 in the second row and third column is multiplied by the weight value w11 in the second row and second column of the convolution kernel to obtain the first convolution result.

[0086] When h2=3, the third processing unit of pixel 12 in the second row and third column is multiplied by the weight in the second row and third column of the convolution kernel, that is, the third processing unit of pixel 12 in the second row and third column is multiplied by the weight value w12 in the second row and third column of the convolution kernel to obtain the first convolution result.

[0087] The method for obtaining the first convolution result for the remaining pixels in the j3rd column and n1st row is the same as in the above embodiment, and will not be repeated here.

[0088] In conjunction with the above embodiments, such as Figure 3 The step S103 shown includes: adding the first convolution result corresponding to the first processing unit of the pixel in the j4th column of each row of the focal plane to the first convolution result corresponding to the n2nd processing unit of the n-1 adjacent pixels in the j5th column on the right, to obtain the second convolution result, where j4<j5≤n+j4-1; n2=j5-j4+1.

[0089] For example, in combination Figure 4 , 5 As in the above embodiment, taking a convolution kernel size of 3×3 and a focal plane column number of 7 as an example, i.e. n=3 and j=7.

[0090] In the first row of the focal plane, the processing unit PE00-1 for pixel 00 in the first row and first column is its first processing unit. The (n-1)th pixels in the j5th column adjacent to pixel 00 in the first row and first column, namely pixel 01 in the second row and first column and pixel 02 in the third row and first column, i.e., 1 < j5 ≤ 3. The first convolution result corresponding to the first processing unit PE00-1 for pixel 00 in the first row and first column is added to the second processing unit PE01-2 for pixel 01 in the second row and first column and the third processing unit PE01-3 for pixel 02 in the third row and first column to obtain the second convolution result.

[0091] In conjunction with the above embodiments, such as Figure 3 The step S104 shown includes: in the column direction of each convolution region, when the number of columns of the n pixels corresponding to each first convolution result is the same in the n second convolution results, the n second convolution results are added together to output the final convolution result.

[0092] Specifically, in combination Figure 4Similar to the above embodiments, there are the following three second convolution results: the second convolution result obtained by the first processing unit PE00-1 of pixel 00 in the first row and first column, the second processing unit PE01-2 of pixel 01 in the first row and second column, and the third processing unit PE02-3 of pixel 02 in the first row and third column; the second convolution result obtained by the first processing unit PE10-1 of pixel 00 in the second row and first column, the second processing unit PE11-2 of pixel 11 in the second row and second column, and the third processing unit PE12-3 of pixel 12 in the second row and third column; and the second convolution result obtained by the first processing unit PE20-1 of pixel 20 in the third row and first column, the second processing unit PE21-2 of pixel 21 in the third row and second column, and the third processing unit PE22-3 of pixel 22 in the third row and third column. As can be seen from the three second convolution results in the above embodiments, they are respectively the second convolution results obtained from pixels 00, 01, and 02 in the first row and first column; pixels 10, 11, and 12 in the second row and second column; and pixels 20, 21, and 22 in the third row and first column. The number of columns for the three pixels corresponding to these three second convolution results are 1, 2, and 3 respectively. Therefore, these three second convolution results are added together to output the final convolution result.

[0093] Figure 6 This is a schematic diagram of a data processing circuit structure provided in an embodiment of the present invention, such as... Figure 6 As shown, a focal plane module 301 and a pulse width generator module 302 are connected. The pulse width generator module 302 is used to output a pulse width modulation signal representing the weights of different convolution kernels to the focal plane. The focal plane module 301 includes multiple pixel circuits 303. The multiple pixel circuits 303 of the focal plane module 301 are used to multiply n identical input values ​​with the corresponding pulse width modulation signals to obtain a first convolution result; to accumulate the first convolution results in the pixel circuits 303 corresponding to the same convolution kernel in the same row to obtain a second convolution result; and to accumulate the second convolution results between rows to output the final convolution result.

[0094] Specifically, the pulse width generator module 302 generates a series of pulse width modulation (PWM) signals based on preset convolution kernel weight values. Optionally, the pulse width of the PWM signal is set according to the value of each weight in the corresponding convolution kernel; the larger the weight, the wider the pulse width; the smaller the weight, the narrower the pulse width.

[0095] After receiving the PWM signal from the pulse width generator module 302, the focal plane module 301 multiplies these signals by n identical input values. Specifically, the input values ​​are the photocurrent values ​​acquired by the focal plane module 301. In the multiple pixel circuits 303 of the focal plane module 301, each pixel circuit 303 includes a photoelectric conversion element. The input value is the light intensity value of the pixel to be convolved. The photoelectric conversion element acquires the light intensity of the pixel to be convolved, and different photocurrent magnitudes are obtained according to different light intensities. In the pixel circuit, different photocurrent magnitudes result in different discharge currents when the energy storage element discharges. Optionally, the photoelectric conversion element is a photodiode, and the energy storage element is a capacitor.

[0096] The "multiplication" process uses a PWM signal to control a switching circuit, allowing the input value to pass through (or accumulate in some way) when the PWM signal is high, and ignoring or attenuating it when it is low. Therefore, each processing unit generates a "weighted" value proportional to the weights based on the corresponding PWM signal; this is the first convolution result.

[0097] Optionally, the pixel circuit 303 in the focal plane module 301 can be controlled by a PWM signal. The pixel circuit 303 converts the input value into a discharge, and the PWM signal controls the discharge time. The longer the pulse width of the PWM signal, the longer the discharge time of the controlled pixel circuit 303; conversely, the shorter the pulse width. Therefore, the weighted calculation of the input value and the convolution kernel is transformed into controlling the discharge time of the pixel circuit 303.

[0098] The convolution operation of the input value in the focal plane module 301 is transformed into the conduction time of the pixel circuit 303 controlled by the pulse width of the input value and the pulse width modulation signal.

[0099] On the same row, the first convolution results generated by all processing units are summed according to their respective convolution kernels to obtain the second convolution result for that row. This achieves weighted summation calculation within the horizontal sliding window of the convolution kernel in the convolution operation.

[0100] Within different rows, the results of the second convolution between rows are summed to obtain the final output of the entire convolution operation. Specifically, based on the position of the pixel corresponding to the second convolution result between rows, the corresponding second convolution kernels are added together to obtain the final convolution result.

[0101] Optionally, the sum of the first convolution result and the second convolution result can be the series connection of the output signals of the pixel circuit 303 in the corresponding focal plane module 301.

[0102] This invention provides a data processing circuit in which a pulse width generator module outputs a pulse width modulation (PWM) signal representing different convolution kernel weight values ​​to a focal plane, performing weighted calculations on the focal plane input values ​​to obtain a first convolution result. Then, the corresponding convolution results between the same row and different rows are accumulated to obtain the final convolution result. This achieves high-precision weight representation through fine control of the PWM signal, thus maintaining the accuracy of the convolution operation. Furthermore, it can simultaneously control the PWM signal of multiple pixel circuits on the focal plane to achieve global parallelism and multi-bit convolution calculation, improving parallelism and operating speed. It avoids complex analog signal processing, improves response speed, and reduces latency. Using the PWM signal to control the switch discharge time to achieve multiplication helps reduce quantization errors and improves the accuracy of convolution calculation.

[0103] Figure 7 A pixel circuit diagram of a focal plane module provided in an embodiment of the present invention, such as... Figure 7 As shown, the focal plane module 301 includes multiple pixel circuits 303, each pixel circuit 303 includes a control terminal PWM_IN, and the output terminal of the pulse width generator module 302 includes multiple sub-output terminals, which are connected one-to-one with the control terminal PWM_IN of the pixel circuit 303 of the focal plane.

[0104] The pulse width generator module 302 generates a pulse width modulation signal related to the convolution kernel weight values. The generated pulse width modulation signal is output through the sub-output terminal of the pulse width generator module 302 and sent to the control terminal PWM_IN of the pixel circuit 303. The pixel circuit 303 performs corresponding data processing based on the received pulse width modulation signal.

[0105] Continue to refer to Figure 7 The focal plane module pixel circuit also includes: a P-type field-effect transistor PMOS1, an N-type field-effect transistor NMOS1, a photodiode PD1, and a first capacitor C1; wherein the first terminal of the P-type field-effect transistor PMOS1, the first terminal of the N-type field-effect transistor NMOS1, and the negative terminal of the photodiode PD1 are connected; the second terminal of the N-type field-effect transistor NMOS1 is connected to the first power supply, the second terminal of the P-type field-effect transistor PMOS1 is connected to the first terminal of the first capacitor C1 and the pixel output terminal, the positive terminal of the photodiode PD1 is grounded, and the second terminal of the first capacitor C1 is grounded.

[0106] Specifically, photodiode PD1 is used to capture optical signals and convert them into electrical signals as input values ​​for subsequent convolution calculations. The control terminal of N-type field-effect transistor NMOS1 can be used as a reset terminal (REST) ​​to receive reset signals sent by the control module.

[0107] During initialization, the pulse width generator module 302 sends a low-level signal, at which time the P-type field-effect transistor PMOS1 is in the conducting state. When the reset terminal REST receives a high-level signal, the N-type field-effect transistor NMOS1 is turned on. At this time, the photodiode PD1 simultaneously charges the first capacitor C1 through the power supply terminal VDD, and the photodiode PD1 is reverse biased.

[0108] After a certain period of time, the first capacitor C1 is fully charged. A low-level signal is then sent to the reset terminal REST, turning off the N-type field-effect transistor NMOS1. The convolution operation begins, and the control terminal PWM_IN receives the PWM signal sent by the pulse width generator module 302. Each pixel circuit 303 is turned on for a corresponding duration according to the pulse width of the corresponding PWM signal. The capacitor discharges by multiplying the photocurrent and the discharge time of the P-type field-effect transistor PMOS1. The multiplied amount is the photocurrent magnitude and the PMOS1 on-time. The photocurrent, through its interaction with the discharge time, yields different discharge values, ultimately leaving different amounts of charge on the capacitor. This process ultimately multiplies the input value in each pixel circuit 303 with the corresponding pulse width modulation signal to obtain the first convolution result.

[0109] Continue to refer to Figure 7 In the focal plane module 301, a first switch is connected in series between the output terminals OUT of the circuits 303 of two adjacent pixels in the same row.

[0110] The output terminal OUT of each column of pixel circuit 303 is connected, and a second switch is connected in series between the output terminals OUT of two adjacent pixel circuits 303 in the same column.

[0111] During initialization, all first and second switches are turned off to charge the first capacitor C1. After the convolution operation is completed, i.e., after the first convolution result is obtained in all pixel circuits 303, the first convolution result of the pixel circuits 303 corresponding to the same convolution kernel in the same row is accumulated by controlling the first switches to obtain the second convolution result.

[0112] By controlling the second switch to turn it on and off, the second convolution results between rows are accumulated, and the final convolution result is output.

[0113] Figure 8 This is a schematic diagram of another data processing circuit structure provided in an embodiment of the present invention. The data processing circuit further includes a controller module 304; the pulse width generator module 302 includes a weighted register unit 3021 and a pulse generation unit 3022.

[0114] The controller module 304 is connected to the weighting register unit 3021, the pulse generation unit, and the focal plane module 301. The weighting register unit 3021 is connected to the pulse generation unit 3022, and the pulse generation unit is connected to the focal plane module 301.

[0115] Specifically, the controller controls the weighting register unit 3021 to generate different multi-bit weights based on the weight values ​​of the convolution kernel and sends them to the pulse generation unit 3022; the pulse generation unit 3022 generates a corresponding pulse width modulation signal based on the received multi-bit weight values. The corresponding pulse width modulation signal is then sent to the pixel circuit 303 of the focal plane module 301 for corresponding data processing operations.

[0116] Optionally, the pulse generation unit 3022 receives multiple weight values, each weight value representing one bit. The pulse width generator module 302 also includes a clock signal generation unit for synchronizing the generation of the pulse width modulation signal.

[0117] Figure 9 This is a flowchart of a data processing circuit control method provided in an embodiment of the present invention, as shown below. Figure 9 As shown, the data processing circuit further includes a controller module; the pulse width generator module includes a weighted register unit and a pulse generation unit, and the method includes:

[0118] S201, the controller module controls the adjacent... Each pixel circuit obtains the value of the same pixel.

[0119] Where n is the size of the convolution kernel and s is the stride of the convolution. When performing convolution on the focal plane, multi-channel convolution operations are performed to utilize multiple identical input values ​​(i.e., the values ​​of pixels on the focal plane) with different weight values ​​(i.e., the corresponding weight values ​​in the convolution kernel) to perform different convolution calculations, resulting in multiple convolution results. Adjacent... Each pixel circuit acquires the value of the same pixel to enable multiple convolution calculations to be performed simultaneously on the same pixel.

[0120] S202, In a single convolution operation, the controller module controls the weighted register unit to generate weight values ​​based on the convolution kernel and send them to the pulse width generator module;

[0121] Specifically, in conjunction with the above embodiments, in the weighting register unit, corresponding multi-bit weight values ​​are generated based on the convolution kernel, and these weight values ​​are sent to the pulse generation unit. The number of bits in the multi-bit weight values ​​is selected based on actual circumstances and is not limited here.

[0122] S203, the controller module controls the pulse generation unit to generate a pulse width modulation signal according to the received weight value and send it to the circuit of each pixel point on the focal plane at the same time;

[0123] The pulse generation unit generates a pulse width modulation signal with a corresponding pulse width based on the received multi-bit weight value. Optionally, the larger the weight value, the larger the pulse width of the generated pulse width modulation signal.

[0124] The generated pulse width modulation signal is sent to the circuitry of each pixel on the focal plane at the same time.

[0125] Specifically, as in the above embodiments, in a data processing method provided by the present invention, the value of a pixel is copied to multiple processing units, and multiple processing units with the same value are multiplied by the weight values ​​of different convolution kernels. Similarly, in a control method for a data processing circuit provided by an embodiment of the present invention, after generating pulse width modulation signals corresponding to different weight values ​​in different convolution kernels, the corresponding pulse width modulation signals are sent to each pixel circuit of the focal plane at the same time.

[0126] For example, for Figure 4 For pixel 03 in the 1st row and 4th column shown, the data processing method replicates pixel 03 in the 1st row and 4th column into three processing units: PE03-1 (1st processing unit), PE03-2 (2nd processing unit), and PE03-3 (3rd processing unit). Then, PE03-1 is multiplied by the weight value w00 in the 1st row and 1st column of the convolution kernel; PE03-2 is multiplied by the weight value w01 in the 1st row and 2nd column of the convolution kernel; and PE03-3 is multiplied by the weight value w02 in the 1st row and 3rd column of the convolution kernel. In the control method of the data processing circuit provided in this embodiment of the invention, for pixel 03 in the first row and fourth column, the pulse generation unit can send the PWM signal corresponding to the weight value w00 in the first row and first column of the convolution kernel to the first pixel circuit that has obtained the value of pixel 03 in the first row and fourth column; the pulse generation unit can send the PWM signal corresponding to the weight value w01 in the first row and second column of the convolution kernel to the second pixel circuit that has obtained the value of pixel 03; the pulse generation unit can send the PWM signal corresponding to the weight value w02 in the first row and third column of the convolution kernel to the third pixel circuit that has obtained the value of pixel 03 in the first row and fourth column. These three transmissions of the PWM signal are performed simultaneously, and the first, second, and third pixel circuits are adjacent.

[0127] Optional, combined Figure 4 As shown, in each convolution region, adjacent pixel circuits that copy the same pixel value are called a pixel circuit unit.

[0128] When processing data in the convolution region of the first to third rows of pixels on the focal plane to be convolved, for the pixel circuit units that acquire the values ​​of pixel 00 in the first row and first column and pixel 03 in the first row and fourth column, the pulse width modulation signal received by the first pixel circuit of each pixel circuit unit is generated based on the weight value w00 in the first row and first column of the convolution kernel. The remaining pixels receive the pulse width modulation signal sent by the pulse generation unit. For the pixel circuit units that acquire the values ​​of pixel 01 in the first row and second column and pixel 04 in the first row and fifth column, the pulse width modulation signal received by the second pixel circuit of each pixel circuit unit is generated based on the weight value w01 in the first row and second column of the convolution kernel; for the pixel circuit units that acquire the values ​​of pixel 02 in the first row and third column and pixel 05 in the first row and sixth column, the pulse width modulation signal received by the third pixel circuit of each pixel circuit unit is generated based on the weight value w02 in the first row and third column of the convolution kernel.

[0129] When processing data in the convolution region of pixels in the first to third rows of the focal plane, for the pixel circuit units that acquire the values ​​of pixel 01 in the first row and second column and pixel 04 in the first row and fifth column, the first pixel circuit of each pixel circuit unit receives a pulse width modulation signal sent by the pulse generation unit. The pulse width modulation signal is generated based on the weight value w00 in the first row and first column of the convolution kernel. The remaining pixels also receive the pulse width modulation signal sent by the pulse generation unit. The pixel circuit units that acquire the pixel value 02 in the 1st row and 3rd column and the pixel value 05 in the 1st row and 6th column have a pulse width modulation signal received by the second pixel circuit of each pixel circuit unit, which is generated based on the weight value w01 in the 1st row and 2nd column of the convolution kernel; the pixel circuit units that acquire the pixel value 03 in the 1st row and 4th column and the pixel value 06 in the 1st row and 7th column have a pulse width modulation signal received by the third pixel circuit of each pixel circuit unit, which is generated based on the weight value w02 in the 1st row and 3rd column of the convolution kernel.

[0130] When processing data in the convolution region of pixels in the first to third rows of the focal plane, for the pixel circuit unit that obtains the value of pixel 02 in the 1st row and 3rd column, the first pixel circuit of each pixel circuit unit receives a pulse width modulation signal sent by the pulse generation unit. The pulse width modulation signal is generated based on the weight value w00 in the 1st row and 1st column of the convolution kernel. The remaining pixels receive the pulse width modulation signal sent by the pulse generation unit. For the pixel circuit unit that obtains the value of pixel 03 in the 1st row and 4th column, the second pixel circuit of each pixel circuit unit receives a pulse width modulation signal generated based on the weight value w01 in the 1st row and 2nd column of the convolution kernel; for the pixel circuit unit that obtains the value of pixel 04 in the 1st row and 5th column, the third pixel circuit of each pixel circuit unit receives a pulse width modulation signal generated based on the weight value w02 in the 1st row and 3rd column of the convolution kernel.

[0131] S204, The controller module controls the accumulation of the output results of the pixel circuits in the focal plane according to the time when each pixel circuit in the focal plane receives the pulse width modulation signal;

[0132] Specifically, such as Figure 4 As described in the above embodiments, in the data processing method provided by the present invention, the first convolution results corresponding to processing units of corresponding bit lengths in the same row are added, and the corresponding second convolution results of adjacent columns in the same convolution region are added. Correspondingly, in the control method of the data processing circuit provided in the embodiments of the present invention, the controller module controls the accumulation of the output results of the pixel circuit in the focal plane according to the time when each pixel receives the pulse width modulation signal.

[0133] For example, in conjunction with the above embodiments and Figure 4As shown, when processing data in the convolution region of the first to third rows of pixels on the focal plane to be convolved, for the pixel circuit units that obtain the values ​​of pixel 00 in the first row and first column and pixel 03 in the first row and fourth column, the pulse width modulation signal received by the first pixel circuit of each pixel circuit unit is generated according to the weight value w00 in the first row and first column of the convolution kernel. The remaining pixels receive the pulse width modulation signal sent by the pulse generation unit. For the pixel circuit units that obtain the values ​​of pixel 01 in the first row and second column and pixel 04 in the first row and fifth column, the pulse width modulation signal received by the second pixel circuit of each pixel circuit unit is generated according to the weight value w01 in the first row and second column of the convolution kernel; for the pixel circuit units that obtain the values ​​of pixel 02 in the first row and third column and pixel 05 in the first row and sixth column, the pulse width modulation signal received by the third pixel circuit of each pixel circuit unit is generated according to the weight value w02 in the first row and third column of the convolution kernel.

[0134] Then, in the row direction, the outputs of the first pixel circuit in the pixel circuit unit corresponding to the value of pixel 00 in the first row and first column, the second pixel circuit in the pixel circuit unit corresponding to the value of pixel 01 in the first row and second column, and the third pixel circuit in the pixel circuit unit corresponding to the value of pixel 02 in the first row and third column are added together.

[0135] The outputs of the pixel circuits corresponding to the first pixel circuit in the pixel circuit unit corresponding to pixel value 03 in the 1st row and 4th column, the second pixel circuit in the pixel circuit unit corresponding to pixel value 04 in the 1st row and 5th column, and the third pixel circuit in the pixel circuit unit corresponding to pixel value 05 in the 1st row and 6th column are added together.

[0136] In the row direction, for pixel circuit units that acquire the values ​​of pixel 10 in the 2nd row, 1st column and pixel 13 in the 2nd row, 4th column, the first pixel circuit of each pixel circuit unit receives a pulse width modulation signal sent by the pulse generation unit. The pulse width modulation signal is generated based on the weight value w10 in the 2nd row, 1st column of the convolution kernel. For pixel circuit units that acquire the values ​​of pixel 11 in the 2nd row, 2nd column and pixel 14 in the 2nd row, the second pixel circuit of each pixel circuit unit receives a pulse width modulation signal generated based on the weight value w11 in the 2nd row, 2nd column of the convolution kernel; for pixel circuit units that acquire the values ​​of pixel 12 in the 2nd row, 3rd column and pixel 15 in the 2nd row, 6th column, the third pixel circuit of each pixel circuit unit receives a pulse width modulation signal generated based on the weight value w12 in the 2nd row, 3rd column of the convolution kernel. In the row direction, the outputs of the pixel circuits corresponding to the first pixel circuit of the pixel circuit unit that obtains the value of pixel 10 in the first column of the second row, the second pixel circuit of the pixel circuit unit that obtains the value of pixel 11 in the second column of the second row, and the third pixel circuit of the pixel circuit unit that obtains the value of pixel 12 in the third column of the second row are added together; the outputs of the pixel circuits corresponding to the first pixel circuit of the pixel circuit unit that obtains the value of pixel 13 in the fourth column of the second row, the second pixel circuit of the pixel circuit unit that obtains the value of pixel 14 in the fifth column of the second row, and the third pixel circuit of the pixel circuit unit that obtains the value of pixel 15 in the sixth column of the second row are added together.

[0137] Similarly, for the pixel circuit units that acquire the values ​​of pixel 20 in the 3rd row and 1st column and pixel 23 in the 3rd row and 4th column, the first pixel circuit of each pixel circuit unit receives a pulse width modulation signal sent by the pulse generation unit. The pulse width modulation signal is generated based on the weight value w20 in the 3rd row and 1st column of the convolution kernel. For the pixel circuit units that acquire the values ​​of pixel 21 in the 3rd row and 2nd column and pixel 24 in the 3rd row and 5th column, the pulse width modulation signal received by the second pixel circuit of each pixel circuit unit is generated based on the weight value w21 in the 3rd row and 2nd column of the convolution kernel; and for the pixel circuit units that acquire the values ​​of pixel 22 in the 3rd row and 3rd column and pixel 25 in the 3rd row and 6th column, the pulse width modulation signal received by the third pixel circuit of each pixel circuit unit is generated based on the weight value w22 in the 3rd row and 3rd column of the convolution kernel. In the row direction, the outputs of the pixel circuits corresponding to the first pixel circuit of the pixel circuit unit that obtains the value of pixel 20 in the 3rd row and 1st column, the second pixel circuit of the pixel circuit unit that obtains the value of pixel 21 in the 3rd row and 2nd column, and the third pixel circuit of the pixel circuit unit that obtains the value of pixel 22 in the 3rd row and 3rd column are added together; the outputs of the pixel circuits corresponding to the first pixel circuit of the pixel circuit unit that obtains the value of pixel 23 in the 3rd row and 4th column, the second pixel circuit of the pixel circuit unit that obtains the value of pixel 24 in the 3rd row and 5th column, and the third pixel circuit of the pixel circuit unit that obtains the value of pixel 25 in the 3rd row and 6th column are added together.

[0138] In the column direction, the result of adding the outputs of the pixel circuits corresponding to the first pixel circuit of the pixel circuit unit that obtains the value of pixel 00 in the first row and first column, the second pixel circuit of the pixel circuit unit that obtains the value of pixel 01 in the first row and second column, and the third pixel circuit of the pixel circuit unit that obtains the value of pixel 02 in the first row and third column, is added to the outputs of the pixel circuits corresponding to the first pixel circuit of the pixel circuit unit that obtains the value of pixel 10 in the second row and first column, the second pixel circuit of the pixel circuit unit that obtains the value of pixel 11 in the second row and second column, and the third pixel circuit of the pixel circuit unit that obtains the value of pixel 12 in the second row and third column, and the outputs of the pixel circuits corresponding to the first pixel circuit of the pixel circuit unit that obtains the value of pixel 20 in the third row and first column, the second pixel circuit of the pixel circuit unit that obtains the value of pixel 21 in the third row and second column, and the third pixel circuit of the pixel circuit unit that obtains the value of pixel 22 in the third row and third column, is added to the outputs of the pixel circuits corresponding to the first pixel circuit of the pixel circuit unit that obtains the value of pixel 20 in the third row and first column, the second pixel circuit of the pixel circuit unit that obtains the value of pixel 21 in the third row and second column, and the third pixel circuit of the pixel circuit unit that obtains the value of pixel 22 in the third row and third column. The output results of the circuits for each pixel in this convolution region are similar and will not be elaborated here.

[0139] The same principle applies when processing data in the convolution region of the fourth to sixth rows of pixels on the focal plane or other pixels at a certain moment, which will not be elaborated here.

[0140] S205, after the pixel circuit output results are accumulated, the controller module controls the output of the accumulated results.

[0141] After obtaining the value output by the corresponding pixel circuit through the steps of the above embodiments, the controller module controls the output of the accumulated result.

[0142] In the data processing circuit control method provided by this invention, the weighting register unit generates corresponding weight signals based on the weight values ​​of the convolution kernel. These weight signals represent the value of each element in the convolution kernel. The pulse generation unit receives the weight signals from the weighting register unit and converts these weight values ​​into pulse width modulation (PWM) signals. Each weight value corresponds to a pulse of a specific width, and the pulse width directly reflects the magnitude of the weight. These PWM signals are then sent to each pixel circuit on the focal plane. The controller module monitors the time when each pixel circuit receives the PWM signal and controls the accumulation of the pixel circuit's output based on these times. This process simulates the multiplication and accumulation steps in convolution operations, but replaces traditional numerical multiplication with PWM signals. Finally, the controller module collects and processes the output results from all pixel circuits, completing the control of the entire data processing circuit. This data processing circuit control method, by converting weight values ​​into PWM signals, improves the parallel processing capability of the data processing circuit and accelerates the convolution operation process. It increases the calculation speed, greatly improves convolution efficiency, and achieves global parallel operation.

[0143] In conjunction with the above embodiments, the data processing circuit control method further includes: before the pulse generation unit generates a pulse width modulation signal according to the received weight value and simultaneously sends it to each pixel circuit of the focal plane, the controller module controls the initialization of the pixel circuit;

[0144] Combination Figure 7 As shown, before the pulse generation unit sends the pulse width modulation signal to each pixel circuit on the focal plane, the pixel circuit needs to be initialized. Specifically, the control terminal of the N-type field-effect transistor NMOS1 in the pixel circuit can be used as a reset terminal REST to receive the reset signal sent by the control module. When the reset terminal REST receives a high-level signal, the N-type field-effect transistor NMOS1 is turned on. At this time, the photodiode PD1 simultaneously charges the first capacitor C1 through the power supply terminal VDD, and the photodiode PD1 is reverse biased.

[0145] After each convolution operation, the weight register unit generates weight values ​​based on the weights of the next row of the convolution kernel and sends them to the pulse width generator module.

[0146] Combining the above embodiments and Figure 4 As shown, in one convolution operation, the pixel circuits in the pixel circuit units corresponding to the pixels in rows 1 to 3 are controlled; simultaneously, the pixel circuits in the pixel circuit units corresponding to the pixels in rows 4 to 6 are also controlled. After this convolution operation, the weighting register unit generates weight values ​​based on the weights of the next row of the convolution kernel and sends them to the pulse width generator module. This initiates another convolution operation. At this time, the pixel circuits in the pixel circuit units corresponding to the pixels in rows 2 to 4 are controlled; and the pixel circuits in the pixel circuit units corresponding to the pixels in rows 5 to 7 are also controlled.

[0147] In conjunction with the above embodiments, steps S202 and S203 of the data processing circuit control method further include:

[0148] When the convolution stride is 1, the focal plane is in column j, the width and height of the convolution kernel are both n, and n≥3;

[0149] In each convolution region, adjacent pixel circuits that copy the same pixel value constitute a pixel circuit unit;

[0150] For the pixel circuit unit in the j1st column and n1st row, the weighting register unit generates an input weight value based on the weight in the n1st row and h1st column of the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the pixel circuit unit in the j1st column and n1st row, where j1 < n, h1 ≤ j1, n1 is the row number of the pixel circuit unit in the convolution region, and n1 ≤ n.

[0151] In each convolution region, for the pixel circuit unit in the j2th column and n1st row, the weighting register unit generates an input weight value based on the weight in the n1st row and h2th column of the convolution kernel, and the pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the pixel circuit unit in the j2th column and n1st row, where j - n + 1 < j2 ≤ j, nj - j2 ≤ h2 ≤ n;

[0152] In each convolution region, for the pixel in the j3rd column and n1st row, the weighting register unit generates an input weight value based on the weight in the n1st row and h3rd column of the convolution kernel, and the pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the pixel circuit unit in the j3rd column and n1st row, where n ≤ j3 ≤ j - n + 1, h3 ≤ n.

[0153] Specifically, with Figure 5 Taking the first convolution region 211 in rows 1 to 3 as an example,

[0154] The first pixel circuit in the circuit unit that obtains the pixel values ​​of the first and fourth columns of the first, second, and third rows performs weight multiplication. The weight register unit generates input weight values ​​based on the weights of the first column of the first, second, and third rows in the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight values ​​and sends it to the first pixel circuit in the circuit unit that obtains the pixel values ​​of the first and fourth columns of the first, second, and third rows.

[0155] Simultaneously, the second pixel circuit in the circuit unit that obtains the pixel values ​​of the 2nd and 5th columns of the 1st, 2nd, and 3rd rows performs weight multiplication. The weighted register unit generates an input weight value based on the weights of the 2nd column of the 1st, 2nd, and 3rd rows in the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the second pixel circuit in the circuit unit that obtains the pixel values ​​of the 2nd and 5th columns of the 1st, 2nd, and 3rd rows.

[0156] Simultaneously, the third pixel circuit in the circuit unit that obtains the pixel values ​​of the 3rd and 6th columns of the 1st, 2nd, and 3rd rows performs weight multiplication. The weighted register unit generates an input weight value based on the weights of the 3rd column of the 1st, 2nd, and 3rd rows in the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the third pixel circuit in the circuit unit that obtains the pixel values ​​of the 3rd and 6th columns of the 1st, 2nd, and 3rd rows.

[0157] Simultaneously, the first pixel circuit in the circuit unit that obtains the pixel values ​​of the 2nd and 5th columns of the 1st, 2nd, and 3rd rows performs weight multiplication. The weight register unit generates input weight values ​​based on the weights of the 1st column of the 1st, 2nd, and 3rd rows in the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight values ​​and sends it to the first pixel circuit in the circuit unit that obtains the pixel values ​​of the 2nd and 5th columns of the 1st, 2nd, and 3rd rows.

[0158] Simultaneously, the second pixel circuit in the circuit unit that obtains the pixel values ​​of the 3rd and 6th columns of the 1st, 2nd, and 3rd rows performs weight multiplication. The weighted register unit generates an input weight value based on the weights of the 2nd column of the 1st, 2nd, and 3rd rows in the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the second pixel circuit in the circuit unit that obtains the pixel values ​​of the 3rd and 6th columns of the 1st, 2nd, and 3rd rows.

[0159] Simultaneously, the third pixel circuit in the circuit unit that obtains the pixel values ​​of the 4th and 7th columns of the 1st, 2nd, and 3rd rows performs weight multiplication. The weighted register unit generates an input weight value based on the weights of the 3rd column of the 1st, 2nd, and 3rd rows in the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the third pixel circuit in the circuit unit that obtains the pixel values ​​of the 4th and 7th columns of the 1st, 2nd, and 3rd rows.

[0160] Simultaneously, the first pixel circuit in the circuit unit that obtains the pixel values ​​of the first row, second row, and third column of the third row performs weight multiplication. The weight register unit generates input weight values ​​based on the weights of the first row, second row, and first column of the third row in the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight values ​​and sends it to the first pixel circuit in the circuit unit that obtains the pixel values ​​of the first row, second row, and third column of the third row.

[0161] Simultaneously, the second pixel circuit in the circuit unit that obtains the pixel values ​​of the 4th column of the 1st, 2nd, and 3rd rows performs weight multiplication. The weighted register unit generates an input weight value based on the weights of the 2nd column of the 1st, 2nd, and 3rd rows in the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the second pixel circuit in the circuit unit that obtains the pixel values ​​of the 4th column of the 1st, 2nd, and 3rd rows.

[0162] Simultaneously, the third pixel circuit in the circuit unit that obtains the pixel values ​​of the 5th column of the 1st, 2nd, and 3rd rows performs weight multiplication. The weighted register unit generates an input weight value based on the weights of the 3rd column of the 1st, 2nd, and 3rd rows in the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the third pixel circuit in the circuit unit that obtains the pixel values ​​of the 5th column of the 1st, 2nd, and 3rd rows.

[0163] The same principle applies to assigning weight values ​​to the pixel circuits in the pixel circuit units that obtain the values ​​of the remaining rows of pixels to be convolved.

[0164] In summary, for each convolutional region, when the kernel size is n and the stride is 1, weight values ​​can be assigned to the entire convolutional region at one time.

[0165] In conjunction with the above embodiments, the output terminals of the pixel circuits in each row are connected, and a first switch is connected in series between two pixel circuits. Step S204 of the data processing circuit control method further includes:

[0166] Close the first switch between the first pixel circuit of the pixel circuit unit in the j4th column of each row of the focal plane and the n2nd pixel circuit of the n-1 adjacent pixel circuit units in the j5th column to the right.

[0167] Specifically, in conjunction with the above embodiments and Figure 4 , 5 As shown. Figure 5 Taking the first convolution region 211 in rows 1 to 3 as an example, among all pixels in the first row of the focal plane to be convolved, the first switch between the first pixel circuit in the pixel circuit unit that obtains the value of pixel 00 in the first column of the first row and the n2nd pixel circuit in the right-hand (n-1)th pixel circuit units in the j5th column is closed. That is, the first switch between the second pixel circuit in the pixel circuit unit that obtains the value of pixel 01 in the second column of the first row and the third pixel circuit in the pixel circuit unit that obtains the value of pixel 02 in the third column of the first row is closed, thus obtaining the second convolution result. The first switch closure of the pixel circuits in the remaining rows and columns or the remaining convolution regions is the same as in the above embodiment, and will not be described in detail here.

[0168] In conjunction with the above embodiments, the output terminals of the pixel circuits in each column are connected, and a second switch is connected in series between two pixels. Step S204 of the data processing circuit control method further includes:

[0169] In the column direction of each convolutional region, when the number of columns of the pixel circuit units included in each adjacent first pixel circuit group formed by n adjacent pixel circuits that have their first switches turned on is the same, the second switch between adjacent first pixel circuit groups is controlled to be turned on. Specifically, in conjunction with the above embodiment, still using... Figure 5Taking the first convolution region 211 as an example, the closing state of the first switch was discussed above. In the column direction of each convolution region, the first pixel circuit group includes: the first pixel circuit in the pixel circuit unit that acquires the value of pixel 00 in the first row and first column, the second pixel circuit in the pixel circuit unit that acquires the value of pixel 01 in the first row and second column, and the third pixel circuit in the pixel circuit unit that acquires the value of pixel 02 in the first row and third column; the first pixel circuit in the pixel circuit unit that acquires the value of pixel 10 in the second row and first column, the second pixel circuit in the pixel circuit unit that acquires the value of pixel 11 in the second row and second column, and the third pixel circuit in the pixel circuit unit that acquires the value of pixel 12 in the second row and third column; the first pixel circuit in the pixel circuit unit that acquires the value of pixel 20 in the third row and first column, the second pixel circuit in the pixel circuit unit that acquires the value of pixel 21 in the third row and second column, and the third pixel circuit in the pixel circuit unit that acquires the value of pixel 22 in the third row and third column. It can be seen that the pixel circuits in the pixel circuit units of this pixel circuit group are all bit-1, bit-2, and bit-3. Therefore, the second switch between these three first pixel circuit groups is turned on.

[0170] The closing status of the second switch in other times, other convolution regions, or other first pixel point circuit groups is the same as in the above embodiments, and will not be repeated here.

[0171] Figure 10 This is a schematic diagram of the structure of a data processing system provided in an embodiment of the present invention, as shown below. Figure 10 As shown, the data processing system includes: a data processing circuit 401 and an analog-to-digital conversion module 402 provided in any embodiment of the present invention;

[0172] The analog-to-digital converter module 402 is connected to the data processing circuit 401 and is used to convert the signal output by the data processing circuit 401 from analog to digital.

[0173] The data processing system provided in this embodiment of the invention can achieve the same technical effects as the data processing circuit provided in the above-described embodiments of the invention, and will not be described again here.

[0174] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data processing method, characterized in that, The method includes: Copy each pixel on the focal plane There are n processing units; where the width and height of the convolution kernel are both n, and n≥1; the convolution stride is s; where s≥1; In a single convolution operation, each processing unit in a pixel is multiplied by the weight of the corresponding convolution kernel to obtain a first convolution result; the first convolution results of the processing units corresponding to the same convolution kernel in the same row are summed to obtain a second convolution result. The results of the second convolution between rows are summed to output the final convolution result; The focal plane is column j. When s=1, in a single convolution operation, each processing unit in the pixel is multiplied by the weight of the corresponding convolution kernel to obtain the first convolution result, including: In each convolution region, for the pixel in the j1-th column and n1-th row, the h1-th bit processing unit of the pixel is multiplied by the weight in the n1-th row and h1-th column of the convolution kernel, where j1 < n, h1 ≤ j1, and n1 is the row number of the pixel in the convolution region, n1 ≤ n. In each convolution region, for the pixel in the j2th column and n1th row, the weights in the n1st row and h2th column of the convolution kernel of the h2th bit processing unit of the pixel are multiplied together, where j - n + 1 < j2 ≤ j, nj - j2 ≤ h2 ≤ n; In each convolution region, for the pixel in the j3rd column and n1st row, the h3rd bit processing unit of the pixel is multiplied by the weight in the n1st row and h3rd column of the convolution kernel to obtain the first convolution result; where n ≤ j3 ≤ j - n + 1, h3 ≤ n; The step of accumulating the first convolution results of the processing units corresponding to the same convolution kernel on the same row to obtain the second convolution result includes: The first convolution result corresponding to the first processing unit of the pixel in the j4th column of each row of the focal plane is added to the first convolution result corresponding to the n2nd processing unit of the n-1 adjacent pixels in the j5th column to the right to obtain the second convolution result, where j4<j5≤n+j4-1; n2=j5-j4+1; The step of accumulating the second convolution results between rows to output the final convolution result includes: In the column direction of each convolution region, when the number of columns of the n pixels corresponding to each of the n second convolution results is the same, the n second convolution results are added together to output the final convolution result.

2. The data processing method according to claim 1, characterized in that, Also includes: After one convolution calculation, the convolution region is moved down by one stride for the next convolution operation until the final convolution result on the entire focal plane is obtained; wherein, the height of the convolution region is equal to the height of the convolution kernel, the width of the convolution region is equal to the width of the focal plane, and adjacent convolution regions are adjacent along the height direction of the focal plane.

3. A data processing circuit, characterized in that, The data processing circuit includes: a focal plane module and a pulse width generator module; The focal plane module and the pulse width generator module are connected; The pulse width generator module is used to output a pulse width modulation signal representing the weights of different convolution kernels to the focal plane; The focal plane module includes multiple pixel circuits; The multiple pixel circuits of the focal plane module are used to multiply n identical input values ​​with the corresponding pulse width modulation signal to obtain the first convolution result; The first convolution result in the pixel circuits corresponding to the same convolution kernel in the same row is accumulated to obtain the second convolution result; The results of the second convolution between rows are summed to output the final convolution result; The pixel circuit includes: a P-type field-effect transistor, an N-type field-effect transistor, a photodiode, and a first capacitor; The first terminal of the P-type field-effect transistor, the first terminal of the N-type field-effect transistor, and the negative terminal of the photodiode are connected. The second terminal of the N-type field-effect transistor is connected to the first power supply, the second terminal of the P-type field-effect transistor is connected to the first terminal of the first capacitor and the pixel output terminal, and the positive terminal of the photodiode is grounded. The second terminal of the first capacitor is grounded; The focal plane module also includes: The output terminals of the pixel circuits in each row are connected, and a first switch is connected in series between the output terminals of two pixel circuits in the same row. The output terminals of each column of pixel circuits are connected, and a second switch is connected in series between the output terminals of two pixel circuits in the same column.

4. The data processing circuit according to claim 3, characterized in that, Each pixel circuit includes a control terminal, and the output terminal of the pulse width generator module includes multiple sub-output terminals, which are connected one-to-one with the control terminal of the pixel circuit of the focal plane.

5. A data processing circuit control method, applied to the data processing circuit of any one of claims 3-4, characterized in that, include: The data processing circuit includes a controller module; the pulse width generator module includes a weighted register unit and a pulse generation unit. The controller module controls the adjacent Each of the aforementioned pixel circuits acquires the same value for a single pixel; where n is the size of the convolution kernel and s is the stride of the convolution. In a single convolution operation, the controller module controls the weighting register unit to generate weight values ​​based on the convolution kernel and send them to the pulse generation unit; the controller module controls the pulse generation unit to generate pulse width modulation signals based on the received weight values ​​and simultaneously send them to the pixel circuit of the focal plane. The controller module controls the accumulation of the output results of the pixel circuits in the focal plane according to the time when each pixel circuit in the focal plane receives the pulse width modulation signal; After the output of the pixel circuit is accumulated, the controller module controls the output of the accumulated result.

6. The data processing circuit control method according to claim 5, characterized in that, The method further includes: Before the pulse generation unit generates a pulse width modulation signal based on the received weight value and simultaneously sends it to each pixel circuit of the focal plane, the controller module controls the initialization of the pixel circuit unit. After each convolution operation, the controller module controls the weight register unit to generate weight values ​​based on the weights of the previous row of the convolution kernel and send them to the pulse width generator module.

7. The data processing circuit control method according to claim 5, characterized in that, The controller module controls the weighted register unit to generate weight values ​​based on the convolution kernel and send them to the pulse generation unit; The controller module controls the pulse generation unit to generate a pulse width modulation signal according to the received weight value and simultaneously send it to the circuit of each pixel point of the focal plane, including: When the convolution stride s=1, the focal plane is in column j, the width and height of the convolution kernel are both n, and n≥1; In each convolution region, adjacent pixel circuits that copy the same pixel value constitute a pixel circuit unit; For the pixel circuit unit in the j1st column and n1st row, the weighting register unit generates an input weight value based on the weight in the n1st row and h1st column of the convolution kernel. The pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the pixel circuit unit in the j1st column and n1st row, where j1 < n, h1 ≤ j1, n1 is the row number of the pixel circuit unit in the convolution region, and n1 ≤ n. In each convolution region, for the pixel circuit unit in the j2th column and n1st row, the weighting register unit generates an input weight value based on the weight in the n1st row and h2th column of the convolution kernel, and the pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the pixel circuit unit in the j2th column and n1st row, where j - n + 1 < j2 ≤ j, nj - j2 ≤ h2 ≤ n; In each convolution region, for the pixel in the j3rd column and n1st row, the weighting register unit generates an input weight value based on the weight in the n1st row and h3rd column of the convolution kernel, and the pulse width generation module generates a pulse width modulation signal based on the received input weight value and sends it to the pixel circuit unit in the j3rd column and n1st row, where n ≤ j3 ≤ j - n + 1, h3 ≤ n; The output terminals of the pixel circuits in each row are connected, and a first switch is connected in series between two pixel circuits. The controller module controls the accumulation of the output results of the pixel circuits in the focal plane according to the time when each pixel circuit in the focal plane receives the pulse width modulation signal, including: Close the first switch between the first pixel circuit of the pixel circuit unit in the j4th column of each row of the focal plane and the n2nd pixel circuit of the n-1 adjacent pixel circuit units in the j5th column to the right, where j4<j5≤n+j4-1;n2=j5-j4+1; The output terminals of the pixel circuits in each column are connected, and a second switch is connected in series between two pixels. The controller module controls the accumulation of the output results of the pixel circuits in the focal plane according to the time when each pixel circuit receives the pulse width modulation signal. It also includes: In the column direction of each convolution region, when the number of columns of the pixel circuit units included in each adjacent first pixel circuit group that has the first switch turned on is the same, the second switch between the adjacent first pixel circuit groups is controlled to be turned on.

8. A data processing system, characterized in that, include: The data processing circuit and analog-to-digital conversion module according to any one of claims 3-4; The analog-to-digital conversion module is connected to the data processing circuit, and the analog-to-digital conversion module is used to convert the signal output by the data processing circuit into an analog-to-digital signal.

Images