An image recognition acceleration method based on in-memory computing chip
By locking the physical column of overlapping pixel data in the in-memory computing chip to a write-free state and using cyclic shifting to compensate for the physical storage offset of pixel data, the high power consumption and latency problems caused by repeated erasing and writing of overlapping pixels are solved, thereby improving the efficiency and performance of image recognition acceleration processing.
Patent Information
- Application Number
- CN202611392863.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-09
- Publication Date
- 2026-11-17
AI Technical Summary
In image recognition acceleration processing, existing in-memory computing chips suffer from high write power consumption and increased processing latency due to repeated erasing and writing of overlapping pixels. Furthermore, the physical storage offset of data after only partial updates makes it difficult to maintain the correspondence between computational weights and row masking control.
By using a circular write offset in the in-memory computing chip to lock the physical columns of overlapping pixel data into a write-free state, and controlling the newly added pixel data to only overwrite the unlocked physical columns, while compensating for the physical storage offset of pixel data through cyclic shifting, the correspondence between computation weight and row masking control is maintained.
It reduces invalid memory erase/write overhead, lowers write power consumption, improves the efficiency of sliding window data update acceleration, and reduces processing latency and invalid analog read power consumption, thereby enhancing the image recognition acceleration effect.
Smart Images

Figure CN123115048A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, specifically to an image recognition acceleration method based on a memory computing chip. Background Technology
[0002] In-memory computing chips can directly perform computational operations within physical storage units, reducing data transfer overhead between storage and computing modules. This makes them suitable for handling numerous local window multiplication and addition operations in image recognition. Existing image recognition acceleration solutions typically employ a sliding window mechanism. The control logic loads the entire image pixels within the computation window into the physical array according to a preset size and applies weight signals to complete the computation. As the computation window slides across the image data, the peripheral control circuit usually rewrites the complete window data as a new data block into the physical column to maintain alignment between the pixel distribution and the input position of the computation weights.
[0003] However, adjacent computation windows contain many overlapping pixels. Rewriting the entire window causes repeated erasure and writing of overlapping pixels, increasing memory write power consumption and processing latency. If only newly added pixels are written, although erasure and writing can be reduced, overlapping pixels continue to reside in the original physical column, causing a physical storage offset of the window data in the physical array. Existing control methods lack weight and mask compensation mechanisms corresponding to this offset state, making it difficult to maintain multiply-accumulate alignment without moving array data, and also difficult to synchronously shut down the pre-charge and discharge path of the read word line corresponding to the zero-weight physical row, thus affecting the efficiency of image recognition acceleration processing and power consumption optimization.
[0004] Therefore, this invention proposes an image recognition acceleration processing method based on an in-memory computing chip to overcome the shortcomings of existing technologies. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention provides an image recognition acceleration processing method based on a memory computing chip. This method solves the problems in existing memory computing image recognition sliding window processing, such as high write power consumption and increased processing latency due to repeated erasing and writing of overlapping pixels, and the physical storage offset of data after only locally updating newly added pixels, making it difficult to maintain the correspondence between computational weights and row masking control.
[0006] To achieve the above objectives, the present invention provides the following technical solution: an image recognition acceleration processing method based on a memory computing chip, comprising the following steps:
[0007] Write the image pixel data of the initial calculation window into the physical column of the physical array, initialize the circular write offset, and convert the sparse operation template into a one-dimensional weight sequence and the original sparse mask sequence.
[0008] The read line enable signals of the corresponding physical rows in the physical array are masked according to the original sparse mask sequence. Multiply-accumulate operations are performed based on the one-dimensional weight sequence and the image pixel data in the unmasked physical rows of the physical array, and the recognition result is output.
[0009] When the calculation window slides, the updated ring write offset is obtained based on the previous state, and the overlapping pixel data and the newly added pixel data are determined.
[0010] Based on the updated circular write offset, the physical column storing the overlapping pixel data is locked to a write-free state, and the newly added pixel data is written to the physical column that is in an unlocked state.
[0011] The one-dimensional weight sequence and the original sparse mask sequence are cyclically shifted using the updated circular write offset to obtain a shifted weight sequence and a shifted sparse mask sequence. The read word line enable signal of the corresponding physical row in the physical array is re-masked using the shifted sparse mask sequence. Multiply-accumulate operations are performed based on the shifted weight sequence and the recognition result is output.
[0012] Preferably, the step of converting the sparse operation template into a one-dimensional weight sequence and the original sparse mask sequence includes:
[0013] The weight elements in the sparse operation template are continuously spliced together in a fixed row-first scanning order to form the one-dimensional weight sequence, and a corresponding mapping relationship is established between the data bit index in the one-dimensional weight sequence and the pin address of the physical row in the physical array.
[0014] Read the weight elements in the one-dimensional weight sequence bit by bit, and generate the first logical value when the weight element has zero weight.
[0015] When the weight element has a non-zero weight, a second logical value is generated;
[0016] The generated first and second logical values are combined according to the same index positions as the one-dimensional weight sequence to obtain the original sparse mask sequence.
[0017] Preferably, the step of masking the read word line enable signal of the corresponding physical row in the physical array according to the original sparse mask sequence includes:
[0018] Analyze the bits in the original sparse mask sequence;
[0019] When the bit in the original sparse mask sequence is parsed to be the first logic value, the read word line enable signal of the corresponding physical row in the physical array is turned off.
[0020] When the bit in the original sparse mask sequence is parsed to be the second logic value, the read word line enable signal of the corresponding physical row in the physical array is turned on.
[0021] Preferably, the step of obtaining the updated ring write offset based on the previous state includes:
[0022] Add the circular write offset of the previous state to the window sliding step size to obtain the accumulated result;
[0023] The accumulated result is subjected to a modulo operation based on the calculation window width, and the result of the modulo operation is used as the updated ring write offset.
[0024] Preferably, the step of determining the overlapping pixel data and the newly added pixel data includes:
[0025] The data intersection width is determined based on the window sliding step size and the calculation window width, wherein the data intersection width is the difference between the calculation window width and the window sliding step size;
[0026] The intersection data is determined based on the circular write offset and the data intersection width of the previous state. The intersection data is identified as the overlapping pixel data, and an overlapping pixel dwell mark is generated for the physical column where the overlapping pixel data is located.
[0027] The image pixel data that is slid into the calculation window after sliding is determined as the newly added pixel data.
[0028] Preferably, the step of locking the physical column storing the overlapping pixel data to a write-free state based on the updated circular write offset, and writing the newly added pixel data to the unlocked physical column, includes:
[0029] A column write enable control word is generated based on the updated ring write offset and the overlapping pixel dwell mark. The bit position in the column write enable control word corresponding to the overlapping pixel dwell mark is set to invalid logic level, and the remaining bits in the column write enable control word are kept to valid logic level.
[0030] The column write enable control word is mapped bit by bit to the write enable pin of each physical column of the physical array, thereby closing the write path of the physical column storing the overlapping pixel data and locking it to the write-free state.
[0031] The newly added pixel data is written sequentially into the physical columns where the write enable pin is in a valid logic level state, and the physical columns where the write enable pin is in a valid logic level state correspond to the physical columns in an unlocked state.
[0032] Preferably, the step of sequentially writing the newly added pixel data into the physical column where the write enable pin is in a valid logic level state includes:
[0033] During the continuous writing of the new pixel data, when the column address pointer accumulates to the maximum boundary of the physical column of the physical array, the column address pointer is wrapped back to the first physical column to continue the writing operation.
[0034] Preferably, the step of performing a cyclic shift on the one-dimensional weight sequence and the original sparse mask sequence with the updated circular write offset to obtain a shifted weight sequence and a shifted sparse mask sequence includes:
[0035] The weight sequence portion of the one-dimensional weight sequence is controlled to perform a cyclic shift in the opposite direction to the physical storage offset generated by the writing of new pixel data. The number of shift steps is equal to the value of the updated cyclic write offset. The shifted weight sequence portion is then combined with the remaining unshifted sequence portion to obtain the shifted weight sequence.
[0036] The mask sequence portion in the original sparse mask sequence is controlled to perform cyclic shifts in the same direction and with the same number of steps as the weight sequence portion in the one-dimensional weight sequence, and the shifted mask sequence portion is combined with the remaining unshifted sequence portion to obtain the shifted sparse mask sequence.
[0037] Preferably, the method further includes:
[0038] When an image horizontal boundary signal indicating that the calculation window has reached the horizontal boundary of the image data is detected, the circular write offset is cleared.
[0039] Restore the write enable signals of all physical columns of the physical array to the valid state;
[0040] Restore the read word line enable signal of all physical rows of the physical array to the enabled state;
[0041] Return to perform the initial loading action for the next image row.
[0042] Preferably, the method further includes:
[0043] Perform bit-width consistency comparison, effective logic bit count comparison, and mask bit state comparison on the actual output shift sparse mask sequence;
[0044] If any comparison result is inconsistent, a shift anomaly is determined to have occurred, a hardware interrupt signal is generated, and the multiply-accumulate operation of the current cycle is suspended.
[0045] The updated circular write offset is reread, and the weight sequence portion of the one-dimensional weight sequence and the mask sequence portion of the original sparse mask sequence are re-performed cyclic shift to obtain the repaired shifted sparse mask sequence and the repaired shifted weight sequence.
[0046] The repaired shift sparse mask sequence is reused to mask the read word line enable signal of the corresponding physical row in the physical array. The repaired shift weight sequence is reused to simulate the excitation conversion, the hardware interrupt state corresponding to the hardware interrupt signal is released, and the suspended multiply-accumulate operation is restored.
[0047] This invention provides an image recognition acceleration processing method based on a memory computing chip. It has the following beneficial effects:
[0048] 1. This invention locks the physical column storing overlapping pixel data into a write-free state according to the circular write offset when the calculation window slides, and controls the newly added pixel data to only overwrite the unlocked physical column. This avoids repeated erasing and writing of overlapping pixel data inside the in-memory computing chip, transforms the overall update of conventional sliding window data into a partial overwrite update, reduces the overhead of invalid memory erasure and writing, and improves the speedup processing efficiency of sliding window data update.
[0049] 2. This invention performs a cyclic shift on a one-dimensional weight sequence using an updated circular write offset, and compensates for the physical storage offset of pixel data in the physical array by logical rearranging the computational weights. This allows the in-memory computing chip to perform multiplication and addition operations based on the shift weights without having to read and move array data. This reduces the processing delay caused by sliding window updates while maintaining the correspondence of image recognition calculations, thereby improving the image recognition acceleration effect.
[0050] 3. This invention controls the original sparse mask sequence to perform the same number of cyclic shifts as the weight sequence, and uses the shifted sparse mask to re-mask the read word line enable signal of the corresponding physical row. This enables the in-memory computing chip to maintain the correspondence between the zero-weight position and the row mask position when the pixel data undergoes a physical storage offset. This reduces the pre-charge and discharge actions of the corresponding read path, lowers the power consumption of invalid analog reading in the image recognition process, and further improves the overall acceleration processing performance. Attached Figure Description
[0051] Figure 1 This is a diagram illustrating the architecture of an image recognition acceleration processing system based on a memory computing chip, according to the present invention.
[0052] Figure 2 This is a flowchart of an image recognition acceleration processing method based on a memory computing chip according to the present invention;
[0053] Figure 3 This is a schematic diagram of the initial data alignment writing and sparse template configuration of the present invention;
[0054] Figure 4 This is a schematic diagram of the forced shielding of the reading line and the in-situ multiplication and addition operation of the present invention;
[0055] Figure 5 This is a schematic diagram of the sliding window offset update and overlapping pixel marking of the present invention;
[0056] Figure 6 This is a schematic diagram of the physical column write-free locking and ring overwrite writing of the present invention;
[0057] Figure 7 This is a schematic diagram of the sequence reverse shift compensation and sliding multiplication-addition calculation of the present invention.
[0058] Among them, 100 is an in-memory computing chip; 110 is an in-memory computing module; 120 is a row address decoding module; 130 is a column write control module; 200 is a logic controller; 210 is an offset tracking module; and 220 is a shift and rearrangement module. Detailed Implementation
[0059] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0060] Reference Figure 1 The present invention provides an image recognition acceleration processing system based on a memory computing chip, including a logic controller 200 and a memory computing chip 100 communicatively connected to the logic controller 200.
[0061] The in-memory computing chip 100 includes an in-memory computing module 110, a row address decoding module 120 connected to the row control port of the in-memory computing module 110, and a column write control module 130 connected to the column data port of the in-memory computing module 110.
[0062] The logic controller 200 includes an offset tracking module 210 and a shift rearrangement module 220.
[0063] The control bus of the logic controller 200 is connected to the control input terminal of the row address decoding module 120, the control input terminal of the column write control module 130, and the analog excitation input port of the memory module 110.
[0064] The logic controller 200 sends a column write enable control word to the column write control module 130 to control the physical column write state, and sends a read word line shielding signal to the row address decoding module 120 to control the physical row pre-charge and discharge.
[0065] Externally input image pixel data is transmitted unidirectionally to the column data port of the memory-to-grid module 110 via the logic controller 200. After completing the parallel multiplication and addition operations, the memory-to-grid module 110 feeds back the calculation result data to the logic controller 200.
[0066] Reference Figure 2 The present invention also provides an image recognition acceleration processing method based on a memory computing chip, comprising the following steps:
[0067] S100: The logic controller 200 acquires the image data to be processed and the preset sparse operation template, extracts a row of image pixel data corresponding to the current parallel operation from the initial calculation window from the image data, and continuously writes the row of image pixel data into the effective physical column of the storage and calculation module 110 through the column write control module 130, while initializing the ring write offset state variable.
[0068] The logic controller 200 flattens the preset sparse operation template into a one-dimensional weight sequence, generates an original sparse mask sequence based on the position information of zero weights in the one-dimensional weight sequence, and determines the weight sequence part used for horizontal sliding compensation in the one-dimensional weight sequence based on the width of the row of image pixel data corresponding to the current parallel operation, and determines the mask sequence part corresponding to the index in the original sparse mask sequence.
[0069] S200: The logic controller 200 sends the original sparse mask sequence to the row address decoding module 120. The row address decoding module 120 forcibly masks the read word line enable signal of the physical row corresponding to zero weight in the storage module 110 according to the original sparse mask sequence.
[0070] The logic controller 200 converts the one-dimensional weight sequence into an analog excitation signal and applies it to the unmasked physical row of the storage module 110. The storage module 110 performs parallel multiplication and addition operations on a row of image pixel data and the non-zero weight data in the one-dimensional weight sequence, and outputs the recognition result of the initial calculation window.
[0071] S300: When the calculation window slides horizontally across the image data to the current window according to the set step size, the offset tracking module 210 performs a modulo operation based on the ring write offset of the previous state and the set step size to obtain the updated ring write offset.
[0072] The offset tracking module 210 determines the newly added pixel data that slides in within the current window and marks the overlapping pixel data that does not need to be updated.
[0073] S400: Logic controller 200 generates a column write enable control word based on the updated ring write offset and the marked overlapping pixel data.
[0074] The column write control module 130 locks the physical column storing the marked overlapping pixel data in the storage module 110 into a write-free state according to the column write enable control word.
[0075] The column write control module 130 sequentially overwrites the determined new pixel data into the unlocked physical column of the storage and calculation module 110, completing the circular local update of the physical array data.
[0076] S500: The shift rearrangement module 220 performs a reverse cyclic shift operation on the weight sequence portion of the one-dimensional weight sequence corresponding to the current horizontal sliding direction and the mask sequence portion of the original sparse mask sequence corresponding to the weight sequence portion, using the updated circular write offset as the shift step.
[0077] The shift rearrangement module 220 combines the shifted sequence portion with the remaining unshifted sequence portion to obtain the shift weight sequence and the shift sparse mask sequence.
[0078] The row address decoding module 120 re-masks the read word line enable signal of the corresponding physical row of the storage and computing module 110 according to the shifted sparse mask sequence.
[0079] The logic controller 200 applies the shift weight sequence to the storage module 110, the storage module 110 performs parallel multiplication and addition operations and outputs the recognition result of the current window, and then returns to step S300 to perform the next window sliding.
[0080] In actual image processing, when the calculation window slides to the end of an image row and prepares to wrap to the next line, the logic controller 200 triggers the line wrap reset logic. The logic controller 200 clears the current circular write offset, releases the column write-free state and row masking state of the storage and calculation module 110, and returns to step S100 to execute the initial loading action of the next image row.
[0081] The following section describes the aforementioned image recognition acceleration processing system and method based on in-memory computing chips, taking into account specific application scenarios and underlying hardware control processes.
[0082] Before processing the externally input image data through the logic controller 200, the logic controller 200 first configures the hardware parameters and state variables related to sliding window writing.
[0083] The calculation window width is denoted as The effective number of physical columns in the storage module 110 is configured to be equal to the width of the calculation window. This allows the pixel data of a single row of calculation windows corresponding to a single parallel operation to be contained in the effective physical column of the storage module 110.
[0084] The window sliding step size parameter is denoted as Window sliding step size parameter This represents the number of pixels the calculation window moves horizontally across the image data in a single operation, and its value ranges from [value range missing]. .
[0085] In order to achieve partial updates without moving the pixel data resident inside the storage module 110, the logic controller 200 initializes and maintains the circular write offset state variable. .
[0086] Circular write offset state variable This is used to represent the physical offset of the starting point of the calculation window data sequence relative to the 0th column of the storage module 110. In the initial alignment state, the offset state variable is written in a circular fashion. The initial value is 0.
[0087] During a window sliding update, the offset state variable is written from the ring. The value read before the update is denoted as the circular write offset before the update. The updated value obtained after modulo operation is recorded as the updated circular write offset. .
[0088] As the calculation window slides, overlapping pixel data in the valid physical columns of the storage module 110 remains in the original physical columns, while newly added pixel data is written to the physical columns containing expired historical data according to the circular boundary mapping rule. The resulting physical storage offset state is written to the circular offset state variable. Record.
[0089] Logic controller 200 completes the calculation of window width Window sliding step size parameter and circular write offset state variable After configuration, save it as hardware status data for subsequent steps.
[0090] Window sliding step parameters Compared to the ring write offset before the update The input is sent to step S300 to perform the modulo update operation. Step S300 obtains the updated circular write offset. Then, update the circular write offset. As a control basis, it is used to participate in the generation of the column write enable control word in step S400, and to control the shift rearrangement module 220 on the one-dimensional weight sequence in step S500. The weight sequence part and the original sparse mask sequence in The mask sequence portion of the data undergoes a reverse compensation shift.
[0091] The logic controller 200 acquires a preset two-dimensional sparse operation template from external input. This preset two-dimensional sparse operation template is a two-dimensional weight matrix containing zero-weight elements and non-zero-weight elements, and converts it into a one-dimensional weight sequence in the internal cache according to the matrix flattening rule. .
[0092] Specifically, the logic controller 200 extracts and concatenates the weight elements of the sparse operation template in the form of a two-dimensional matrix in a fixed row-first scanning order to form a one-dimensional weight sequence arranged in a single row. .
[0093] One-dimensional weight sequence The sequence length is aligned with the input dimension required for a single multiply-accumulate operation. This one-dimensional weight sequence Each data bit index in the memory is associated with a pin address of the valid physical rows R0 to R(H-1) in the memory module 110 that participate in the calculation, where H is a one-dimensional weight sequence. The sequence length.
[0094] Following row-first scan order, the one-dimensional weight sequence The weight sequence consists of consecutive weight elements that match the width of a row of image pixel data corresponding to the current parallel operation.
[0095] After completing the one-dimensional weight sequence While flattening, the logic controller 200 reads the one-dimensional weight sequence one by one. The weight values in the sequence are used, and a one-dimensional weight sequence is generated based on whether the weight values are zero. Equal-length original sparse mask sequence Original sparse mask sequence Used to indicate the control level status of each input pin of the row address decoding module 120.
[0096] Original sparse mask sequence The continuous mask bits with the same index range constitute the mask sequence part corresponding to the weight sequence part.
[0097] Original sparse mask sequence The level mapping rule between the input pins of the row address decoding module 120 is as follows: when the one-dimensional weight sequence When the element at the current position in the original sparse mask sequence has zero weight, the logic controller 200... The same index position generates a logic value "0". The logic value "0" is mapped as a low-level mask signal at the hardware interface, which is used to instruct the row address decoding module 120 to shut down the read word line enable path of the corresponding physical row.
[0098] When a one-dimensional weight sequence When the element at the current position in the original sparse mask sequence has a non-zero weight, the logic controller 200... The same index position generates a logic value "1". The logic value "1" is mapped to a high-level enable signal at the hardware interface, which is used to indicate that the row address decoding module 120 enables the pre-charging and discharging action of the read word line of the corresponding physical row.
[0099] After completing the above mapping and mask generation, the logic controller 200 saves the one-dimensional weight sequence. With the original sparse mask sequence .
[0100] In step S200, the original sparse mask sequence Used to drive the row address decoding module 120 to complete the read line masking in the initial alignment state, a one-dimensional weight sequence. Used to generate the corresponding simulated excitation signal. In step S500, a one-dimensional weight sequence... The weight sequence part and the original sparse mask sequence The mask sequence portion is used as the base data, along with the updated circular write offset. Complete the synchronous shift during the subsequent sliding process.
[0101] Reference Figure 3 When the in-memory computing chip 100 is in an idle reset state, the logic controller 200 acquires the raw image data and the preset sparse operation template from the external input, and executes step S100 to complete the physical alignment writing of a row of window data in the first calculation window, as well as the initial configuration of the sparse operation template. Step S100 specifically includes sub-steps S101 and S102.
[0102] In sub-step S101, the logic controller 200 extracts a row of window data corresponding to the current parallel operation from the first calculation window of the externally input raw image data, and controls the column write control module 130 to continuously write the pixel data in the row of window data into the storage and calculation module 110 in physical order.
[0103] The specific process is as follows: The logic controller 200 initializes the column address pointer internally and points the column address pointer to the first valid physical column (i.e., column 0) of the memory module 110. During continuous writing, the logic controller 200 sends the image pixel data in one row of window data in the first calculation window to the column write control module 130 in sequence according to the external working clock signal.
[0104] Within the current clock cycle of receiving data, the column write control module 130 enables the physical column write signal pointed to by the current column address pointer, writes the current pixel data into the corresponding valid physical column, and after the current write operation is completed, causes the column address pointer to step to the next adjacent physical column.
[0105] The above incremental writing process continues until all pixel data in one row of window data in the first calculation window is continuously written into the valid physical column of the storage and calculation module 110.
[0106] After the current processing, the pixel data physically loaded in the storage module 110 is used as the initial ready image calculation input data, which is then used to perform in-situ multiplication and addition operations in the subsequent step S200.
[0107] In sub-step S102, the logic controller 200 reads the preset sparse operation template and converts the sparse operation template into a control sequence used by the in-memory computing module 110 for calculation.
[0108] The specific process is as follows: The logic controller 200 flattens the preset sparse operation template according to the matrix flattening logic determined above, and generates a one-dimensional weight sequence. And based on the one-dimensional weight sequence The zero-weight position information is used to generate the original sparse mask sequence. .
[0109] The logic controller 200 also adjusts the one-dimensional weight sequence based on the width of a row of data corresponding to the current parallel operation. The weight sequence portion used for horizontal sliding compensation is determined in the original sparse mask sequence. The part of the mask sequence corresponding to the index range is determined in the middle.
[0110] In generating the original sparse mask sequence During the process, the logic controller 200 establishes the original sparse mask sequence. The correspondence between the bits and the input pins of the row address decoding module 120.
[0111] The logic controller 200 converts the original sparse mask sequence It is constructed as a fixed-length multi-bit control word. Each bit in this fixed-length multi-bit control word is mapped to an input pin of the row address decoding module 120 in a fixed index order.
[0112] The low or high level of a specific bit in the fixed-length multi-bit control word determines the level state of the corresponding input pin of the row address decoding module 120. The currently processed one-dimensional weight sequence... and the original sparse mask sequence It is stored in the internal register of the logic controller 200.
[0113] In the subsequent step S200, the one-dimensional weight sequence Used to simulate excitation transformation, the original sparse mask sequence Used for physical row masking. In subsequent step S500, a one-dimensional weight sequence... The weight sequence part and the original sparse mask sequence in The mask sequence portion is output to the shift and rearrangement module 220 for performing a reverse cyclic shift operation.
[0114] Reference Figure 4 Based on the one-dimensional weight sequence obtained in step S100 Original sparse mask sequence In addition to the pixel data already written into the storage and calculation module 110, the logic controller 200 executes step S200 to perform image recognition calculations for the initial calculation window under physical address alignment. Step S200 specifically includes sub-steps S201 and S202.
[0115] In sub-step S201, the logic controller 200 converts the original sparse mask sequence The data is sent to the row address decoding module 120. The row address decoding module 120 decodes the data based on the original sparse mask sequence. Disable the read line enable path for the physical row corresponding to zero weight.
[0116] The specific process is as follows: Before sending the weight calculation stimulus to the memory computing module 110, the logic controller 200, through timing control logic, advances the original sparse mask sequence by more than one clock cycle. The control pins of the row address decoding module 120 are applied.
[0117] Row address decoding module 120 parses the original sparse mask sequence. The level state. When the corresponding bit is detected to be low, indicating zero weight, the row address decoding module 120 disables the read word line (RWL) enable signal of the corresponding valid physical row in the storage module 110.
[0118] After the read word line enable signal is shielded, the access transistors between the memory cells inside the shielded physical row and the array read bit lines remain in the cutoff region, preventing that row from participating in the bit line pre-charging and subsequent discharging actions during the read cycle, thereby reducing the power consumption of invalid analog reads corresponding to zero weight. The current processing determines the read enable state of the memory module 110, providing a hardware state basis for subsequent in-situ multiply-accumulate operations.
[0119] In sub-step S202, the logic controller 200 applies weighting excitation to the unmasked physical rows, and the storage and computation module 110 completes parallel multiply-accumulate calculations based on the written pixel data and outputs the calculation results.
[0120] The specific process is as follows: After the word line shielding action stabilizes, the logic controller 200 will apply a one-dimensional weight sequence. The numerical values are converted into corresponding analog excitation signals, such as voltage signals with specific amplitudes or pulse widths, and applied to the word lines of the unshielded valid physical rows in the memory module 110 via the input port.
[0121] Within the storage module 110, the storage cells in the unshielded physical rows respond to the analog excitation signal and generate local currents or local charges that characterize the product result using the internally resident pixel data conductivity values.
[0122] The local currents or local charges generated by each valid physical row on the same physical column converge and accumulate along the bitline of the corresponding physical column, thereby completing the in-situ multiplication and summation of the analog domain without reading the pixel data out of the array.
[0123] Subsequently, the peripheral sensing amplification circuit at the column end of the memory-computing module 110 converts the aggregated analog signal into a digital signal and outputs the recognition result of the initial calculation window to the logic controller 200. After the recognition result of the initial calculation window is generated and stored, the logic controller 200 enters step S300 to perform window sliding and offset state updates.
[0124] Reference Figure 5 After the initial recognition result is output, the calculation window slides according to the set window sliding step parameters. When the image data is slid horizontally, the offset tracking module 210 inside the logic controller 200 is activated and tracks the window sliding step size parameter. Calculate the ring write offset. Step S300 specifically includes sub-steps S301 and S302.
[0125] In sub-step S301, the offset tracking module 210 is based on the ring write offset before the update. and window sliding step parameters Perform a modulo operation and obtain the updated circular write offset. .
[0126] Specifically, the offset tracking module 210 reads the currently registered pre-update circular write offset. During the first slide, update the previous circular write offset. The value is 0. The offset tracking module 210 combines the calculation window width. The updated circular write offset of the current window in the physical column space of the storage module 110 is calculated using the following modulo operation formula. :
[0127] ;
[0128] In the formula, Indicates the calculation window width Perform modulo operation and calculate the window width. This is equal to the number of valid physical columns in storage module 110.
[0129] Through modulo operation, the offset tracking module 210 converts the horizontal displacement of the calculation window on the image data into a cyclic address offset within the effective physical column range of the storage module 110.
[0130] The current process generates the updated circular write offset. Updated circular write offset This is used to indicate the physical column address of the first pixel in the current window after the current slide in the storage module 110, and to provide an address basis for subsequently determining the physical column where the overlapping pixel data is located.
[0131] In sub-step S302, the offset tracking module 210 writes the updated offset in a circular fashion after calculating it. At the same time, the overlapping pixel data is determined based on the intersection of the previous calculation window and the current window, and the overlapping pixel data that does not need to be updated is marked in the current window, while the newly added pixel data that slides in is also determined.
[0132] Due to the window sliding step parameter Smaller than the calculation window width The current calculation window is separated from the previous calculation window by a width of [missing value]. The data intersection.
[0133] Offset tracking module 210 writes the offset in a circular pattern before the update. and overlap width The system determines the corresponding ring-shaped physical column address segment in the memory-computing module 110 for the intersection data, and generates overlapping pixel dwell markers for these physical column addresses. The overlapping pixel dwell markers serve as internal marker signals to identify which physical columns in the memory-computing module 110 are still valid and do not need to be erased or rewritten during the current sliding cycle.
[0134] The updated circular write offset formed by the current processing Overwrite the ring write offset state variable The original values in the table are then output along with the overlapping pixel dwell markers to subsequent steps.
[0135] Among them, the overlapping pixel dwell marker and the updated circular write offset The message is sent to step S400, whereby the column write control module 130 generates a column write enable control word and performs a circular partial update. The updated circular write offset is then determined. It is also sent to the shift rearrangement module 220 in step S500 as the offset step number for subsequent mask synchronization shift and weight sequence partial shift.
[0136] Reference Figure 6 Based on the updated circular write offset The logic controller 200 executes step S400, which includes the overlapping pixel dwell markers and the newly added pixel data that slides into the current window as the window slides.
[0137] In step S400, the logic controller 200 adjusts the updated ring write offset according to the updated ring write offset. With the overlapping pixel residency mark, the column write control module 130 closes the write path of the physical column where the overlapping pixel data is located, and writes the newly added pixel data to the unlocked physical column. Step S400 specifically includes sub-steps S401 and S402.
[0138] In sub-step S401, the logic controller 200 adjusts the updated ring write offset. The overlapping pixel residency marker generates a column write enable control word. Based on the column write enable control word, the column write control module 130 locks the physical column storing the overlapping pixel data in the storage module 110 to a write-free state.
[0139] The specific process is as follows: The logic controller 200 converts the ring-shaped physical column address range indicated by the overlapping pixel dwell marker into a bitmask form, and generates a column write enable control word with a bit width equal to the number of effective column channels of the storage and computing module 110, wherein the number of effective column channels of the storage and computing module 110 is equal to the width of the computing window. .
[0140] In the column write enable control word, the bit position corresponding to the overlapping pixel dwell mark is an invalid logic level, such as a low level, while the remaining bits remain at an valid logic level. After receiving the column write enable control word, the column write control module 130 maps the column write enable control word bit by bit to the write enable (WE) pin of each physical column of the memory module 110.
[0141] By using this column write control method, the physical column storing overlapping pixel data closes the write path, thereby avoiding repeated writing of overlapping pixel data and reducing the invalid power consumption and write time overhead caused by SRAM or RRAM write flipping operations.
[0142] After the above processing, the storage module 110 forms a physical column write-locked state, providing write protection for the directional overlay of subsequent newly added pixel data.
[0143] In sub-step S402, the column write control module 130 sequentially overwrites the newly added pixel data that slides into the current window into the physical column of the storage module 110 that is in an unlocked state, thus completing the circular local update of the pixel data in the storage module 110.
[0144] The specific process is as follows: The column write control module 130 generates a local column address pointer corresponding to the current write action internally based on the received new pixel data.
[0145] The column write control module 130 writes the new pixel data into the unlocked physical column where the write enable signal is active, according to the column write enable control word.
[0146] During the continuous writing of S new pixel data, when the local column address pointer is incremented to the maximum physical column boundary of the storage module 110 (i.e., the first...),... When writing to column 0, the column write control module 130 wraps the local column address pointer back to column 0 and continues to perform the write operation.
[0147] By using the above column address wraparound method, the newly added pixel data is circularly overlaid onto the physical column where the expired historical data is located in the storage module 110, while the locked overlapping pixel data remains in the original physical column.
[0148] After the current processing is completed, the pixel data in the storage module 110 forms a locally overlaid updated physical storage offset state. This physical storage offset state is retained in the storage module 110 and serves as the basis for the subsequent step S500 to perform reverse cyclic shift compensation.
[0149] Reference Figure 7 Based on the updated circular write offset and the original one-dimensional weight sequence stored internally. and the original sparse mask sequence The logic controller 200 executes step S500, which compensates for the physical storage offset of pixel data formed after the circular local update by reverse cyclic shifting of the weight sequence part and the mask sequence part, and maintains the masking relationship of the physical row corresponding to zero weight. Step S500 specifically includes sub-steps S501 and S502.
[0150] In sub-step S501, the shift rearrangement module 220 inside the logic controller 200 writes the updated circular offset. The number of shift steps is the number of shift steps for the original one-dimensional weight sequence. The weight sequence part and the original sparse mask sequence in The mask sequence portion is subjected to a reverse cyclic shift operation, and the shifted sequence portion is combined with the remaining unshifted sequence portion to obtain the shift weight sequence. and shift sparse mask sequence .
[0151] The specific process is as follows: the shift rearrangement module 220 writes the updated circular offset. As a basis for control, a corresponding control relationship is established between data column updates and partial shifts in the weight sequence.
[0152] When the calculation window slides over the image data, for example, to the right, and causes a shift in the write position of newly added pixel data in the storage module 110, the shift rearrangement module 220 controls the original one-dimensional weight sequence. The weight sequence portion is moved in the opposite direction to the physical storage offset of the pixel data, with the number of movement steps equal to the updated circular write offset. The values are consistent.
[0153] Meanwhile, the original sparse mask sequence The mask sequence portion follows the original one-dimensional weight sequence. The weight sequence portion performs synchronous cyclic shifts in the same direction and with the same number of steps. The current process generates the shift weight sequence within the logic controller 200. and shift sparse mask sequence Shift weight sequence Used for subsequent analog stimulus transformation, shifted sparse mask sequence Used for subsequent word line shielding.
[0154] In sub-step S502, the row address decoding module 120 determines the row address based on the shifted sparse mask sequence. The read word line enable signal of the corresponding physical row in the storage module 110 is re-shielded.
[0155] Logic controller 200 will shift the weight sequence The operation is applied to the storage module 110, which performs parallel multiplication and addition operations and outputs the recognition result of the current window.
[0156] The specific process is as follows: the logic controller 200 shifts the sparse mask sequence. The data is sent to the control input terminal of the row address decoding module 120. The row address decoding module 120 determines the data based on the shifted sparse mask sequence. The level of each bit is recorded, and the level of the corresponding input pin is updated accordingly.
[0157] Due to the shifted sparse mask sequence With shift weight sequence In synchronous shifting, the mask bits corresponding to zero weights can maintain the same position as the shift weight sequence. The zero-weight positions are consistent. Based on this consistency, the row address decoding module 120 turns off the read word line enable signal of the corresponding physical row to block the pre-charge and discharge of the physical row corresponding to the zero weight.
[0158] After the read line shielding takes effect, the logic controller 200 will shift the weight sequence. It is converted into an analog excitation signal and applied to the unmasked physical row.
[0159] The storage and processing module 110 is based on the pixel data that is physically stored at an offset state, and according to the shift weight sequence. The generated analog excitation signal is used to perform in-situ parallel multiply-accumulate operations.
[0160] The recognition result of the current window is output to an external receiving device or cascaded buffer. Subsequently, the logic controller 200 returns to step S300, continues to update the circular write offset during the next window slide, and performs the next round of circular local update and shift compensation calculation.
[0161] In one implementation, during the continuous sliding calculation process from steps S300 to S500, the logic controller 200 monitors the externally input image horizontal boundary signal. When the image horizontal boundary signal indicates that the calculation window has reached the horizontal boundary of the image data and is preparing to wrap to the next line, the logic controller 200 executes the line wrap reset logic. The logic controller 200 clears the offset status register in the offset tracking module 210 through an internal reset circuit and writes the offset status variable in a circular fashion. The system is restored to its initial value of 0. Simultaneously, the logic controller 200 issues a global unlock command to the column write control module 130 and the row address decoding module 120. According to the global unlock command, the column write control module 130 restores the write enable signals of all physical columns of the storage module 110 to the active state, releasing the column write-free lock. According to the global unlock command, the row address decoding module 120 restores the read word line enable signals of all active physical rows to the enabled state, releasing the row mask. After completing the above reset, the logic controller 200 returns to step S100 to execute the continuous loading of one row of window data and initial state configuration for the initial calculation window in the next image row.
[0162] In one implementation, during the continuous sliding calculation of image data, the logic controller 200 sends a shifted sparse mask sequence to the row address decoding module 120. At the output port, hardware monitoring logic is used to monitor the actual output shift sparse mask sequence. Perform bit-width consistency checks, count of valid logic bits, and mask bit state comparisons. When a shifted sparse mask sequence is detected... The bit width and the original sparse mask sequence Inconsistent bit widths, or shifted sparse mask sequences The number of valid logical bits and the original sparse mask sequence The number of valid logic bits in the output is inconsistent, or the actual output is a sparse shift mask sequence. The mask bit state, and the shift rearrangement module 220 based on the original sparse mask sequence. And the latest updated circular write offset If the obtained mask bit states are inconsistent, it is determined that an offset calculation error or a mask shift mismatch has occurred. The logic controller 200 generates an internal hardware interrupt and suspends the parallel multiply-accumulate calculation operation of the in-memory computing chip 100 in the current cycle. In the calculation suspension state, the logic controller 200 rereads the latest updated circular write offset generated in step S300 from the offset tracking module 210. And control the shift rearrangement module 220 to write the updated circular offset generated in step S300. To determine the correct approach, re-evaluate the original one-dimensional weight sequence. The weight sequence part and the original sparse mask sequence in The mask sequence portion undergoes a reverse cyclic shift operation. The shift rearrangement module 220 combines the re-shifted sequence portion with the remaining unshifted sequence portion to obtain the repaired shift weight sequence. and the repaired shift sparse mask sequence And complete the state synchronization repair of the register data. Repaired shift sparse mask sequence The corrected shift weight sequence was re-distributed to the row address decoding module 120. It is reused for analog excitation conversion. Logic controller 200 completes the shift sparse mask sequence. Reissue and shift weight sequence After the conversion, the hardware interrupt status is lifted, and the suspended word line shielding and parallel multiply-accumulate calculation operations of the memory-accumulate module 110 are restored.
[0163] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these specific embodiments are merely illustrative. Those skilled in the art can omit, substitute, and modify the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, combining the above method steps to perform substantially the same function and achieve substantially the same result using substantially the same method falls within the scope of the present invention.
Claims
1. A method for accelerating image recognition processing based on a memory computing chip, characterized in that, Includes the following steps: Write the image pixel data of the initial calculation window into the physical column of the physical array, initialize the circular write offset, and convert the sparse operation template into a one-dimensional weight sequence and the original sparse mask sequence. The read line enable signals of the corresponding physical rows in the physical array are masked according to the original sparse mask sequence. Multiply-accumulate operations are performed based on the one-dimensional weight sequence and the image pixel data in the unmasked physical rows of the physical array, and the recognition result is output. When the calculation window slides, the updated ring write offset is obtained based on the previous state, and the overlapping pixel data and the newly added pixel data are determined. Based on the updated circular write offset, the physical column storing the overlapping pixel data is locked to a write-free state, and the newly added pixel data is written to the physical column that is in an unlocked state. The one-dimensional weight sequence and the original sparse mask sequence are cyclically shifted using the updated circular write offset to obtain a shifted weight sequence and a shifted sparse mask sequence. The read word line enable signal of the corresponding physical row in the physical array is re-masked using the shifted sparse mask sequence. Multiply-accumulate operations are performed based on the shifted weight sequence and the recognition result is output.
2. The image recognition acceleration processing method based on a memory computing chip according to claim 1, characterized in that, The step of converting the sparse operation template into a one-dimensional weight sequence and the original sparse mask sequence includes: The weight elements in the sparse operation template are continuously spliced together in a fixed row-first scanning order to form the one-dimensional weight sequence, and a corresponding mapping relationship is established between the data bit index in the one-dimensional weight sequence and the pin address of the physical row in the physical array. Read the weight elements in the one-dimensional weight sequence bit by bit, and generate the first logical value when the weight element has zero weight. When the weight element has a non-zero weight, a second logical value is generated; The generated first and second logical values are combined according to the same index positions as the one-dimensional weight sequence to obtain the original sparse mask sequence.
3. The image recognition acceleration processing method based on a memory computing chip according to claim 2, characterized in that, The step of masking the read word line enable signal of the corresponding physical row in the physical array according to the original sparse mask sequence includes: Analyze the bits in the original sparse mask sequence; When the bit in the original sparse mask sequence is parsed to be the first logic value, the read word line enable signal of the corresponding physical row in the physical array is turned off. When the bit in the original sparse mask sequence is parsed to be the second logic value, the read word line enable signal of the corresponding physical row in the physical array is turned on.
4. The image recognition acceleration processing method based on a memory computing chip according to claim 1, characterized in that, The step of obtaining the updated ring write offset based on the previous state includes: Add the circular write offset of the previous state to the window sliding step size to obtain the accumulated result; The accumulated result is subjected to a modulo operation based on the calculation window width, and the result of the modulo operation is used as the updated ring write offset.
5. The image recognition acceleration processing method based on a memory computing chip according to claim 4, characterized in that, The steps for determining overlapping pixel data and newly added pixel data include: The data intersection width is determined based on the window sliding step size and the calculation window width, wherein the data intersection width is the difference between the calculation window width and the window sliding step size; The intersection data is determined based on the circular write offset and the data intersection width of the previous state. The intersection data is identified as the overlapping pixel data, and an overlapping pixel dwell mark is generated for the physical column where the overlapping pixel data is located. The image pixel data that is slid into the calculation window after sliding is determined as the newly added pixel data.
6. The image recognition acceleration processing method based on a memory computing chip according to claim 5, characterized in that, The step of locking the physical column storing the overlapping pixel data to a write-free state based on the updated circular write offset, and writing the newly added pixel data to the unlocked physical column, includes: A column write enable control word is generated based on the updated ring write offset and the overlapping pixel dwell mark. The bit position in the column write enable control word corresponding to the overlapping pixel dwell mark is set to invalid logic level, and the remaining bits in the column write enable control word are kept to valid logic level. The column write enable control word is mapped bit by bit to the write enable pin of each physical column of the physical array, thereby closing the write path of the physical column storing the overlapping pixel data and locking it to the write-free state. The newly added pixel data is written sequentially into the physical columns where the write enable pin is in a valid logic level state, and the physical columns where the write enable pin is in a valid logic level state correspond to the physical columns in an unlocked state.
7. The image recognition acceleration processing method based on a memory computing chip according to claim 6, characterized in that, The step of sequentially writing the newly added pixel data into the physical column where the write enable pin is in a valid logic level state includes: During the continuous writing of the new pixel data, when the column address pointer accumulates to the maximum boundary of the physical column of the physical array, the column address pointer is wrapped back to the first physical column to continue the writing operation.
8. The image recognition acceleration processing method based on a memory computing chip according to claim 1, characterized in that, The step of performing a cyclic shift on the one-dimensional weight sequence and the original sparse mask sequence using the updated circular write offset to obtain a shifted weight sequence and a shifted sparse mask sequence includes: The weight sequence portion of the one-dimensional weight sequence is controlled to perform a cyclic shift in the opposite direction to the physical storage offset generated by the writing of new pixel data. The number of shift steps is equal to the value of the updated cyclic write offset. The shifted weight sequence portion is then combined with the remaining unshifted sequence portion to obtain the shifted weight sequence. The mask sequence portion in the original sparse mask sequence is controlled to perform cyclic shifts in the same direction and with the same number of steps as the weight sequence portion in the one-dimensional weight sequence, and the shifted mask sequence portion is combined with the remaining unshifted sequence portion to obtain the shifted sparse mask sequence.
9. The image recognition acceleration processing method based on a memory computing chip according to claim 1, characterized in that, The method further includes: When an image horizontal boundary signal indicating that the calculation window has reached the horizontal boundary of the image data is detected, the circular write offset is cleared. Restore the write enable signals of all physical columns of the physical array to the valid state; Restore the read word line enable signal of all physical rows of the physical array to the enabled state; Return to perform the initial loading action for the next image row.
10. The image recognition acceleration processing method based on a memory computing chip according to claim 1, characterized in that, The method further includes: Perform bit-width consistency comparison, effective logic bit count comparison, and mask bit state comparison on the actual output shift sparse mask sequence; If any comparison result is inconsistent, a shift anomaly is determined to have occurred, a hardware interrupt signal is generated, and the multiply-accumulate operation of the current cycle is suspended. The updated circular write offset is reread, and the weight sequence portion of the one-dimensional weight sequence and the mask sequence portion of the original sparse mask sequence are re-performed cyclic shift to obtain the repaired shifted sparse mask sequence and the repaired shifted weight sequence. The repaired shift sparse mask sequence is reused to mask the read word line enable signal of the corresponding physical row in the physical array. The repaired shift weight sequence is reused to simulate the excitation conversion, the hardware interrupt state corresponding to the hardware interrupt signal is released, and the suspended multiply-accumulate operation is restored.
