Image recognition method, product, device and medium based on data multiplexing

Abstract

The application discloses an image recognition method, product, equipment and medium based on data multiplexing, and is applied to an image recognition circuit comprising multiple sets of recognition sub-circuits; comprising: storing initial image data of an original image to be recognized in a preset global cache; controlling a convolution window of an image recognition CNN to slide in the initial image data; each recognition sub-circuit corresponds to a different convolution window, and comprises a selector, a data register and a memristor array; the selector is used for acquiring non-multiplexed and multiplexed target data from the preset global cache and a data register corresponding to a previous convolution window respectively; the target data is stored in the data register as convolution data to be convolved of a current convolution window, the convolution data to be convolved is processed by the memristor array to obtain a convolution result of the current convolution window; and a feature map of the original image to be recognized is generated according to the convolution results of the convolution windows to perform image recognition. Transmission delay and hardware power consumption in the image recognition process are reduced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to image recognition methods, products, devices and media based on data reuse. Background Technology

[0002] In the field of image recognition, convolutional neural networks are widely used due to their excellent feature extraction and recognition capabilities. To improve their computational efficiency, dedicated hardware circuits are often used to accelerate convolution operations. Memristor arrays, with their in-memory computing characteristics, have become an important carrier for the hardware implementation of convolution operations. In existing technologies, memristor arrays are often combined with CMOS (Complementary Metal-Oxide-Semiconductor) technology to construct image recognition circuits. Convolution operations are performed by sliding a convolution window on the preprocessed initial image data to extract image features, thereby achieving image recognition.

[0003] When performing convolution operations, such image recognition circuits need to load all window data, resulting in huge bus data transmission volume and excessive bus load. At the same time, frequent global cache access increases data transmission latency and hardware power consumption.

[0004] It is evident that reducing transmission latency and hardware power consumption during image recognition is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide an image recognition method, apparatus, device, and medium based on data reuse, reducing transmission latency and hardware power consumption during the image recognition process. The specific solution is as follows:

[0006] In a first aspect, the present invention discloses an image recognition method based on data multiplexing, applied to an image recognition circuit, the image recognition circuit comprising multiple sets of recognition sub-circuits arranged in a two-dimensional grid; wherein, the method comprises:

[0007] The original image to be identified is preprocessed to generate initial image data in the form of a two-dimensional matrix, and the initial image data is stored in a preset global cache;

[0008] The convolution window of the target convolutional neural network model is controlled to slide in the initial image data; wherein, the target convolutional neural network model is used for image recognition; each of the recognition sub-circuits corresponds to a different convolutional window, and the recognition sub-circuit includes a selector, a data register and a memristor array, and the selector is connected to the preset global cache through a bus;

[0009] The first target data of the current convolutional window is obtained from the preset global cache via the bus using the selector corresponding to the current convolutional window, and the second target data of the current convolutional window is obtained from the data register corresponding to the previous convolutional window; wherein, the first target data and the second target data are respectively non-reused data and reused data between the current convolutional window and the previous convolutional window;

[0010] The first target data and the second target data are used as the data to be convolved in the current convolution window and stored in the data register corresponding to the current convolution window. The data to be convolved is then processed by the memristor array corresponding to the current convolution window to obtain the convolution result of the current convolution window.

[0011] The feature map of the original image to be recognized is generated based on the convolution results of each convolution window, and image recognition is performed based on the feature map to obtain the image recognition result.

[0012] Optionally, the preprocessing of the original image to be identified to generate initial image data in the form of a two-dimensional matrix includes:

[0013] The image size of the original image to be recognized is adjusted based on the preset size requirements to obtain the first preprocessed image;

[0014] The pixel values ​​in the first preprocessed image are normalized to generate initial image data in the form of a two-dimensional matrix based on the obtained normalized pixel values.

[0015] Optionally, the image recognition method based on data reuse further includes:

[0016] The weights of the convolution kernels of each layer in the target convolutional neural network model are stored in the memristor array in the form of electrical conductance;

[0017] Accordingly, the step of performing convolution processing on the data to be convolved using the memristor array corresponding to the current convolution window includes:

[0018] The data to be convolved, stored in the data register corresponding to the current convolution window, is loaded to both ends of the memristor array corresponding to the current convolution window, and the data to be convolved is processed using the convolution kernel weights in the memristor array.

[0019] Optionally, the step of performing convolution processing on the data to be convolved using a memristor array corresponding to the current convolution window to obtain the convolution result of the current convolution window includes:

[0020] The column lines of the memristor array corresponding to the current convolution window are clamped and controlled at a preset reference level, so as to generate current by the interaction between the data to be convolved and the weights of the convolution kernel in the memristor array and the current is collected on the column lines of the memristor array. The convolution result of the current convolution window is determined according to the magnitude of the current collected on the column lines.

[0021] Optionally, determining the convolution result of the current convolution window based on the magnitude of the current collected on the column lines includes:

[0022] The magnitude of the current collected on the column lines is determined as the multiplication and accumulation result; wherein, the multiplication and accumulation result is a current signal;

[0023] The multiplication and accumulation result is converted into a digital signal to obtain the convolution result of the current convolution window.

[0024] Optionally, generating the feature map of the original image to be recognized based on the convolution results of each convolution window includes:

[0025] The feature map size of the original image to be identified is determined based on the size of the initial image data and the kernel size of the target convolutional neural network model.

[0026] Based on the feature map size, establish a mapping relationship between the position of each convolution window and the pixel coordinates of the feature map of the original image to be recognized;

[0027] Based on the mapping relationship, the convolution results of each convolution window are concatenated in the form of a two-dimensional matrix to obtain the feature map of the original image to be identified.

[0028] Optionally, the convolution window of the control target convolutional neural network model slides across the initial image data, including:

[0029] Set the size of the convolution window and the stride of the target convolutional neural network model;

[0030] The convolution window is controlled to alternately flip and slide horizontally across the initial image data.

[0031] Optionally, the number of data registers corresponds to the kernel size of the target convolutional neural network model; the step of using the first target data and the second target data as the data to be convolved in the current convolutional window and storing them in the data register corresponding to the current convolutional window includes:

[0032] Based on the sliding direction of the current convolution window relative to the previous convolution window, the first target data and the second target data are moved to the corresponding target positions in the current convolution window to obtain the data to be convolved in the current convolution window;

[0033] The data to be convolved in the current convolution window is stored in the data register corresponding to the current convolution window.

[0034] Optionally, moving the first target data and the second target data to the corresponding target position in the current convolution window according to the sliding direction of the current convolution window relative to the previous convolution window includes:

[0035] If the sliding direction of the current convolution window relative to the previous convolution window is a horizontal sliding direction to the right, then the second target data is shifted to the left according to the sliding step size to move to the left array position in the current convolution window, and the first target data is moved to the empty position in the current convolution window.

[0036] If the sliding direction of the current convolution window relative to the previous convolution window is a horizontal sliding direction to the left, then the second target data is shifted to the right according to the sliding step size, so as to move to the right array position in the current convolution window, and the first target data is moved to the empty position in the current convolution window.

[0037] Optionally, moving the first target data and the second target data to the corresponding target position in the current convolution window according to the sliding direction of the current convolution window relative to the previous convolution window includes:

[0038] If the sliding direction of the current convolution window relative to the previous convolution window is a downward vertical sliding direction, then the second target data is moved upward according to the sliding step size to the upper array position in the current convolution window, and the first target data is moved to the empty position in the current convolution window.

[0039] If the sliding direction of the current convolution window relative to the previous convolution window is an upward vertical sliding direction, then the second target data is shifted downward according to the sliding step size to the lower array position in the current convolution window, and the first target data is moved to the empty position in the current convolution window.

[0040] Optionally, the step of using the selector corresponding to the current convolution window to obtain the first target data of the current convolution window from the preset global cache and the second target data of the current convolution window from the data register corresponding to the previous convolution window via the bus includes:

[0041] If the current convolution window is the first convolution window, the first target data of the current convolution window is obtained from the preset global cache through the bus using the selector corresponding to the current convolution window, and the second target data of the current convolution window is set to empty;

[0042] If the current convolution window is not the first convolution window, the first target data of the current convolution window is obtained from the preset global cache through the bus using the selector corresponding to the current convolution window, and the positional overlap data between the current convolution window and the previous convolution window is determined as the second target data of the current convolution window. The second target data of the current convolution window is obtained from the data register corresponding to the previous convolution window using the selector corresponding to the current convolution window.

[0043] Optionally, the step of performing image recognition based on the feature map to obtain an image recognition result includes:

[0044] The effective feature regions in the feature map are filtered based on a preset threshold, and invalid feature values ​​at the edges of the feature map are removed to obtain the cropped feature map.

[0045] The cropped feature map is input into the classifier in the target convolutional neural network model to determine the category of the original image to be recognized and output the image recognition result.

[0046] In a second aspect, the present invention discloses a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the aforementioned image recognition method based on data reuse.

[0047] Thirdly, the present invention discloses an electronic device, comprising:

[0048] Memory, used to store computer programs;

[0049] A processor for executing computer programs to implement the steps of the aforementioned disclosed image recognition method based on data reuse.

[0050] Fourthly, the present invention discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the steps of the aforementioned image recognition method based on data reuse.

[0051] Therefore, the present invention is applied to an image recognition circuit, which includes multiple sets of recognition sub-circuits arranged in a two-dimensional grid. The method includes: preprocessing the original image to be recognized to generate initial image data in the form of a two-dimensional matrix, and storing the initial image data in a preset global cache; controlling the convolution window of a target convolutional neural network model to slide within the initial image data; wherein the target convolutional neural network model is used for image recognition; each recognition sub-circuit corresponds to a different convolution window, and each recognition sub-circuit includes a selector, a data register, and a memristor array, the selector being connected to the preset global cache via a bus; and using the selector corresponding to the current convolution window to retrieve data from the preset global cache via the bus. The system retrieves the first target data of the current convolutional window from the cache and the second target data of the current convolutional window from the data register corresponding to the previous convolutional window. The first target data and the second target data are respectively the non-reused data and reused data between the current convolutional window and the previous convolutional window. The first target data and the second target data are used as the data to be convolved in the current convolutional window and stored in the data register corresponding to the current convolutional window. The data to be convolved is then convolved using a memristor array corresponding to the current convolutional window to obtain the convolution result of the current convolutional window. A feature map of the original image to be recognized is generated based on the convolution results of each convolutional window, and image recognition is performed based on the feature map to obtain the image recognition result.

[0052] The beneficial effects are as follows: This invention sets the image recognition circuit as multiple sets of recognition sub-circuits arranged in a two-dimensional grid, with each sub-circuit corresponding to a different convolution window. Each sub-circuit integrates a selector, data register, and memristor array. The selector is connected to a preset global cache storing the initial image data via a bus. This allows the selector corresponding to the current convolution window to directly obtain non-reused first target data from the preset global cache and reused second target data from the data register corresponding to the previous convolution window, without repeatedly loading reused data from the global cache. This significantly reduces the amount of data transmitted on the bus, lowers the bus load and data transmission latency, and reduces the frequency of global cache access, thus improving data retrieval efficiency. The data to be convolved is stored in the corresponding... The data register is used for convolution processing, which is performed by a matching memristor array. Relying on the in-memory computing characteristics of the memristor array, the storage of convolution kernel weights and multiplication-accumulation operations are completed directly at the hardware level. This avoids frequent data transfer between storage and computing units, reduces the hardware power consumption of convolution operations, and improves the speed of single convolution window convolution processing. Through the collaborative completion of convolution processing of all convolution windows and generation of feature maps by various recognition sub-circuits, the entire process of convolution operation from data acquisition, temporary storage to processing is hardware-based and parallelized. This makes the convolution operation of the target convolutional neural network model highly compatible with the hardware architecture of the image recognition circuit, effectively improving the efficiency and accuracy of feature map generation, thereby ensuring the overall efficiency and recognition accuracy of subsequent image recognition. Attached Figure Description

[0053] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 A flowchart of an image recognition method based on data reuse provided in an embodiment of the present invention;

[0055] Figure 2 A specific identification sub-circuit diagram is provided for an embodiment of the present invention;

[0056] Figure 3 A flowchart illustrating a specific image recognition method based on data reuse provided in an embodiment of the present invention;

[0057] Figure 4 A specific convolution operation flowchart is provided for an embodiment of the present invention;

[0058] Figure 5 A specific flowchart for obtaining convolution results is provided in this embodiment of the invention;

[0059] Figure 6 A schematic diagram of the structure of an image recognition device based on data reuse provided in an embodiment of the present invention;

[0060] Figure 7 This is a structural diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0062] In the field of image recognition, convolutional neural networks are widely used due to their excellent feature extraction and recognition capabilities. To improve their computational efficiency, dedicated hardware circuits are often used to accelerate convolution operations. Memristor arrays, with their in-memory computing characteristics, have become an important carrier for the hardware implementation of convolution operations. In existing technologies, memristor arrays are often combined with CMOS technology to construct image recognition circuits. Convolution operations are performed by sliding a convolution window on the preprocessed initial image data to extract image features, thereby achieving image recognition.

[0063] When performing convolution operations, such image recognition circuits need to load all window data, resulting in huge bus data transmission volume and excessive bus load. At the same time, frequent global cache access increases data transmission latency and hardware power consumption.

[0064] The terms "comprising" and "having," and any variations thereof, in the specification and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may include steps or units not listed.

[0065] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] Next, we will describe in detail an image recognition scheme based on data reuse provided by an embodiment of the present invention. Figure 1 An image recognition method based on data reuse, provided in this embodiment of the invention, is applied to an image recognition circuit, the image recognition circuit comprising multiple sets of recognition sub-circuits arranged in a two-dimensional grid; wherein, the method includes:

[0067] Step S11: Preprocess the original image to be identified to generate initial image data in the form of a two-dimensional matrix, and store the initial image data in a preset global cache.

[0068] The original image to be identified is preprocessed and converted into initial image data in the form of a two-dimensional matrix that can be adapted to the convolution calculation of the memristor array. The row and column dimensions of the two-dimensional matrix can be flexibly set according to the actual convolution kernel size and stride requirements. Then, the initial image data in the form of the two-dimensional matrix is ​​stored in a preset global cache to provide data support for data reuse during subsequent convolution window sliding, reduce the frequency of subsequent data loading from the external bus, and reduce data bus load and access latency.

[0069] Step S12: Control the convolution window of the target convolutional neural network model to slide in the initial image data; wherein, the target convolutional neural network model is used for image recognition; each of the recognition sub-circuits corresponds to a different convolutional window, and the recognition sub-circuit includes a selector, a data register and a memristor array, and the selector is connected to the preset global cache through a bus.

[0070] In this embodiment, the method further includes storing the kernel weights of each layer in the target convolutional neural network model in the form of electrical conductance in the memristor array.

[0071] For each convolutional layer of the target convolutional neural network model, the corresponding convolutional kernel weight parameters are extracted. Based on the adjustable and non-volatile characteristics of memristors, these convolutional kernel weight parameters are converted into corresponding conductance values ​​according to a preset mapping relationship. Then, the converted conductance values ​​are written into each memristor cell of the memristor array through an external driving circuit, thus completing the storage of the convolutional kernel weights of each layer in the memristor array. This allows the memristor array to directly use the stored weights to achieve "in-memory computing" multiplication and accumulation operations in subsequent calculations, avoiding the additional power consumption and latency caused by weight data movement in traditional calculations.

[0072] In this embodiment, controlling the convolution window of the target convolutional neural network model to slide in the initial image data includes: setting the size and sliding stride of the convolution window of the target convolutional neural network model; and controlling the convolution window to alternately flip and slide in the initial image data in the horizontal direction.

[0073] Based on the convolution calculation requirements of the target convolutional neural network model, the size of the convolution window and the sliding stride are pre-set to match the memristor array operation specifications. Then, based on the two-dimensional matrix features of the initial image data, the pre-set convolution window is controlled to slide in a horizontally alternating manner within the two-dimensional matrix of the initial image data. During the sliding process, the data reuse characteristics between adjacent convolution windows are fully utilized, avoiding the repeated loading of a large amount of redundant data when using traditional sliding windows, reducing the load pressure on the data bus, and providing accurate and efficient data input support for subsequent multiply-accumulate operations based on the memristor array.

[0074] Based on the image recognition task requirements of the target convolutional neural network model, a convolutional window with a pre-defined size and stride is controlled to slide sequentially through initial image data in the form of a two-dimensional matrix stored in a preset global cache. Each recognition sub-circuit corresponds to a different convolutional window, for example... Figure 2 As shown, each recognition sub-circuit consists of a selector, a data register, and a memristor array. The selector is connected to a preset global cache via a bus, the data register receives data from the selector, and the memristor array receives data from the data register to perform multiplication and accumulation operations, thereby improving the overall efficiency of convolution calculations and reducing power consumption in image recognition tasks.

[0075] Step S13: Using the selector corresponding to the current convolution window, obtain the first target data of the current convolution window from the preset global cache and the second target data of the current convolution window from the data register corresponding to the previous convolution window via the bus; wherein, the first target data and the second target data are respectively the non-reused data and the reused data between the current convolution window and the previous convolution window.

[0076] In this embodiment, the step of using the selector corresponding to the current convolutional window to obtain the first target data of the current convolutional window from the preset global cache via the bus and the second target data of the current convolutional window from the data register corresponding to the previous convolutional window includes: if the current convolutional window is the first convolutional window, then using the selector corresponding to the current convolutional window to obtain the first target data of the current convolutional window from the preset global cache via the bus, and setting the second target data of the current convolutional window to null; if the current convolutional window is not the first convolutional window, then using the selector corresponding to the current convolutional window to obtain the first target data of the current convolutional window from the preset global cache via the bus, determining the positional overlap data between the current convolutional window and the previous convolutional window as the second target data of the current convolutional window, and using the selector corresponding to the current convolutional window to obtain the second target data of the current convolutional window from the data register corresponding to the previous convolutional window.

[0077] Based on the image recognition convolution operation requirements of the target convolutional neural network model, the data required for the operation is obtained by using the selector corresponding to the current convolutional window while controlling the convolution window to slide in the initial image data.

[0078] If the current convolution window is the first convolution window, then there is no previous convolution window. In other words, all image data in the first field of view is acquired for the first time and there is no reused image data. Therefore, it is necessary to use the selector corresponding to the current convolution window to obtain the first target data of the current convolution window from the preset global cache through the bus, and set the second target data of the current convolution window to empty.

[0079] If the current convolution window is not the first convolution window, meaning there is a previous convolution window, then there is overlapping image data between the previous and current convolution windows, which is reusable image data. Therefore, the selector corresponding to the current convolution window is used to retrieve the first target data of the current convolution window from the preset global cache via the bus, and the positional overlap data between the current and previous convolution windows is determined as the second target data of the current convolution window. Then, the selector corresponding to the current convolution window is used to retrieve the second target data from the data register corresponding to the previous convolution window. In this way, data reuse between adjacent convolution windows is achieved, reducing the number of times data is read from the global cache, and reducing the data bus load and computation latency.

[0080] Step S14: The first target data and the second target data are used as the data to be convolved in the current convolution window and stored in the data register corresponding to the current convolution window. The data to be convolved is processed by the memristor array corresponding to the current convolution window to obtain the convolution result of the current convolution window.

[0081] In this embodiment, the number of data registers corresponds to the kernel size of the target convolutional neural network model; the step of using the first target data and the second target data as the data to be convolved in the current convolutional window and storing them in the data register corresponding to the current convolutional window includes: moving the first target data and the second target data to the target position corresponding to the current convolutional window according to the sliding direction of the current convolutional window relative to the previous convolutional window, so as to obtain the data to be convolved in the current convolutional window; and storing the data to be convolved in the current convolutional window in the data register corresponding to the current convolutional window.

[0082] The number of data registers corresponds to the kernel size of the target convolutional neural network model; that is, the number of data registers matches the number of rows and columns of the convolution kernel, ensuring that all the data to be processed within a single convolution window can be fully contained. The first target data and the second target data are used as the data to be convolved in the current convolution window and stored in the data register corresponding to the current convolution window. Based on the sliding direction of the current convolution window relative to the previous convolution window, the first target data obtained from the preset global cache and the second target data obtained from the data register corresponding to the previous convolution window are moved to the corresponding target positions in the current convolution window according to the position matching requirements of the convolution operation. This is used to integrate and form the data to be convolved in the current convolution window. The integrated data to be convolved in the current convolution window is then stored in the data register corresponding to the current convolution window, providing accurate data input for subsequent multiply-accumulate operations based on the memristor array. At the same time, data reuse reduces the frequency of external data loading, thereby reducing data bus load and computational latency.

[0083] In one specific embodiment, moving the first target data and the second target data to the corresponding target position in the current convolutional window according to the sliding direction of the current convolutional window relative to the previous convolutional window includes: if the sliding direction of the current convolutional window relative to the previous convolutional window is a horizontal sliding direction to the right, then the second target data is shifted to the left according to the sliding step to move to the left array position in the current convolutional window, and the first target data is moved to the empty position in the current convolutional window; if the sliding direction of the current convolutional window relative to the previous convolutional window is a horizontal sliding direction to the left, then the second target data is shifted to the right according to the sliding step to move to the right array position in the current convolutional window, and the first target data is moved to the empty position in the current convolutional window.

[0084] If the current convolution window slides horizontally to the right relative to the previous convolution window, the second target data obtained from the corresponding data register of the previous convolution window is shifted to the left according to the preset sliding step, moving it to the left array position in the current convolution window. At the same time, the first target data obtained from the preset global cache via the bus is moved to the empty position created by the sliding of the current convolution window, thus realizing data reuse when sliding horizontally to the right. If the current convolution window slides horizontally to the left relative to the previous convolution window, the second target data obtained from the corresponding data register of the previous convolution window is shifted to the right according to the preset sliding step, moving it to the right array position in the current convolution window. At the same time, the first target data obtained from the preset global cache via the bus is moved to the empty position created by the sliding of the current convolution window. Through this shifting and data filling method, the overlapping data of adjacent convolution windows is fully utilized, the number of times data is loaded from the global cache is reduced, the data bus load and operation latency are reduced, and the requirements of convolution multiplication and accumulation operations based on memristor arrays are adapted.

[0085] In another specific embodiment, the step of moving the first target data and the second target data to the corresponding target position in the current convolutional window according to the sliding direction of the current convolutional window relative to the previous convolutional window includes: if the sliding direction of the current convolutional window relative to the previous convolutional window is a downward vertical sliding direction, then the second target data is moved upward according to the sliding step size to move to the upper array position in the current convolutional window, and the first target data is moved to the empty position in the current convolutional window; if the sliding direction of the current convolutional window relative to the previous convolutional window is an upward vertical sliding direction, then the second target data is moved downward according to the sliding step size to move to the lower array position in the current convolutional window, and the first target data is moved to the empty position in the current convolutional window.

[0086] If the current convolution window slides vertically downwards relative to the previous convolution window, the second target data obtained from the corresponding data register of the previous convolution window is shifted upwards according to a preset sliding step, moving it to the upper array position in the current convolution window. Simultaneously, the first target data obtained from the preset global cache via the bus is moved to the empty position created by the vertical downward sliding of the current convolution window, thus achieving data reuse during vertical downward sliding. If the current convolution window slides vertically upwards relative to the previous convolution window, the second target data obtained from the corresponding data register of the previous convolution window is shifted downwards according to a preset sliding step, moving it to the lower array position in the current convolution window. Simultaneously, the first target data obtained from the preset global cache via the bus is moved to the empty position created by the vertical upward sliding of the current convolution window. Through this vertical shifting and data filling method, the overlapping data of adjacent convolution windows in the vertical direction is fully utilized, reducing the number of times data is loaded from the global cache, reducing data bus load and computational latency, and adapting to the data input requirements of convolution multiplication-accumulation operations based on memristor arrays.

[0087] In this embodiment, the step of performing convolution processing on the data to be convolved using the memristor array corresponding to the current convolution window includes: loading the data to be convolved stored in the data register corresponding to the current convolution window to both ends of the memristor array corresponding to the current convolution window, and performing convolution processing on the data to be convolved using the convolution kernel weights in the memristor array.

[0088] The data to be convolved stored in the data register corresponding to the current convolution window is loaded as a voltage signal onto both ends of the memristor array corresponding to the current convolution window. This ensures that the conductance and voltage of the data to be convolved are matched with the pre-fixed convolution kernel weights in the memristor array. Relying on the in-memory computing characteristic of the memristor array, the data to be convolved and the convolution kernel weights are multiplied and accumulated through the conductivity of the memristor. This completes the convolution processing of the loaded data to be convolved and outputs the convolution operation result of the current convolution window.

[0089] In this embodiment, the step of performing convolution processing on the data to be convolved using the memristor array corresponding to the current convolution window to obtain the convolution result of the current convolution window includes: clamping and controlling the column lines of the memristor array corresponding to the current convolution window at a preset reference level, so as to generate current by the interaction between the data to be convolved and the weights of the convolution kernel in the memristor array and the current is collected on the column lines of the memristor array, and determining the convolution result of the current convolution window based on the magnitude of the current collected on the column lines.

[0090] The column lines of the memristor array corresponding to the current convolution window are clamped and controlled at a preset reference level, so that the column line potential of the memristor array remains stable and in a preset reference state. This provides a stable potential environment for the electrical signal interaction between the data to be convolved and the convolution kernel weights. When the data to be convolved is loaded onto the row lines of the memristor array in the form of a voltage signal, the data to be convolved in this voltage form interacts with the pre-fixed convolution kernel weights stored in the memristor array in the form of conductance. According to Ohm's law, a current matching the result of the interaction is generated on each memristor device in the memristor array. This current is collected and superimposed on the column lines of the memristor array clamped at the preset reference level, forming the total column line current corresponding to the multiplication and accumulation result of the convolution operation in the current convolution window. By detecting and collecting the specific magnitude of the total current collected on each column line of the memristor array, and combining the preset mapping relationship between the current magnitude and the convolution operation result, the current signal is converted into a digital feature value, thereby accurately determining the convolution result corresponding to the current convolution window.

[0091] In this embodiment, determining the convolution result of the current convolution window based on the magnitude of the current collected on the column lines includes: determining the magnitude of the current collected on the column lines as a multiplication-accumulation result; wherein the multiplication-accumulation result is a current signal; and converting the multiplication-accumulation result into a digital signal to obtain the convolution result of the current convolution window.

[0092] The total current collected on the column lines of the memristor array is directly determined as the multiplication and accumulation result required for the convolution operation. This multiplication and accumulation result exists in the form of an analog current signal. Its current amplitude has a precise linear correspondence with the value accumulated after element-wise multiplication of the data to be convolved and the weights of the convolution kernel. It can truly reflect the multiplication and accumulation calculation result of the convolution operation under the current convolution window. The analog current form of the multiplication and accumulation result is processed by the digital-to-analog converter module to convert the continuously changing current signal into a discrete digital signal. During the conversion process, the signal is sampled, quantized and encoded to eliminate the slight noise interference introduced by the hardware operation and ensure the accuracy of the signal conversion. Finally, the convolution result of the current convolution window is presented in the form of digital feature values. This result can be directly used for subsequent feature map generation and image recognition processing.

[0093] Step S15: Generate a feature map of the original image to be recognized based on the convolution results of each convolution window, and perform image recognition based on the feature map to obtain the image recognition result.

[0094] In this embodiment, generating the feature map of the original image to be recognized based on the convolution results of each convolution window includes: determining the feature map size of the original image to be recognized based on the size of the initial image data and the kernel size of the target convolutional neural network model; establishing a mapping relationship between the position of each convolution window and the pixel coordinates of the feature map of the original image to be recognized based on the feature map size; and concatenating the convolution results of each convolution window in a two-dimensional matrix form according to the mapping relationship to obtain the feature map of the original image to be recognized.

[0095] Based on the row and column dimensions of the initial image data in the form of a two-dimensional matrix stored in a preset global cache, and combined with the convolution kernel size stored in the memristor array in the form of conductance in the target convolutional neural network model, the size of the feature map generated after the original image to be recognized is calculated and determined. Then, based on the feature map size, a one-to-one correspondence mapping relationship is established between the sliding position of each convolution window on the initial image data and the pixel coordinates of the feature map of the original image to be recognized. Finally, according to this mapping relationship, the convolution results of each convolution window output after the memristor array completes the multiplication and accumulation operation are stitched together in an ordered two-dimensional matrix form to obtain the feature map of the original image to be recognized. This ensures that the pixel arrangement of the feature map corresponds precisely to the operation position of the convolution window, thereby improving the accuracy of subsequent image recognition tasks.

[0096] Based on the feature map obtained by splicing in the form of a two-dimensional matrix, it is input into the subsequent network layer of the target convolutional neural network model for processing. The subsequent network layer uses the "in-memory computing" characteristic of the memristor array to complete further feature extraction and data operation of the feature map. The results obtained through layer-by-layer operation realize the classification or detection of the original image to be recognized, and finally obtain the corresponding image recognition result.

[0097] Therefore, the present invention is applied to an image recognition circuit, which includes multiple sets of recognition sub-circuits arranged in a two-dimensional grid. The method includes: preprocessing the original image to be recognized to generate initial image data in the form of a two-dimensional matrix, and storing the initial image data in a preset global cache; controlling the convolution window of a target convolutional neural network model to slide within the initial image data; wherein the target convolutional neural network model is used for image recognition; each recognition sub-circuit corresponds to a different convolution window, and each recognition sub-circuit includes a selector, a data register, and a memristor array, the selector being connected to the preset global cache via a bus; and using the selector corresponding to the current convolution window to retrieve data from the preset global cache via the bus. The system retrieves the first target data of the current convolutional window from the cache and the second target data of the current convolutional window from the data register corresponding to the previous convolutional window. The first target data and the second target data are respectively the non-reused data and reused data between the current convolutional window and the previous convolutional window. The first target data and the second target data are used as the data to be convolved in the current convolutional window and stored in the data register corresponding to the current convolutional window. The data to be convolved is then convolved using a memristor array corresponding to the current convolutional window to obtain the convolution result of the current convolutional window. A feature map of the original image to be recognized is generated based on the convolution results of each convolutional window, and image recognition is performed based on the feature map to obtain the image recognition result.Therefore, this invention sets the image recognition circuit as multiple recognition sub-circuits arranged in a two-dimensional grid, with each sub-circuit corresponding to a different convolution window. Each sub-circuit integrates a selector, data register, and memristor array. The selector is connected to a preset global cache storing the initial image data via a bus. This allows the selector corresponding to the current convolution window to directly retrieve non-reused first target data from the preset global cache and reused second target data from the data register corresponding to the previous convolution window, eliminating the need to repeatedly load reused data from the global cache. This significantly reduces the amount of data transmitted on the bus, lowers bus load and data transmission latency, and reduces the frequency of global cache access, thus improving data retrieval efficiency. The data to be convolved is stored in the corresponding... The convolution process is performed in the data register by a matching memristor array. Leveraging the in-memory computing capability of the memristor array, the storage of convolution kernel weights and multiplication-accumulation operations are performed directly at the hardware level. This avoids frequent data transfer between storage and computing units, reduces the hardware power consumption of convolution operations, and improves the speed of single convolution window processing. Through the collaborative efforts of various recognition sub-circuits, the convolution processing of all convolution windows is completed and feature maps are generated. This achieves full hardware-based and parallel processing of the convolution operation from data acquisition, temporary storage to processing. This ensures that the convolution operation of the target convolutional neural network model is highly compatible with the hardware architecture of the image recognition circuit, effectively improving the efficiency and accuracy of feature map generation, thereby guaranteeing the overall efficiency and recognition accuracy of subsequent image recognition.

[0098] See Figure 3 This invention discloses a specific image recognition method based on data reuse. Compared to the previous embodiment, this embodiment further explains and optimizes the technical solution. It includes:

[0099] Step S21: Adjust the image size of the original image to be recognized based on the preset size requirements to obtain the first preprocessed image; normalize the pixel values ​​in the first preprocessed image to generate initial image data in the form of a two-dimensional matrix based on the obtained normalized pixel values, and store the initial image data in a preset global cache.

[0100] Based on the preset size requirements of the target convolutional neural network model adapted to the convolutional computation of the memristor array, the original image to be identified is scaled or cropped to adjust its image size, resulting in a first preprocessed image that meets the computational specifications. Then, the pixel values ​​of each pixel in the first preprocessed image are normalized to map the pixel values ​​to a preset numerical range, eliminating the influence of differences in the magnitude of different pixel values ​​on subsequent convolutional operations. Subsequently, based on the obtained normalized pixel values, initial image data in the form of a two-dimensional matrix that can be directly called by the memristor array is generated, and the initial image data is stored in a preset global cache, providing stable and efficient data support for data reuse and multiplication-accumulation operations during subsequent convolution window sliding, reducing the frequency and latency of remote data movement.

[0101] Step S22: Control the convolution window of the target convolutional neural network model to slide in the initial image data; wherein, the target convolutional neural network model is used for image recognition; each of the recognition sub-circuits corresponds to a different convolutional window, and the recognition sub-circuit includes a selector, a data register and a memristor array, and the selector is connected to the preset global cache through a bus.

[0102] For example Figure 4 As shown, the size of the convolutional window and the stride of the target convolutional neural network model are set. For example, the initial image data is 6×6, the kernel size is 3×3, the convolutional window size is 3×3, and the stride is 1. The convolutional window is controlled to alternately flip and slide horizontally in the initial image data. Specifically, the first convolutional window corresponds to the image data 0, 1, 2, 6, 7, 8, 12, 13, 14, and slides horizontally to the right by 1 stride. The second convolutional window corresponds to the image data 1, 2, 3, 7, 8, 9, 13, 14, 15, and slides horizontally to the right by 1 stride. With a stride of 1, the image data corresponding to the third convolution window are 3, 4, 5, 9, 10, 11, 15, 16, and 17. Since it is no longer possible to slide horizontally to the right at this point, it needs to slide vertically downwards by 1 stride. The image data corresponding to the fourth convolution window are 9, 10, 11, 15, 16, 17, 21, 22, and 23. Then, it slides horizontally to the left by 1 stride. The image data corresponding to the fifth convolution window are 8, 9, 10, 14, 15, 16, 20, 21, and 22, and so on. With the sliding of windows in one layer, there are a total of 16 convolution windows.

[0103] Step S23: Using the selector corresponding to the current convolution window, obtain the first target data of the current convolution window from the preset global cache and the second target data of the current convolution window from the data register corresponding to the previous convolution window via the bus; wherein, the first target data and the second target data are respectively the non-reused data and the reused data between the current convolution window and the previous convolution window.

[0104] like Figure 4 As shown, there is overlapping image data between the first and second convolutional windows (i.e., image data 1, 2, 7, 8, 13, 14 is reused), and there is overlapping image data between the second and third convolutional windows (i.e., image data 2, 3, 8, 9, 14, 15 is reused). Therefore, if the current convolutional window is the first convolutional window, the first target data of the current convolutional window is obtained from the preset global cache via the bus using the selector corresponding to the current convolutional window, and the second target data of the current convolutional window is set to empty. If the current convolutional window is not the first convolutional window, the current convolutional window is obtained from the preset global cache via the bus using the selector corresponding to the current convolutional window. The first target data of the window is determined, and the overlapping data between the current convolution window and the previous convolution window is determined as the second target data of the current convolution window. For example, the multiplexed image data 1, 2, 7, 8, 13, 14 between the first and second convolution windows, and the multiplexed image data 2, 3, 8, 9, 14, 15 between the second and third convolution windows. Then, the second target data is obtained from the data register corresponding to the previous convolution window using the selector corresponding to the current convolution window. This realizes data reuse between adjacent convolution windows, reduces the number of times data is read from the global cache, reduces the data bus load, and adapts to the convolution multiplication and accumulation operation requirements based on memristor arrays.

[0105] When retrieving the second target data of the current convolution window (i.e., the overlapping data between the previous and current convolution windows) from the data register corresponding to the previous convolution window, the overlapping data can be first subjected to lightweight compression processing. Specifically, run-length encoding can be used to encode and compress the continuously repeating image data. The compressed data is then transmitted through the internal data path to the selector corresponding to the current convolution window. The data is then restored through the matching fast decompression logic. This reduces the number of bits of multiplexed data transmitted between registers, reduces the transmission load and latency of the internal data path, and reduces power consumption during data transmission. Moreover, this compressed transmission method does not destroy the integrity of the multiplexed data, ensuring the accuracy of subsequent convolution multiplication and accumulation operations based on the memristor array.

[0106] Step S24: The first target data and the second target data are used as the data to be convolved in the current convolution window and stored in the data register corresponding to the current convolution window. The data to be convolved is processed by the memristor array corresponding to the current convolution window to obtain the convolution result of the current convolution window.

[0107] If the current convolutional window slides horizontally to the right relative to the previous convolutional window, the second target data is shifted to the left by a preset sliding step, moving it to the left array position in the current convolutional window, and the first target data is moved to the empty position created by the sliding of the current convolutional window; if the current convolutional window slides horizontally to the left relative to the previous convolutional window, the second target data is shifted to the right by a preset sliding step, moving it to the right array position in the current convolutional window, and the first target data is moved to the empty position created by the sliding of the current convolutional window; if the current convolutional window slides vertically downwards relative to the previous convolutional window, the second target data is shifted to the right by a preset sliding step. The target data is shifted upwards to the upper array position in the current convolution window, and the first target data is moved to the empty position created by the sliding of the current convolution window. If the sliding direction of the current convolution window relative to the previous convolution window is vertical upwards, the second target data is shifted downwards according to a preset sliding step to the lower array position in the current convolution window, and the first target data is moved to the empty position created by the sliding of the current convolution window. Through this multi-directional shifting and data filling strategy, the overlapping data of adjacent convolution windows is fully utilized to achieve reuse, reducing the frequency of loading data from the global cache, reducing data bus load and computational latency, and adapting to the convolution multiplication and accumulation operation requirements based on memristor arrays.

[0108] For example Figure 4 As shown, the image data 1, 2, 7, 8, 13, 14 are reused between the first and second convolutional windows. That is, the second target data of the second convolutional window is 1, 2, 7, 8, 13, 14. Because the second convolutional window slides horizontally to the right by 1 step relative to the first convolutional window, the second target data 1, 2, 7, 8, 13, 14 are shifted to the left by 1 step, so that they are moved to the left array position in the second convolutional window.

[0109] Step S25: Generate a feature map of the original image to be recognized based on the convolution results of each convolution window, and perform image recognition based on the feature map to obtain the image recognition result.

[0110] In this embodiment, the step of performing image recognition based on the feature map to obtain image recognition results includes: filtering effective feature regions in the feature map based on a preset threshold and removing invalid feature values ​​at the edges of the feature map to obtain a cropped feature map; inputting the cropped feature map into the classifier in the target convolutional neural network model to determine the category of the original image to be recognized, and outputting the image recognition results.

[0111] Based on a preset feature validity threshold, the spliced ​​two-dimensional matrix feature map is filtered to accurately locate and extract the effective feature regions. At the same time, invalid feature values ​​generated by the boundary effect of convolution operation at the edge of the feature map are removed, thus obtaining a cropped feature map that is denoised and focuses on the core features. Then, the cropped feature map is input into the classifier in the target convolutional neural network model. The classifier combines the feature data obtained by the memristor array to determine the category, and finally determines the category of the original image to be recognized and outputs the corresponding image recognition result.

[0112] This invention utilizes a circuit composed of a selector, a data register, and a memristor array to reuse overlapping data between adjacent convolutional windows. This avoids the problem of repeatedly loading all data from the global cache with each window sliding, as is common in traditional methods. This significantly reduces the amount of remote data movement and the number of times data is accessed from the global cache, lowering the load on the data bus. Simultaneously, the "in-memory computing" characteristic of the memristor array allows weight storage and multiplication-accumulation operations to be completed simultaneously, eliminating the additional power consumption and latency caused by data movement. This effectively improves the computational efficiency of convolutional neural network models in tasks such as image recognition. Preprocessing stages, including image resizing and pixel value normalization, as well as subsequent effective region filtering and edge invalid value removal of feature maps, further optimize input data quality and feature extraction accuracy, ensuring the accuracy of image recognition results. The overall solution is adapted to the hardware characteristics of the memristor array, providing an efficient and low-power implementation path for hardware acceleration of neural network models.

[0113] The following is based on Figure 5 The invention will be described using an example. The image recognition circuit includes multiple sets of recognition sub-circuits arranged in a two-dimensional grid. Each recognition sub-circuit includes a selector, a data register, and a memristor array. The selector is connected to a preset global cache via a bus. The weights of the convolutional kernels of each layer in the target convolutional neural network model used for image recognition are stored in the memristor array in the form of conductance. The size of the convolutional window and the stride of the target convolutional neural network model are set. Each recognition sub-circuit corresponds to a different convolutional window. The number of data registers corresponds to the size of the convolutional kernels of the target convolutional neural network model. The original image to be recognized is preprocessed to generate initial image data in the form of a two-dimensional matrix, and the initial image data is stored in the preset global cache.

[0114] The convolutional window is controlled to alternately flip and slide horizontally in the initial image data. Using a selector corresponding to the current convolutional window, the first target data (i.e., non-reused data between the current and previous convolutional windows) is retrieved from a preset global buffer via a bus, and the second target data (i.e., reused data between the current and previous convolutional windows) is retrieved from the data register corresponding to the previous convolutional window. In one specific embodiment, if the current convolutional window slides horizontally to the right relative to the previous convolutional window, then the second target data is the right-hand array data of the previous convolutional window. In another specific embodiment, if the current convolutional window slides horizontally downwards relative to the previous convolutional window, then the second target data is the lower array data of the previous convolutional window.

[0115] The first target data and the second target data are used as the data to be convolved in the current convolution window and stored in the data register corresponding to the current convolution window. The memristor array corresponding to the current convolution window is used to perform convolution processing on the data to be convolved to obtain the convolution result of the current convolution window. In one specific embodiment, if the current convolution window slides horizontally to the right relative to the previous convolution window, then the second target data needs to be shifted to the left, that is, the second target data is stored in the left register of the data register. In another specific embodiment, if the current convolution window slides horizontally downwards relative to the previous convolution window, then the second target data needs to be shifted upwards, that is, the second target data is stored in the upper left register of the data register.

[0116] The convolution window is repeatedly slid, and the data to be convolved in each convolution window is acquired during the sliding process. Convolution processing is performed on each data to be convolved, that is, multiplication and accumulation operation, until the convolution operation is completed.

[0117] The feature map of the original image to be recognized is generated based on the convolution results of each convolution window, and image recognition is performed based on the feature map to obtain the image recognition result.

[0118] Figure 6 This is a schematic diagram of the structure of an image recognition device based on data multiplexing, provided in an embodiment of the present invention. The device is applied to an image recognition circuit, which includes multiple sets of recognition sub-circuits arranged in a two-dimensional grid. The method includes:

[0119] The preprocessing module 11 is used to preprocess the original image to be identified to generate initial image data in the form of a two-dimensional matrix, and store the initial image data in a preset global cache;

[0120] The window sliding module 12 is used to control the sliding of the convolution window of the target convolutional neural network model in the initial image data; wherein, the target convolutional neural network model is used for image recognition; each of the recognition sub-circuits corresponds to a different convolutional window, and the recognition sub-circuit includes a selector, a data register and a memristor array, and the selector is connected to the preset global cache through a bus;

[0121] The data acquisition module 13 is used to acquire the first target data of the current convolutional window from the preset global cache and the second target data of the current convolutional window from the data register corresponding to the previous convolutional window via the bus using a selector corresponding to the current convolutional window; wherein the first target data and the second target data are respectively non-reused data and reused data between the current convolutional window and the previous convolutional window;

[0122] The convolution module 14 is used to take the first target data and the second target data as the data to be convolved in the current convolution window and store them in the data register corresponding to the current convolution window, and use the memristor array corresponding to the current convolution window to perform convolution processing on the data to be convolved to obtain the convolution result of the current convolution window.

[0123] The image recognition module 15 is used to generate a feature map of the original image to be recognized based on the convolution result of each convolution window, and to perform image recognition based on the feature map to obtain the image recognition result.

[0124] This invention is applied to an image recognition circuit comprising multiple sets of two-dimensional grid-arranged recognition sub-circuits. A preprocessing module preprocesses the original image to be recognized, generating initial image data in the form of a two-dimensional matrix, which is then stored in a preset global cache. This provides a standardized and easily accessible data foundation for subsequent calculations. A window sliding module controls the sliding of the convolutional window of the target convolutional neural network model within the initial image data. Each recognition sub-circuit corresponds to a different convolutional window and integrates a selector, data register, and memristor array. The selector is connected to the global cache bus to ensure smooth data transmission. The data acquisition module uses the selector to retrieve non-reused first target data from the global cache and reused second target data from the data register corresponding to the previous convolutional window. This fully leverages the overlapping data characteristics of adjacent convolutional windows, significantly reducing the number of times data is read from the global cache. To reduce data bus load and data transmission latency, the convolution module integrates two types of target data into convolution data and stores it in the corresponding data register. Leveraging the "in-memory computing" characteristic of the memristor array, it directly completes the convolution process, eliminating the step of moving weight data in traditional operations, reducing power consumption and improving convolution processing efficiency. The image recognition module generates feature maps based on the convolution results of each convolution window and completes image recognition, ensuring the accuracy of the recognition results. The overall solution achieves efficient data reuse and hardware architecture adaptation through the collaborative cooperation between modules. It not only improves the computational efficiency of the image recognition circuit, reduces system power consumption and latency, but also optimizes the data processing flow. It provides an efficient and feasible implementation solution for hardware acceleration of neural networks based on memristor arrays, and has significant performance advantages in artificial intelligence applications such as image recognition.

[0125] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the data multiplexing-based image recognition method disclosed in any of the foregoing embodiments. Furthermore, the electronic device in this embodiment may specifically be an electronic computer.

[0126] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0127] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0128] The operating system 221 is used to manage and control the various hardware devices on the electronic device and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the data reuse-based image recognition method executed by the electronic device as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0129] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned image recognition method based on data reuse. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0130] Furthermore, embodiments of this application also disclose a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the image recognition method based on data reuse disclosed in any of the foregoing embodiments.

[0131] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0132] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0134] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image recognition method based on data reuse, characterized in that, The method is applied to an image recognition circuit, which includes multiple sets of recognition sub-circuits arranged in a two-dimensional grid; wherein the method includes: The original image to be identified is preprocessed to generate initial image data in the form of a two-dimensional matrix, and the initial image data is stored in a preset global cache; The convolution window of the target convolutional neural network model is controlled to slide in the initial image data; wherein, the target convolutional neural network model is used for image recognition; each of the recognition sub-circuits corresponds to a different convolutional window, and the recognition sub-circuit includes a selector, a data register and a memristor array, and the selector is connected to the preset global cache through a bus; The first target data of the current convolutional window is obtained from the preset global cache via the bus using the selector corresponding to the current convolutional window, and the second target data of the current convolutional window is obtained from the data register corresponding to the previous convolutional window; wherein, the first target data and the second target data are respectively non-reused data and reused data between the current convolutional window and the previous convolutional window; The first target data and the second target data are stored as the data to be convolved in the current convolution window in the data register corresponding to the current convolution window. The data to be convolved is processed by the memristor array corresponding to the current convolution window to obtain the convolution result of the current convolution window. The feature map of the original image to be recognized is generated based on the convolution result of each convolution window, and image recognition is performed based on the feature map to obtain the image recognition result; Obtain the second target data of the current convolution window from the data register corresponding to the previous convolution window, including: Run-length encoding is used to encode and compress the continuously repeating image data in the second target data of the current convolution window in the data register corresponding to the previous convolution window. The compressed data is then transmitted through the internal data path to the selector corresponding to the current convolution window. The data is then restored through fast decompression logic to obtain the second target data of the current convolution window.

2. The image recognition method based on data reuse according to claim 1, characterized in that, The preprocessing of the original image to be identified to generate initial image data in the form of a two-dimensional matrix includes: The image size of the original image to be recognized is adjusted based on the preset size requirements to obtain the first preprocessed image; The pixel values ​​in the first preprocessed image are normalized to generate initial image data in the form of a two-dimensional matrix based on the obtained normalized pixel values.

3. The image recognition method based on data reuse according to claim 1, characterized in that, Also includes: The weights of the convolution kernels of each layer in the target convolutional neural network model are stored in the memristor array in the form of electrical conductance; Accordingly, the step of performing convolution processing on the data to be convolved using the memristor array corresponding to the current convolution window includes: The data to be convolved, stored in the data register corresponding to the current convolution window, is loaded to both ends of the memristor array corresponding to the current convolution window, and the data to be convolved is processed using the convolution kernel weights in the memristor array.

4. The image recognition method based on data reuse according to claim 3, characterized in that, The step of performing convolution processing on the data to be convolved using a memristor array corresponding to the current convolution window to obtain the convolution result of the current convolution window includes: The column lines of the memristor array corresponding to the current convolution window are clamped and controlled at a preset reference level, so as to generate current by the interaction between the data to be convolved and the weights of the convolution kernel in the memristor array and the current is collected on the column lines of the memristor array. The convolution result of the current convolution window is determined according to the magnitude of the current collected on the column lines.

5. The image recognition method based on data reuse according to claim 4, characterized in that, Determining the convolution result of the current convolution window based on the magnitude of the current collected on the column lines includes: The magnitude of the current collected on the column lines is determined as the multiplication and accumulation result; wherein, the multiplication and accumulation result is a current signal; The multiplication and accumulation result is converted into a digital signal to obtain the convolution result of the current convolution window.

6. The image recognition method based on data reuse according to claim 1, characterized in that, The step of generating the feature map of the original image to be recognized based on the convolution results of each convolution window includes: The feature map size of the original image to be identified is determined based on the size of the initial image data and the kernel size of the target convolutional neural network model. Based on the feature map size, establish a mapping relationship between the position of each convolution window and the pixel coordinates of the feature map of the original image to be recognized; Based on the mapping relationship, the convolution results of each convolution window are concatenated in the form of a two-dimensional matrix to obtain the feature map of the original image to be identified.

7. The image recognition method based on data reuse according to claim 1, characterized in that, The convolution window of the control target convolutional neural network model slides across the initial image data, including: Set the size of the convolution window and the stride of the target convolutional neural network model; The convolution window is controlled to alternately flip and slide horizontally across the initial image data.

8. The image recognition method based on data reuse according to claim 7, characterized in that, The number of data registers corresponds to the kernel size of the target convolutional neural network model; The step of storing the first target data and the second target data as the data to be convolved in the current convolution window in the data register corresponding to the current convolution window includes: Based on the sliding direction of the current convolution window relative to the previous convolution window, the first target data and the second target data are moved to the corresponding target positions in the current convolution window to obtain the data to be convolved in the current convolution window; Store the data to be convolved in the current convolution window in the data register corresponding to the current convolution window.

9. The image recognition method based on data reuse according to claim 8, characterized in that, The step of moving the first target data and the second target data to the corresponding target position in the current convolution window according to the sliding direction of the current convolution window relative to the previous convolution window includes: If the sliding direction of the current convolution window relative to the previous convolution window is a horizontal sliding direction to the right, then the second target data is shifted to the left according to the sliding step size to move to the left array position in the current convolution window, and the first target data is moved to the empty position in the current convolution window. If the sliding direction of the current convolution window relative to the previous convolution window is a horizontal sliding direction to the left, then the second target data is shifted to the right according to the sliding step size, so as to move to the right array position in the current convolution window, and the first target data is moved to the empty position in the current convolution window.

10. The image recognition method based on data reuse according to claim 8, characterized in that, The step of moving the first target data and the second target data to the corresponding target position in the current convolution window according to the sliding direction of the current convolution window relative to the previous convolution window includes: If the sliding direction of the current convolution window relative to the previous convolution window is a downward vertical sliding direction, then the second target data is moved upward according to the sliding step size to the upper array position in the current convolution window, and the first target data is moved to the empty position in the current convolution window. If the sliding direction of the current convolution window relative to the previous convolution window is an upward vertical sliding direction, then the second target data is shifted downward according to the sliding step size to the lower array position in the current convolution window, and the first target data is moved to the empty position in the current convolution window.

11. The image recognition method based on data reuse according to claim 1, characterized in that, The step of retrieving the first target data of the current convolutional window from the preset global cache via the bus using the selector corresponding to the current convolutional window and retrieving the second target data of the current convolutional window from the data register corresponding to the previous convolutional window includes: If the current convolution window is the first convolution window, the first target data of the current convolution window is obtained from the preset global cache through the bus using the selector corresponding to the current convolution window, and the second target data of the current convolution window is set to empty; If the current convolution window is not the first convolution window, the first target data of the current convolution window is obtained from the preset global cache through the bus using the selector corresponding to the current convolution window, and the positional overlap data between the current convolution window and the previous convolution window is determined as the second target data of the current convolution window. The second target data of the current convolution window is obtained from the data register corresponding to the previous convolution window using the selector corresponding to the current convolution window.

12. The image recognition method based on data reuse according to claim 1, characterized in that, The image recognition based on the feature map to obtain the image recognition result includes: The effective feature regions in the feature map are filtered based on a preset threshold, and invalid feature values ​​at the edges of the feature map are removed to obtain the cropped feature map. The cropped feature map is input into the classifier in the target convolutional neural network model to determine the category of the original image to be recognized and output the image recognition result.

13. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the image recognition method based on data reuse as described in any one of claims 1 to 12.

14. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the image recognition method based on data multiplexing as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image recognition method based on data reuse as described in any one of claims 1 to 12.

Images