Feature extraction system and data sparsification method for target detection

Abstract

The invention discloses a feature extraction system for target detection and a data sparsification method, and belongs to the technical field of electronic information. The system comprises an on-chip feature extraction module, an on-chip threshold extraction module and an off-chip detection and identification module. The on-chip feature extraction module performs feature extraction on an input image based on a convolution kernel to generate multi-scale feature map data; the on-chip threshold extraction module dynamically generates a sparse threshold according to the statistical characteristics of the current image or the feature map, rejects elements of which the absolute values are smaller than the threshold in the feature map, and encodes and compresses reserved non-zero features; and the off-chip detection and identification module receives the sparse features, and outputs tasks such as target detection or classification and the like. According to the method, feature screening and channel slimming are completed in a chip, the data size transmitted out of the chip and the back-end calculation burden are reduced, high target detection accuracy can still be obtained under the conditions of low bandwidth and low power consumption, and a complex target detection algorithm can be efficiently operated on resource-limited platforms such as an FPGA (Field Programmable Gate Array).

Description

Technical Field

[0001] This invention relates to a feature extraction system and a data sparsification method for target detection, belonging to the field of electronic information technology. Background Technology

[0002] With the rapid development of artificial intelligence and deep learning technologies, object detection methods based on Convolutional Neural Networks (CNNs) have been widely applied in scenarios such as autonomous driving, video surveillance, industrial inspection, and remote sensing recognition. CNNs learn and extract spatial features of images through layers of convolution, and then use these features to identify and locate target objects. These methods typically require multi-scale feature extraction and complex post-processing of high-resolution images, resulting in a large number of model parameters and computational demands, placing high demands on computing resources and storage bandwidth.

[0003] On server or GPU platforms, high computing power and large bandwidth can support real-time inference of complex network structures; however, on embedded devices, mobile terminals and edge computing platforms, due to limitations in on-chip storage capacity, computing resources and off-chip bandwidth, direct deployment of mainstream target detection networks often leads to excessive latency or power consumption, making it difficult to meet the requirements of real-time performance and low power consumption.

[0004] Feature sparsity, as an effective model compression and acceleration strategy, can significantly reduce storage and computational overhead by identifying and removing redundant features. Existing methods often apply sparsity regularization or pruning operations during offline training, failing to dynamically adjust the sparsity strategy based on the feature distribution of each input image, making it difficult to dynamically adapt to the feature differences of each image. This is because dynamically adjusting the sparsity strategy requires real-time computation of the feature distribution of each input image and making decisions based on this distribution, which requires additional computation, increasing computational costs. Furthermore, dynamically adjusting the sparsity strategy requires complex training mechanisms and hardware support; otherwise, it will severely impact model performance. In addition, the irregularities and unpredictability introduced by on-chip computation can lead to problems with the computational kernel. The parallelism and pipeline efficiency of the chip are significantly reduced. Irregular sparse data streams lead to uneven hardware load and reduce the actual utilization of hardware computing power. During the sparsification process, in addition to storing the non-zero values ​​themselves, their position information must also be stored. After reading the compressed data, a dedicated decoding circuit is required to restore the original position for alignment calculation. This decoding process increases circuit complexity, power consumption, and latency. Although off-chip transmission reduces the theoretical amount of data, it requires the addition of metadata to maintain the integrity of communication, introducing additional overhead. Furthermore, the random access mode will reduce the effective memory bandwidth utilization. This lack of coordinated optimization between on-chip computation and off-chip transmission at the system level makes it difficult to fully unleash the hardware's potential.

[0005] Therefore, how to efficiently complete feature extraction using a lightweight structure within the chip, combine it with an adaptive threshold mechanism to achieve dynamic sparsity adjustment based on image content, and then perform high-precision target detection on the sparse features through an off-chip module has become a key issue in improving the overall performance of the edge target detection system. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a feature extraction system and a data sparsity reduction method for target detection, as follows: The first objective of this invention is to provide a feature extraction system for target detection that can achieve efficient preprocessing, adaptive sparsification, and accurate recognition of input images, and is particularly suitable for edge or terminal platforms with high requirements for low power consumption, storage, and computing.

[0007] The system includes on-chip modules and off-chip modules, which can communicate with each other. The on-chip modules include row input circuits, column input circuits, data storage units, weight storage units, feature extraction modules, and threshold extraction modules, while the off-chip modules include detection and recognition modules. The feature extraction module uses lightweight convolutional kernels to extract features from the input raw image data and output feature map data. The threshold extraction module uses an adaptive thresholding algorithm to learn an adaptive threshold and filter the feature map based on the threshold to reduce invalid features and reduce the computational burden of subsequent processing. The detection and recognition module is responsible for the final recognition task of the sparsified features and outputs the data to the chip.

[0008] Each module plays a different role in the overall image recognition process to improve the recognition rate on low-feature data.

[0009] Furthermore, the feature extraction module and the threshold extraction module share a two-dimensional array structure. The feature extraction module is used to extract feature images, and the threshold extraction module dynamically generates a sparsification threshold for each image or each channel based on the statistical properties of the image or feature map, thereby achieving adaptive feature selection. Furthermore, both the row input circuit and the column input circuit are connected to the data storage unit, which is also connected to the threshold extraction array and the feature extraction array. The weight storage unit is connected to the feature extraction module, and the feature extraction module is connected to the threshold extraction module. Furthermore, the row input circuit and column input circuit are used to read in and read out the original image data, respectively; Furthermore, the data storage unit is used to store the input raw image data; Furthermore, the weight cache unit is used to store the convolution kernel weight parameters; Furthermore, the feature extraction module is used to implement lightweight convolution operations and generate multi-scale feature map data; Furthermore, the threshold extraction module is used to perform statistical analysis and network operations on image or multi-scale feature map data, and output an adaptive threshold.

[0010] Furthermore, the feature extraction module is configured with one or more of the following: depthwise separable convolution, group convolution, bottleneck convolution, factor convolution, dilated convolution, hybrid depthwise convolution, or deformable convolution, in order to reduce computational complexity. Furthermore, the threshold extraction module is configured with a shallow convolutional network or a small-scale fully connected network to perform statistical modeling of global or local features and obtain threshold vectors that are adapted to different image contents. Furthermore, the detection and recognition module includes several feature input modules, feature fusion modules, feature normalization modules, activation modules, adaptive pooling modules, fully connected modules, classification modules, and output management modules; Furthermore, several feature input modules are connected to the feature fusion module, and the feature fusion module, feature normalization module, activation module, adaptive pooling module, fully connected module, classification module and output management module are connected in sequence; Furthermore, several feature input modules are used to receive and buffer sparse feature maps from within the chip, and recover them into sparse or approximately dense representations according to their encoding format; Furthermore, the feature fusion module is used to perform weighted fusion or attention fusion on features of different scales and channels to improve the discriminative power of the features; Furthermore, the feature normalization module is used to perform batch normalization or layer normalization to suppress internal covariate shifts. Furthermore, the activation module is used to apply a non-linear activation function to the features; Furthermore, the adaptive pooling module is used to adaptively pool the feature map in the spatial dimension to obtain a fixed-size feature representation, which can then be used for adaptive sparsity processing. Furthermore, the fully connected module is used to map the pooled features to a high-dimensional semantic space; Furthermore, the classification module is used to output object detection-related results, including class probabilities, bounding box parameters, and / or other task outputs; Furthermore, the output management module is used to organize the test results and send them back to the chip or transmit them to the host system.

[0011] By further employing feature channel slimming, attention mechanisms, and multi-task branching structures in off-chip modules, detection performance can be improved without significantly increasing computational load. Furthermore, the activation module supports a feature channel slimming mechanism, which uses regularization constraints or gating units to dynamically prune low-contribution channels during the inference phase, further compressing the model structure. Furthermore, a high-speed transmission channel for compressed sparse data is supported between on-chip and off-chip, encoding and compressing sparse features, transmitting only non-zero elements and their position information, reducing interface bandwidth pressure.

[0012] The second objective of this invention is to provide a data sparsity method for target detection, which is based on a feature extraction system for target detection described above. The data sparsity method includes input, extraction, threshold generation, filtering, encoding and compression, transmission, recognition and feedback stages. The specific workflow is as follows: Step 1: Input phase; Furthermore, the original input image data is acquired and written into the data storage unit based on the row input circuit and column input circuit; and the convolution kernel weight parameters are written into the weight cache unit for multiple selections and pipelined readings in subsequent convolution operations. Step 2: Extraction stage; Furthermore, the feature extraction module performs lightweight convolution operations and nonlinear transformations on the input raw image data based on lightweight convolution kernels, thereby generating multi-scale feature map data and storing it in the data storage unit; Step 3: Threshold generation stage; Furthermore, the threshold extraction module performs statistical analysis and network operations on the input raw image data and / or the multi-scale feature map data output from step 2 to dynamically generate a sparsity threshold. The sparsity threshold is associated with specific input image features and can be generated independently for each image or for each channel to meet preset sparsity rate, sparsity range and detection accuracy constraints. Step 4: Screening phase; Furthermore, the multi-scale feature map data obtained in step 2 is compared element by element with the sparsification threshold obtained in step 3. Multi-scale feature map data with an absolute value lower than the sparsification threshold is set to zero or discarded, while multi-scale feature map data with an absolute value not lower than the sparsification threshold is retained as a valid non-zero feature, thereby forming a sparsified feature map. Step 5: Encoding and compression stage; Furthermore, the sparse feature map obtained in step 4 is sparsely encoded and compressed according to a predetermined format to generate sparse feature data; the sparse feature data includes at least non-zero feature values ​​and their position index information, which are used for subsequent off-chip reconstruction and computation. Furthermore, sparse coding and compression are implemented using one of the following methods: bitmap tagging and non-zero value sequence combination, run-length encoding, differential encoding, and / or fixed-point quantization packing, to further reduce the amount of data transmitted; the location index information can be represented by linear address, two-dimensional / three-dimensional coordinates (including channel index and spatial location index), or block index; Step 6: Transmission phase; Furthermore, the sparse feature data obtained in step 5 is sent to the detection and recognition module through the communication bus or high-speed data channel between the on-chip module and the off-chip module. Only non-zero elements and their position information are transmitted to reduce the bandwidth pressure of the on-chip and off-chip interfaces. Furthermore, the in-chip and out-of-chip modules work together to support a feature channel slimming mechanism. Through regularization sparsity constraints or gating mechanisms, low-contribution channels are dynamically pruned during the inference stage to further compress the model structure and improve inference efficiency. Step 7: Identification Phase; Furthermore, the detection and recognition module receives sparse feature data transmitted from the on-chip module and restores it to a sparse representation or an approximately dense representation according to its encoding format. Subsequently, the CNN3 detection and recognition module sequentially performs feature fusion, feature normalization, function activation, adaptive pooling, fully connected and classification processing, and finally outputs the target detection results, including class probability, bounding box parameters and / or other task outputs. Step 8: Feedback Phase; Furthermore, the target detection results obtained in step 7 are processed by the output management system and sent back to the on-chip module or transmitted to the host system for display, storage and / or subsequent control decisions.

[0013] The beneficial effects of this invention are: This invention provides a feature extraction system and a data sparsification method for target detection. By constructing a feature extraction system specifically for target detection, the data sparsification method is implemented. Adaptive sparsity processing within the on-chip module reduces redundant features by more than 50% in subsequent calculations, significantly improving computational efficiency. Encoding and compressing sparse features transmits only non-zero elements and their positional information, reducing bandwidth pressure on on-chip and off-chip interfaces. Furthermore, by employing lightweight convolution, channel pruning, and sparsification in combination, the power consumption of the entire system is reduced without affecting detection accuracy. The modular design of this system is applicable to FPGA / ASIC platforms of different process nodes and scales, facilitating integration with existing image acquisition front-ends and control systems, and exhibiting strong scalability. The system provided by this invention effectively reduces computational overhead and energy consumption without significantly sacrificing recognition performance. It possesses high scalability, low latency, and good portability, and can be applied to scenarios with strict requirements for real-time performance, energy consumption, and size, such as mobile terminals, industrial vision equipment, unmanned platforms, and remote sensing payloads. It is particularly suitable for real-time edge recognition scenarios, such as mobile devices, industrial cameras, unmanned platforms, and space remote sensing systems. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram of the overall process of a data sparsity method for target detection provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of an on-chip module in a feature extraction system for target detection provided in Embodiment 1 of the present invention; Figure 3 This is a schematic diagram of the structure of the detection and recognition module in a feature extraction system for target detection provided in Embodiment 1 of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0017] Example 1 This embodiment provides a feature extraction system for target detection. The architecture of the system is as follows: Figure 1 As shown, it includes on-chip modules and off-chip modules. The on-chip modules include a feature extraction module and a threshold extraction module, which extract feature maps and thresholds respectively. The off-chip modules include a detection and recognition module. These parts constitute an end-to-end processing path from image input to detection result output.

[0018] The system includes on-chip modules and off-chip modules, which can communicate with each other. The on-chip modules include row input circuits, column input circuits, data storage units, weight storage units, array feature extraction modules, and threshold extraction modules, while the off-chip modules include detection and recognition modules. The feature extraction module uses lightweight convolutional kernels to extract features from the input image and output multi-scale feature map data. The threshold extraction module uses an adaptive thresholding algorithm to learn an adaptive threshold and filters the feature map based on this threshold to reduce invalid features and reduce the computational burden of subsequent processing. The detection and recognition module is responsible for the final recognition task of the sparsed features and outputs the data to the chip. Each module plays a different role in the whole image recognition process to improve the recognition rate under low feature data.

[0019] The feature extraction module and the threshold extraction module share a two-dimensional array structure. The feature extraction module is used to extract feature images, and the threshold extraction module dynamically generates a sparsification threshold for each image or each channel based on the statistical properties of the image or feature map, thereby achieving adaptive feature selection. The structure of the on-chip module is as follows Figure 2 As shown, it includes a row input circuit, a column input circuit, a data storage unit, a weight storage unit, a feature extraction module, and a threshold extraction module; both the row input circuit and the column input circuit are connected to the data storage unit, the data storage unit is connected to both the threshold extraction module and the feature extraction module, the weight storage unit is connected to the feature extraction module, and the feature extraction module is connected to the threshold extraction module. The row input circuit and column input circuit are used to read in and read out the original image data, respectively. The data storage unit is used to store the input original image data. The weight cache unit is used to store the convolution kernel weight parameters. The feature extraction module is used to implement lightweight convolution operations and generate multi-scale feature map data. The threshold extraction module is used to perform statistical analysis and network operations on the image or multi-scale feature map data and output an adaptive threshold.

[0020] The feature extraction module is configured with one or more of the following: depthwise separable convolution, group convolution, bottleneck convolution, factor convolution, dilated convolution, hybrid depthwise convolution, or deformable convolution, in order to reduce computational complexity; the threshold extraction module is configured with a shallow convolutional network or a small-scale fully connected network to perform statistical modeling of global or local features and obtain threshold vectors that are adapted to different image contents. The structure of the detection and recognition module is as follows: Figure 3 As shown, it includes several feature input modules, feature fusion modules, feature normalization modules, activation modules, adaptive pooling modules, fully connected modules, classification modules, and output management modules; Several feature input modules are connected to the feature fusion module. The feature fusion module, feature normalization module, activation module, adaptive pooling module, fully connected module, classification module, and output management module are connected in sequence. Several feature input modules are used to receive and cache sparse feature maps from within the chip, and restore them to sparse or approximately dense representations according to their encoding format. The feature fusion module is used to perform weighted fusion or attention fusion on features of different scales and channels to improve the discriminativeness of the features. The feature normalization module is used to perform batch normalization or layer normalization to suppress internal covariate shift. The activation module is used to apply nonlinear activation functions to the features. The adaptive pooling module is used to perform adaptive pooling on the feature maps in the spatial dimension to obtain feature representations of fixed size, which can then be used for adaptive sparsity processing. The fully connected module is used to map the pooled features to a high-dimensional semantic space. The classification module is used to output object detection-related results, including class probabilities, bounding box parameters and / or other task outputs. The output management module is used to organize the detection results and send them back to the chip or transmit them to the host system.

[0021] The activation module in the off-chip module adopts feature channel slimming, attention mechanism and multi-task branch structure, which can improve detection performance without significantly increasing the amount of computation; the activation module supports feature channel slimming mechanism, which uses regularization constraints or gating units to dynamically prune low contribution channels during the inference stage, further compressing the model structure; The on-chip and off-chip modules support a high-speed transmission channel for compressed sparse data, which encodes and compresses sparse features, transmitting only non-zero elements and their position information to reduce interface bandwidth pressure. By employing feature channel slimming, attention mechanisms, and multi-task branching structures in the off-chip modules, detection performance can be improved without significantly increasing computational load. The activation module in the off-chip modules supports a feature channel slimming mechanism, which uses regularization constraints or gating units to dynamically prune low-contribution channels during the inference phase, further compressing the model structure.

[0022] Example 2 This embodiment provides a data sparsity method for target detection. This method is based on a feature extraction system for target detection described in Embodiment 1. The data sparsity method writes the original image data into the data storage unit through row input circuits and column input circuits. Then, the feature extraction module performs multi-level convolution and nonlinear transformation to generate a set of multi-scale feature map data. Then, the threshold extraction module calculates the adaptive sparsity threshold corresponding to the multi-scale feature map data of the original image data. Then, based on the adaptive sparsity threshold, the multi-scale feature map data is filtered element by element to form a sparse feature map. After encoding the position and value of non-zero elements, it is transmitted to the detection and recognition module to realize target detection and output, thereby achieving end-to-end target detection under low bandwidth and low power consumption conditions.

[0023] This data sparsity method includes input, extraction, threshold generation, filtering, encoding and compression, transmission, recognition, and feedback stages. The specific workflow is as follows: Step 1: Input stage; acquire the original input image data, and write the input image data into the data storage unit based on the row input circuit and column input circuit; and write the convolution kernel weight parameters into the weight cache unit for multiple selections and pipelined readings in subsequent convolution operations; Step 2: Extraction stage; The feature extraction module performs lightweight convolution operations and nonlinear transformations on the input raw image data based on lightweight convolution kernels, thereby generating multi-scale feature map data and storing it in the data storage unit; Step 3: Threshold generation stage; The threshold extraction module performs statistical analysis and network operations on the input raw image data or the multi-scale feature map data output in Step 2 to dynamically generate a sparsity threshold; The sparsity threshold is associated with the specific input image features and can be generated independently for each image or for each channel to meet the preset sparsity rate, sparsity range and detection accuracy constraints. Step 4: Filtering stage; The multi-scale feature map data obtained in Step 2 is compared element by element with the sparsification threshold obtained in Step 3. Multi-scale feature map data with an absolute value lower than the sparsification threshold is set to zero or discarded, while multi-scale feature map data with an absolute value not lower than the sparsification threshold is retained as a valid non-zero feature, thus forming a sparsification feature map. Step 5: Encoding and compression stage; The sparse feature map obtained in step 4 is sparsely encoded and compressed according to a predetermined format to generate sparse feature data; The sparse feature data includes at least non-zero feature values ​​and their position index information, which are used for subsequent off-chip reconstruction and computation; Among them, sparse coding and compression are implemented by one of the following methods: bitmap tagging and non-zero value sequence combination, run-length encoding, differential encoding and / or fixed-point quantization packing, in order to further reduce the amount of transmitted data; the location index information can be represented by linear address, two-dimensional / three-dimensional coordinates (including channel index and spatial location index) or block index; Step 6: Transmission stage; The sparse feature data obtained in Step 5 is sent to the detection and recognition module through the communication bus or high-speed data channel between the on-chip module and the off-chip module. During the transmission process, the sparse feature data is encoded and compressed, and only non-zero elements and their position information are transmitted to reduce the bandwidth pressure of the on-chip and off-chip interfaces. Among them, the in-chip module and the out-of-chip module work together to support the feature channel slimming mechanism. Through regularization sparsity constraints or gating mechanisms, low-contribution channels are dynamically pruned during the inference stage to further compress the model structure and improve inference efficiency. Step 7: Recognition Stage; The detection and recognition module receives sparse feature data transmitted from the on-chip module and restores the sparse feature data to a sparse representation or an approximately dense representation according to the encoding format of the sparse feature data; Subsequently, the detection and recognition module sequentially performs feature fusion, feature normalization, function activation, adaptive pooling, fully connected and classification processing, and finally outputs the target detection results, including class probability, bounding box parameters and / or other task outputs; Step 8: Feedback phase; The target detection results obtained in step 7 are processed by the output management module and sent back to the on-chip module or transmitted to the host system for display, storage and / or subsequent control decisions.

[0024] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0025] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data sparsity method for target detection, characterized in that, The data sparsity method is based on a feature extraction system that includes on-chip modules and off-chip modules. The on-chip modules include row input circuits, column input circuits, data storage units, weight storage units, feature extraction modules, and threshold extraction modules. The off-chip modules include a detection and recognition module. The data sparsity methods include: Step 1: Input the raw image data into the feature extraction system; Step 2: The feature extraction module extracts features from the original image data to obtain multi-scale feature map data; Step 3: The threshold extraction module analyzes the multi-scale feature map data obtained in Step 2 based on the adaptive threshold algorithm and calculates the sparsity threshold. Step 4: Based on the sparsification threshold obtained in Step 3, filter the multi-scale feature map data obtained in Step 2 to obtain a sparsified feature map; Step 5: Perform sparse coding and compression on the sparsed feature map obtained in Step 4 to generate sparse feature data; Step 6: Transmit the sparse feature data obtained in Step 5 to an off-chip module for detection and recognition; Step 7: The detection and recognition module restores the sparse feature data to a sparse representation or an approximately dense representation based on the encoding format of the sparse feature data, and outputs the target detection result after processing; Step 8: Feed back the target detection results obtained in Step 7 to the on-chip module or transmit them to the host system to achieve target detection with sparse data.

2. The data sparsity method according to claim 1, characterized in that, In step 1, the original image data is written into the data storage unit based on the row input circuit and the column input circuit, and the convolution kernel weight parameters are written into the weight cache unit. In step 2, the feature extraction module performs lightweight convolution operations and nonlinear transformations on the input raw image data based on the convolution kernel, generates multi-scale feature map data, and stores it in the data storage unit. In step 3, the threshold extraction module performs statistical analysis and network operations on the input raw image data or the multi-scale feature map data output in step 2 to dynamically generate a sparsity threshold. The sparsification threshold is associated with the input image features and is generated independently for the original image data or for each channel separately, in order to meet the preset sparsity rate, sparsity range and detection accuracy constraints. In step 4, the absolute value of the multi-scale feature map data obtained in step 2 is compared element by element with the sparsification threshold obtained in step 3. Multi-scale feature map data with an absolute value lower than the sparsification threshold is set to zero or discarded, while multi-scale feature map data with an absolute value not lower than the sparsification threshold is retained as a valid non-zero feature, thereby forming a sparsification feature map. Step 5 includes sparse encoding and compression of the sparsed feature map obtained in step 4 to generate sparse feature data, wherein the sparse feature data includes non-zero feature values ​​and their position index information. In step 6, only the non-zero elements and their position information in the sparse feature data obtained in step 5 are transmitted.

3. The data sparsity method according to claim 2, characterized in that, In step 5, sparse coding and compression are implemented using one of the following methods: bitmap tagging and non-zero value sequence combination, run-length encoding, differential encoding, or fixed-point quantization packing. The location index information is represented using one of the following: linear address, two-dimensional or three-dimensional coordinates, or block index; In step 6, the on-chip module and off-chip module work together to support the feature channel slimming mechanism, which dynamically trims contributing channels through regularization sparsity constraints or gating mechanisms. The off-chip module employs feature channel slimming, attention mechanism, and multi-task branching structure.

4. The data sparsity method according to claim 3, characterized in that, Both the row input circuit and the column input circuit are connected to the data storage unit. The data storage unit is also connected to the threshold extraction module and the feature extraction module. The weight storage unit is connected to the feature extraction module, and the feature extraction module is connected to the threshold extraction module. The feature extraction module and the threshold extraction module share a two-dimensional array structure composed of row input circuits and column input circuits. The row input circuits and column input circuits are used to read in and read out the original image data, respectively. The data storage unit is used to store the input original image data, and the weight cache unit is used to store the convolution kernel weight parameters. The feature extraction module is used to implement lightweight convolution operations and generate multi-scale feature map data. The threshold extraction module is used to perform statistical analysis and network operations on the image or multi-scale feature map data and output a sparsity threshold.

5. The data sparsity method according to claim 4, characterized in that, The convolution kernel in the feature extraction module is one or a combination of several of the following: depthwise separable convolution, group convolution, bottleneck convolution, factor convolution, dilated convolution, mixed depthwise convolution, or deformable convolution. The threshold extraction module is configured as a convolutional network or a fully connected network to perform statistical modeling of global or local features and obtain threshold vectors adapted to different image content.

6. The data sparsity method according to claim 5, characterized in that, The detection and recognition module includes several feature input modules, feature fusion modules, feature normalization modules, activation modules, adaptive pooling modules, fully connected modules, classification modules, and output management modules; The feature input modules are all connected to the feature fusion module. The feature fusion module, feature normalization module, activation module, adaptive pooling module, fully connected module, classification module, and output management module are connected in sequence.

7. The data sparsity method according to claim 6, characterized in that, The aforementioned feature input modules are used to receive and cache sparse feature maps from within the chip, and recover them into sparse or approximately dense representations according to their encoding format. The feature fusion module is used to perform weighted fusion or attention fusion on features of different scales and different channels; The feature normalization module is used to perform batch normalization or layer normalization; The activation module is used to apply a nonlinear activation function to the feature; The adaptive pooling module is used to adaptively pool the feature map in the spatial dimension to obtain a feature representation of a fixed size; The fully connected module is used to map the pooled features to a high-dimensional semantic space; The classification module is used to output object detection related results, including class probability, bounding box parameters and / or other task outputs; The output management module is used to organize the detection results and send them back to the chip or transmit them to the host system.

Images