Efficient compressive sensing target recognition system and method based on discrete cosine transform

Abstract

The application discloses a high-efficiency compressed sensing target recognition system and method based on discrete cosine transform, and belongs to the technical field of computer vision.The application decomposes the separable characteristic of the discrete cosine transform into sequentially executed row direction one-dimensional discrete cosine transform and column direction one-dimensional discrete cosine transform, and then directly outputs the discrete cosine transform coefficient by using programmable resistance or capacitance to form a corresponding hardware circuit, and a special neural network model for directly completing an image recognition task in a frequency domain is designed, so that the image is not restored from the frequency domain to a pixel space domain by inverse discrete cosine transform, but target recognition is directly performed by using the frequency domain feature, so that redundant calculation links are eliminated, and the overall calculation amount is reduced by about 1 / 3; through software and hardware collaborative optimization, the application can be deployed on a resource-limited edge device, has low delay and meets the real-time requirement, and can help the development of the Internet of Things, automatic driving and other fields.

Description

Technical Field

[0001] This invention relates to an efficient compressed sensing target recognition system and method based on discrete cosine transform, belonging to the field of computer vision technology. Background Technology

[0002] With the development of computer vision technology, neural network-based target recognition technology has been used in various industries, such as autonomous driving, security monitoring, drone vision, and the Internet of Things. These application scenarios usually perform target recognition on resource-constrained embedded or mobile platforms, which have high requirements for the real-time performance and low power consumption of the system. Therefore, efficient data transmission and processing are the key challenges to achieving real-time target recognition.

[0003] The current mainstream deployment approach involves sending image data to a centrally located computing terminal for centralized processing due to insufficient computing resources on edge devices. The image recognition results are then fed back to the edge devices. However, this approach struggles to meet the aforementioned requirements for low latency and low power consumption. First, image sensors (especially high-resolution sensors) capture massive amounts of raw pixel data. Directly sending this data to the computing terminal creates significant bandwidth pressure on the data transmission channel between the edge and the terminal, becoming a system bottleneck and resulting in unacceptable latency and power consumption. Second, if the raw image data is compressed at the edge, for example, through a discrete cosine transform (DCT), the computing terminal, in order to meet the input requirements of traditional recognition networks, typically performs a time-consuming inverse DCT to recover the image before inputting the recovered pixel data into the target recognition network for inference. The computational cost of the inverse DCT is substantial, especially on resource-constrained devices, significantly increasing the overall computational overhead and latency of target recognition. These bottlenecks collectively lead to increased overall recognition latency, higher power consumption, and stringent bandwidth requirements, limiting its application in bandwidth-constrained or extremely low-latency scenarios.

[0004] In summary, existing target recognition deployment schemes (especially those for high-resolution, real-time scenarios) have significant problems in terms of transmission bandwidth requirements and computational efficiency. There is an urgent need to find a new solution that can minimize transmission bandwidth requirements while maintaining recognition accuracy, and also provide computational efficiency to adapt to low-latency scenarios. Summary of the Invention

[0005] To address the issues of high transmission bandwidth pressure and low computational efficiency in existing target recognition deployment schemes, thereby enabling target recognition to be more widely applied to edge devices and low-latency application scenarios, this application proposes a high-efficiency compressed sensing target recognition system based on discrete cosine transform. The system includes an image sensor, a discrete cosine transform readout circuit module, an image recognition module, and a system control module. The image sensor, discrete cosine transform readout circuit module, and image recognition module are connected in sequence and are all connected to the system control module.

[0006] The image sensor converts the incident light signal into an electrical signal spatially distributed in the pixel domain. The discrete cosine transform readout circuit module receives the electrical signal spatially distributed in the pixel domain and obtains the two-dimensional discrete cosine transform result through a two-stage analog vector matrix multiplication circuit, directly outputting the compressed discrete cosine transform frequency domain coefficients. The image recognition module directly uses the discrete cosine transform frequency domain coefficients as input to obtain the target recognition result. The system control module is used to control the overall interaction between the system and external components, store configuration information, and control the imaging timing of the image sensor, the readout timing of the discrete cosine transform readout circuit module, and the recognition behavior of the image recognition module according to the configuration information.

[0007] Optionally, the image sensor includes a pixel array, a row driving logic unit, a column driving logic unit, and a sensor-internal system control unit; wherein, the sensor-internal system control unit is connected to the system control module; the sensor-internal system control unit is connected to the row driving logic unit and the column driving logic unit respectively; the row driving logic unit is coupled to the row lines of the pixel array to provide the pixel array with a timing-flexible adjustable row driving state voltage; the column driving logic unit is coupled to the column lines of the pixel array to process and read signals in the column direction; under the synchronous control of the sensor-internal system control unit, the working timing of the column driving logic unit is precisely matched with the row scanning timing of the row driving logic unit, thereby realizing the parallel reading and processing of pixel signals in each column of the selected row to obtain the original analog data of the pixel domain distributed in space, i.e., pixel domain data; the pixel array is composed of M×N photosensitive devices in a two-dimensional matrix form.

[0008] Optionally, the discrete cosine transform readout circuit module includes a sample-and-hold circuit, a row-direction analog vector matrix multiplication circuit, a column-direction analog vector matrix multiplication circuit, an analog-to-digital converter circuit, a pipeline unit, and a mobile industry processor interface unit connected in sequence. The sample-and-hold circuit receives pixel domain data from the image sensor, converts the charge value data into voltage values ​​and holds them, and centers the values. The row-direction analog vector matrix multiplication circuit and the column-direction analog vector matrix multiplication circuit respectively perform discrete cosine transform on the one-dimensional row vector to obtain the analog voltage of the two-dimensional discrete cosine transform. The analog-to-digital converter circuit converts the analog voltage into discrete cosine transform coefficients of the digital signal. The pipeline unit is used to temporarily store and time-reform the discrete cosine transform coefficients. The mobile industry processor interface unit is used to package the data according to a predetermined standard protocol format and output it.

[0009] Optionally, the row-direction analog vector matrix multiplication circuit and the column-direction analog vector matrix multiplication circuit are composed of a matrix network consisting of several programmable resistors or programmable capacitors. The values ​​of the programmable resistors are encoded as the reciprocals of the cosine basis in the discrete cosine transform, and the values ​​of the programmable capacitors are encoded as the cosine basis in the discrete cosine transform.

[0010] Optionally, the image recognition module includes a mobile industry processor interface unit two, a neural network processing unit, a static random access memory (SRAM), and an output packaging unit; wherein, the mobile industry processor interface unit two, the neural network processing unit, and the output packaging unit are connected in sequence, and the SRAM is connected separately to the neural network processing unit; the mobile industry processor interface unit two is used to receive data output by the mobile industry processor interface unit one, configure it to align with the standard protocol of the discrete cosine transform readout circuit module, and perform preprocessing operations; the neural network processing unit is used to perform image recognition tasks directly in the frequency domain using a pre-trained neural network model; the output packaging unit is used to encapsulate the image recognition result data using a communication protocol and then output it; the SRAM is used to store the weights and bias parameters of the neural network.

[0011] Optionally, the neural network model includes an input layer, a Zig-Zag encoding layer, an overlay information layer, a same-frequency fusion layer, a cross-band multi-head attention feature extraction layer, and an output layer connected in sequence.

[0012] The system comprises the following layers: an input layer for receiving the discrete cosine transform coefficient matrix; a Zig-Zag encoding layer for dividing the discrete cosine transform coefficient matrix into blocks and performing Zig-Zag encoding; an overlay information layer for overlaying a quantization quality factor matrix and a classification label onto the Zig-Zag encoded matrix; a co-frequency fusion layer for fusing information from the same frequency band in different blocks; a cross-band multi-head attention feature extraction layer employing a Transformer-like architecture, using a 12-layer encoder for feature extraction, with each encoder layer consisting of a multi-head attention mechanism and a feedforward network; and an output layer for obtaining the target recognition result through the classification label.

[0013] Optionally, the system control module includes an on-chip slave unit, an on-chip master unit, and a top-level control unit, wherein the on-chip slave unit and the on-chip master unit are both connected to the top-level control unit;

[0014] The on-chip slave unit communicates with the external host via the SPI protocol to configure and monitor the system's operating parameters;

[0015] The on-chip host unit communicates with other functional modules in the system using the APB protocol;

[0016] The top-level control unit includes a register file containing a series of configuration registers and status registers mapped to specific memory addresses, which ultimately generate three sets of key control command signals: "control imaging", "control readout", and "control recognition".

[0017] This invention also provides an efficient compressed sensing target recognition method based on discrete cosine transform, the method being implemented based on the above-mentioned system, the method comprising:

[0018] Step 1: The system control module is powered on and started. Based on the preset configuration parameters or the configuration parameters received from the external host, the image sensor, the discrete cosine transform readout circuit module and the image recognition module are initialized and configured.

[0019] Step 2: The image sensor uses a pixel array to sense light in response to the incident light signal, converts the light signal into an analog electrical signal, and outputs pixel domain data.

[0020] Step 3: After receiving the pixel domain data, the discrete cosine transform readout circuit module performs block-based two-dimensional discrete cosine transform calculation on the input data, converting the pixel domain image data into a discrete cosine transform coefficient data stream in the frequency domain and transmitting it to the image recognition module.

[0021] Step 4: The image recognition module uses a pre-trained neural network model to perform image recognition tasks directly in the frequency domain and outputs the recognition results.

[0022] Optionally, the neural network model is trained using gradient descent with a cross-entropy loss function.

[0023] The present invention also provides applications of the above-mentioned efficient compressed sensing target recognition system and / or method based on discrete cosine transform in autonomous driving, security monitoring, drone vision and the Internet of Things.

[0024] The beneficial effects of this invention are:

[0025] (1) The high-efficiency compressed sensing target recognition system based on discrete cosine transform proposed in this invention performs hardware-level discrete cosine transform at the imaging end. Based on the separable characteristics of discrete cosine transform, it is decomposed into sequentially executed row-direction one-dimensional discrete cosine transform and column-direction one-dimensional discrete cosine transform. One two-dimensional discrete cosine transform can be equivalent to two one-dimensional discrete cosine transforms. Then, the corresponding hardware implementation circuit is designed to directly output the discrete cosine transform coefficients, which can achieve a data compression rate of more than 10 times. It can significantly reduce the data communication bandwidth pressure between the imaging end and the computing end, thereby reducing the latency and power consumption of the entire system.

[0026] (2) The efficient compressed sensing target recognition method based on discrete cosine transform proposed in this invention has specially designed a neural network model, so that when the compressed discrete cosine transform coefficients are processed by the neural network, it is not necessary to perform inverse discrete cosine transform to restore the image from the frequency domain to the pixel space domain. Instead, the target recognition is directly performed using the frequency domain features, thereby eliminating redundant calculation links. Furthermore, the compression of discrete cosine transform reduces the network input data, thereby reducing the computational resources required for inference and achieving an overall reduction of about 1 / 3 of the computational load.

[0027] (3) Through the above-mentioned hardware and software co-optimization, the present invention enables the target recognition algorithm based on neural networks to be deployed on edge devices with limited resources, and has low latency and meets the requirements of real-time performance, which can help the development of the Internet of Things, autonomous driving and other fields. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is the overall architecture diagram of the efficient compressed sensing target recognition system based on discrete cosine transform provided in the embodiments of the present invention;

[0030] Figure 2This is a structural diagram of the image sensor in the high-efficiency compressed sensing target recognition system based on discrete cosine transform provided in this embodiment of the invention;

[0031] Figure 3 This is a flowchart of the discrete cosine transform algorithm in the efficient compressed sensing target recognition system based on discrete cosine transform provided in the embodiments of the present invention;

[0032] Figure 4 This is a structural diagram of the discrete cosine transform readout circuit module in the high-efficiency compressed sensing target recognition system based on discrete cosine transform provided in this embodiment of the invention;

[0033] Figure 5 This is a simplified structural diagram of the row direction analog vector matrix multiplication circuit in the discrete cosine transform readout circuit module;

[0034] Figure 6 This is a structural diagram of the image recognition module in the efficient compressed sensing target recognition system based on discrete cosine transform provided in this embodiment of the invention;

[0035] Figure 7 This is a diagram of the dedicated neural network model structure proposed in this application, which can directly complete image recognition tasks in the frequency domain;

[0036] Figure 8 This is a structural diagram of the system control module in the efficient compressed sensing target recognition system based on discrete cosine transform provided in this embodiment of the invention;

[0037] Figure 9 This is a flowchart of an efficient compressed sensing target recognition method based on discrete cosine transform provided in an embodiment of the present invention;

[0038] In each diagram: 1-Image sensor, 2-Discrete cosine transform readout circuit module, 3-Image recognition module, 4-System control module; 6-Pixel array, 7-Row drive logic unit, 8-Column drive logic unit, 9-In-sensor system control unit; 10-Sample-and-hold circuit, 11-Row direction analog vector matrix multiplication circuit, 12-Column direction analog vector matrix multiplication circuit, 13-Analog-to-digital conversion circuit, 14-Pipeline unit, 15-Mobile industry processor interface unit one, 16-Programmable resistor; 17-Mobile industry processor interface unit two, 18-Neural network processing unit, 19-Static random access memory, 20-Output packing unit; 21-Slave unit SPI, 22-On-chip master unit APB, 23-Top-level control unit, 24-Register file. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0040] Example 1

[0041] This embodiment provides a high-efficiency compressed sensing target recognition system based on discrete cosine transform. See [link to documentation]. Figure 1 The system includes an image sensor 1, a discrete cosine transform readout circuit module 2, an image recognition module 3, and a system control module 4; the image sensor 1, the discrete cosine transform readout circuit module 2, and the image recognition module 3 are connected in sequence and are all connected to the system control module 4.

[0042] Image sensor 1, as the front-end photosensitive unit of the system, can convert incident light signals into electrical signals spatially distributed in the pixel domain. Discrete cosine transform readout circuit module 2 integrates a dedicated analog circuit for offline cosine transform, which can receive pixel domain data of electrical signals from image sensor 1 in real time, perform block discrete cosine transform calculations at the hardware level, and output the discrete cosine transform coefficient data of the image through analog-to-digital conversion (ADC). It is one of the core modules for achieving data compression and bandwidth reduction in this invention. Image recognition module 3 contains a pre-trained dedicated neural network model, which can take the discrete cosine transform coefficient data stream from discrete cosine transform readout circuit module 2 as input and finally output the target recognition result. System control module 4 is responsible for the overall interaction with the external implementation control, stores configuration information, and controls the imaging timing of image sensor, the readout timing of discrete cosine transform circuit module, and the recognition behavior of image recognition module according to the configuration information.

[0043] The block diagram of image sensor 1 is as follows Figure 2As shown, it includes a pixel array 6, a row driving logic unit 7, a column driving logic unit 8, and a sensor-internal system control unit 9. The pixel array 6 consists of M×N photosensitive devices arranged in a two-dimensional matrix. The array size (M, N) can be determined according to the actual task requirements. These photosensitive devices can respond to incident light signals, converting light signals of different intensities into analog electrical signals of corresponding intensities to achieve the basic functions of photoelectric conversion and imaging. The column driving logic unit 8 is coupled to the row lines (wordlines) of the pixel array 6 and is used to provide the pixel array with a time-flexible and adjustable row driving state voltage. Under the control of the sensor-internal system control unit 9, this unit sequentially selects pixel devices in different rows through row-by-row scanning or random access, causing them to enter the reset, exposure, and readout working states in sequence. The column drive logic unit 8 is coupled to the column lines (bitlines) of the pixel array 6 and is used to process and read signals in the column direction. Under the synchronous control of the system control unit 9 within the sensor, its operating timing is precisely matched with the row scanning timing of the row drive logic unit 7, thereby realizing parallel reading and processing of pixel signals in each column of the selected row. The output of the column drive logic unit 8 is directly coupled to the subsequent discrete cosine transform readout circuit module 2, providing it with raw analog data of the pixel domain spatially distributed. The system control unit 9 within the sensor serves as the timing and control core of the entire image sensor 1, and typically contains a programmable state machine or a microcontroller core. This unit receives "imaging control" instructions (such as exposure time, gain setting, operating mode, etc.) from the system control module 4 and generates all internal control signals and clock timing accordingly. These signals are sent to the row drive logic unit 7 and the column drive logic unit 8 respectively to precisely coordinate every step of the entire imaging process, such as row gating, column processing, and data output, ensuring that each unit works collaboratively and ultimately outputs high-quality and timing-correct image data.

[0044] Discrete Cosine Transform (DCT) readout circuit module 2 is used to perform the Discrete Cosine Transform (DCT) of the image. DCT is a commonly used image compression algorithm, and its flow is as follows: Figure 3 As shown:

[0045] Step S1: Image segmentation, the original image (e.g., 512×512 pixels) is divided into 8×8 blocks.

[0046] Step S2: Pixel value centering. Since pixel values ​​are generally in the range of 0~255 (in the 8-bit case), the Discrete Cosine Transform requires the input signal to be close to 0. Therefore, each pixel value needs to be subtracted by 128 (in the 8-bit case), that is:

[0047]

[0048] in These are image pixel values. It is the centered image pixel value.

[0049] Step S3: Perform a two-dimensional discrete cosine transform. The discrete cosine transform is defined as follows:

[0050]

[0051] in It is the centered input pixel value. These are the output discrete cosine transform coefficients, where u and v are the frequency domain coordinates (0-7). When u = 0, otherwise , Similarly, the coefficients obtained after discrete cosine transform represent the energy intensity of this region of the image in the frequency domain. For example, the upper left F(0,0) represents the DC component, which represents the average brightness of the entire block, while the value in the lower right corner represents the high-frequency component, which represents the details and edges in the block.

[0052] Step S4: Quantization. Based on the human eye's sensitivity to different frequencies, a quantization table is used to divide and round the coefficients obtained from the discrete cosine transform.

[0053]

[0054] in These are the discrete cosine transform coefficients. This corresponds to the quantization step size. These are the quantized discrete cosine transform coefficients.

[0055] Step S5: Output coefficients. Arrange the 8×8 coefficient matrix into a 1-dimensional sequence according to the zigzag path and output it.

[0056] From the calculation formula of the two-dimensional discrete cosine transform in step S3, it can be seen that the horizontal and vertical cosine quantities are multiplied, indicating that the two-dimensional discrete cosine transform is separable. That is, one two-dimensional discrete cosine transform is equivalent to two one-dimensional discrete cosine transforms, i.e.:

[0057]

[0058] in It is an image pixel matrix. It is the transformation matrix of the one-dimensional discrete cosine transform. It is a transformation matrix transpose, It is the coefficient matrix after the two-dimensional discrete cosine transform. It can be understood as performing a one-dimensional discrete cosine transform on the image patch in the row direction, and then performing a one-dimensional discrete cosine transform in the column direction to finally obtain the result of the two-dimensional discrete cosine transform. That is, the image is "projected" onto the cosine basis in the horizontal direction and then "projected" in the vertical direction to complete the two-dimensional decomposition.

[0059] The Discrete Cosine Transform (DCT) transforms an image from the spatial domain to the frequency domain, concentrating over 85% of the image's energy in low-frequency coefficients. This highly concentrated energy improves compression efficiency. Compared to the spatial domain, the correlation between adjacent pixels decreases from 0.9 to <0.1 in the frequency domain, effectively removing redundancy and reducing information entropy. The DCT is widely used in the JPEG standard, typically achieving compression ratios of over 10 times. This is the fundamental principle behind this invention's ability to effectively reduce data volume and communication bandwidth.

[0060] Based on the above principles, this application designs Figure 4 The discrete cosine transform readout circuit module 2 shown includes a sample-and-hold circuit 10, a row-direction analog vector matrix multiplication circuit 11, a column-direction analog vector matrix multiplication circuit 12, an analog-to-digital converter circuit 13, a pipeline unit 14, and a mobile industry processor interface unit 15. The sample-and-hold circuit 10 receives pixel domain data from the image sensor 1, converts the charge value data into voltage values ​​and holds them, and performs value centering in this process, providing a stable and accurate input for subsequent analog calculations.

[0061] The sampled and held data is fed into the core transformation unit. This invention utilizes the separability of the two-dimensional discrete cosine transform (DCT) to decompose it into sequentially executed row-direction one-dimensional DCT and column-direction one-dimensional DCT. The transformation is first performed in the row direction: data blocks are fed into the row-direction analog vector-matrix multiplication circuit 11. This circuit is crucial for implementing the discrete cosine transform, such as... Figure 5 As shown, the row-direction analog vector matrix multiplication circuit 11 is internally composed of a matrix network of programmable resistors 16. The characteristic of the programmable resistors 16 is that their resistance values ​​can be encoded as needed, thus corresponding to the reciprocal of the cosine basis in the discrete cosine transform, i.e. The resistance values ​​in this matrix are pre-encoded into the matrix described in the above principle. The input vector X(i) is physically represented as voltage V(i). Therefore, by using a pre-encoded programmable resistor array, it can directly perform vector-matrix multiplication of the input pixel vector and the cosine basis matrix in the analog domain, efficiently completing the one-dimensional discrete cosine transform. This projects the row pixel signals of the image onto the frequency domain basis of the discrete cosine transform, outputting analog signals representing different frequency components of that row. The intermediate results after the row transform (i.e., the data after the row-direction discrete cosine transform) are temporarily stored and then fed into the column-direction analog vector-matrix multiplication circuit 12. This circuit performs another-dimensional discrete cosine transform in the column direction using the exact same operating principle. Through these two stages of cascaded analog vector-matrix multiplication operations, the final two-dimensional discrete cosine transform is fully and efficiently achieved, yielding the final two-dimensional discrete cosine transform coefficients.

[0062] After two stages of analog vector matrix multiplication, the resulting signal is an analog voltage representing the discrete cosine transform coefficients. This analog signal is fed into an analog-to-digital converter (ADC) 13, which converts it into digital discrete cosine transform coefficients. The quantization precision (e.g., 10-bit, 12-bit) can be configured according to the system's requirements for precision and data volume. The quantized digital discrete cosine transform coefficient data stream is fed into a pipeline unit (FIFO, First-In-First-Out Memory) 14, which solves the matching problem between data processing and output rates, temporarily storing and timing-reorganizing the discrete cosine transform coefficient data to ensure that the data can be transmitted at a stable and compliant rate. The discrete cosine transform coefficient data stream shaped by the pipeline unit 14 is then fed into a Mobile Industry Processor Interface (MIPI) unit 15. This MIPI unit packages the data into a format conforming to standard protocols such as MIPI CSI-2, and outputs it as the final DCT domain data to the subsequent image recognition module 3. This standardized interface greatly enhances the universality and compatibility of this module's connection with external processing units.

[0063] In summary, the Discrete Cosine Transform (DCT) readout circuit module 2 innovatively employs a two-stage analog vector matrix multiplication circuit to efficiently and with low power consumption perform separable two-dimensional DCT transformations directly in the analog domain. This avoids the overhead of performing numerous multiplication and addition operations in the digital domain. Furthermore, this design enables the image sensor to directly output compressed DCT frequency domain coefficients, rather than the massive amount of raw pixel data, fundamentally solving the bottleneck problem of system transmission bandwidth.

[0064] It should be noted that, in addition to programmable resistors, programmable capacitors can also be used in the row-direction analog vector matrix multiplication circuit 11 and the column-direction analog vector matrix multiplication circuit 12. If a programmable capacitor is used, its capacitance value is encoded as the cosine basis in the discrete cosine transform.

[0065] The block diagram of image recognition module 3 is as follows: Figure 6 As shown, it includes a mobile industry processor interface unit 2 (17), a neural network processing unit (18), a static random access memory (SRAM) 19, and an output packing unit 20. The mobile industry processor interface unit 2 (17) receives the compressed DCT coefficient data stream, is configured to align with the output protocol of the discrete cosine transform readout circuit module 2, and is responsible for preprocessing the input data stream, such as unpacking and decoding, converting it into a standard data format that can be processed by subsequent computing units. The neural network processing unit 18 is a hardware accelerator specifically designed for neural network algorithms. It contains a highly parallel computing array (such as a multiply-accumulator array) for efficiently performing neural network operations such as matrix multiplication, convolution, pooling, and activation functions, and can deploy and infer neural network models. Another key innovation of this invention is that the neural network model deployed inside the neural network processing unit 18 is a specially designed, trained network capable of directly performing image recognition task inference in the frequency domain.

[0066] The neural network model comprises an input layer, a Zig-Zag encoding layer, a superimposed information layer, a same-frequency fusion layer, a cross-band multi-head attention feature extraction layer, and an output layer, all connected in sequence. The input layer receives the discrete cosine transform coefficient matrix. The Zig-Zag encoding layer divides the discrete cosine transform coefficient matrix into blocks and performs Zig-Zag encoding. The superimposed information layer superimposes a quantization quality factor matrix and a classification label onto the Zig-Zag encoded matrix. The same-frequency fusion layer fuses information from the same frequency band in different blocks. The cross-band multi-head attention feature extraction layer adopts a Transformer-like architecture, extracting features through a 12-layer encoder, each layer consisting of a multi-head attention mechanism and a feedforward network. The output layer obtains the target recognition result through the classification label.

[0067] Specifically, its input layer and network weights are optimized for the statistical characteristics of the discrete cosine transform coefficient block, enabling it to directly extract effective information from frequency domain features, completely avoiding the huge computational overhead of inversely transforming the discrete cosine transform coefficients back to the pixel domain. For example... Figure 7As shown, the network model designed in this invention uses a 224×224 DCT coefficient matrix as its input. The model then preprocesses the input by dividing the coefficient matrix into 8×8 blocks. Each sub-block is then Zig-Zag encoded, meaning the data is rearranged into a 1D vector in a "Z" shape, starting from the top left corner. This rearrangement prioritizes low-frequency signals over high-frequency signals. All the rearranged 1D vectors are then rewritten and integrated into a 64×784 input matrix, with the same frequency bands grouped together. Finally, a quantization quality factor matrix corresponding to the frequency band is superimposed on this matrix, which is the principle described in the Discrete Cosine Transform explanation. The purpose is to include the quantization table as one of the input features to prevent the model from being sensitive to changes in the quantization quality factor. Furthermore, after adding the quantization quality factor matrix, a "classification label" row is inserted before the first row of the matrix to record classification feature information used as the final classification basis. Before formally starting feature extraction, a frequency fusion operation is required on the input. This is done by multiplying the input matrix with a learnable weight matrix to fuse information from the same frequency band in different blocks. After the above preprocessing of the frequency domain input, the neural network model formally begins feature extraction, using a Transformer-like architecture. It extracts features from the frequency domain matrix through a 12-layer encoder. Each encoder layer consists of a multi-head attention mechanism and a feedforward network. First, the input frequency domain matrix is ​​processed through three different linear layers to obtain the corresponding Q, K, and V matrices. The Q matrix represents "what information I want to find in the frequency band," the K matrix represents "what information this frequency band can provide," and the V matrix represents "the specific information content that this frequency band can provide." Then, the formula is used... Calculate the attention score, where It is a scaling factor. This cross-band multi-head attention mechanism effectively captures the correlation between different frequency bands. The results of the multi-head attention mechanism are further extracted into deeper features through a feedforward network consisting of a 1-layer MLP and a 1-layer GeLU activation function, thus preserving the perception of spatial local structure. Through the above encoder structure, valuable information in the image frequency domain data is automatically extracted into the vector corresponding to the "classification label." Finally, the "classification label" vector is taken out and fed into a classifier consisting of a 1-layer MLP to calculate the probability, thus obtaining the final classification result.

[0068] In training the neural network model designed in this application, the cross-entropy loss function is used for gradient descent training. Normalization layers and residual connections are introduced into the network structure to maintain training stability, and frequency band dropout is introduced to randomly discard frequency band information, improving the model's robustness. Through the above model structure and training strategy, this network can directly complete image classification, detection, and other recognition tasks in the frequency domain. Compared to traditional schemes that require inverse transformation, this scheme does not require inverse discrete cosine transform before inference. Furthermore, due to the data compression capability of discrete cosine transform, the network input data is greatly reduced, thus significantly reducing the computational load required for network inference. Overall, this results in a reduction of approximately one-third of the computational load, leading to faster inference speed and lower latency. Simultaneously, its inference accuracy does not decrease significantly, and with model fine-tuning, it can even surpass the performance of some neural networks that directly input image domain data.

[0069] The static random access memory 19 in the image recognition module 3 is tightly coupled to the neural network processing unit 18 and is specifically used to store the weight and bias parameters of the dedicated neural network. During system startup or initialization, the weight data of the neural network model is loaded from external non-volatile memory (such as Flash) into the static random access memory 19. During inference, the neural network processing unit 18 directly and rapidly reads the weight parameters from the on-chip static random access memory 19 and performs calculations with the input data. This on-chip storage and on-chip computation architecture greatly reduces the power consumption and latency caused by accessing external memory, providing a crucial guarantee for achieving low-power, high-real-time recognition. After completing the forward propagation calculation, the neural network processing unit 18 obtains the final target recognition result. This result is sent to the output packaging unit 20, which encapsulates the result data using a communication protocol and finally outputs it to the host computer or other execution mechanisms. The output result can be defined as a classification label, target bounding box coordinates, semantic segmentation map, etc., according to task requirements. In summary, the image recognition module 3 of this embodiment can run a neural network model specifically designed for discrete cosine transform (DCT). Under the coordinated operation of the "control recognition" command, it achieves high-speed, low-power intelligent parsing of compressed domain data. This module works perfectly in conjunction with the aforementioned imaging front-end that can output DCT coefficient data, together forming an end-to-end efficient processing system from information perception to information understanding, completely eliminating redundant data conversion and transmission links in the traditional process.

[0070] System control module 4 is the system-level control and management hub. As a single master device, it communicates and coordinates with all other modules within the system through different bus protocols, ensuring the orderly and efficient execution of the entire process from image acquisition to target recognition. Its block diagram is as follows: Figure 8As shown, the system includes an on-chip slave unit SPI 21, an on-chip master unit APB 22, and a top-level control unit 23. The on-chip slave unit SPI 21 communicates with an external host (such as a host computer or application processor) using the SPI protocol. This interface includes four standard signal lines: Chip Select (CS), Master Output Slave Input Data Line (MOSI), Master Input Slave Output Data Line (MISO), and Serial Clock (SCK). The external host, acting as the SPI master, accesses (reads and writes) the register file 24 inside the system control module 4 by writing specific data frames to this interface, thereby enabling flexible configuration and status monitoring of the entire system's operating parameters. The on-chip master unit APB 22 communicates with other functional modules within the system (image sensor 1, discrete cosine transform readout circuit module 2, and image recognition module 3). This unit contains a complete set of APB protocol signals, including: address signal (PAddr), write data signal (PWData), read data signal (PRData), write enable (PWrite), device select signal (PSel), and enable signal (PEnable). System control module 4, acting as the master device on the APB bus, actively initiates read / write transactions through this interface, accurately writing configuration parameters stored in the register file into the control registers of each slave device, or reading the status registers of the slave devices. The top-level control unit 23 determines the system's operation at the highest level. Internally, it includes a register file 24 containing a series of configuration registers and status registers mapped to specific memory addresses. The configuration registers store control parameters from the external host, such as the image sensor's exposure time, operating mode, DCT transform block size, and neural network model selection. The status registers reflect the current operating status (e.g., "busy," "idle," "complete") of each submodule (e.g., sensor, DCT circuit, NPU) in real time, for kernel querying or interrupt generation. Based on the register configuration and system status, three key control command signals are ultimately generated: "Control Imaging," "Control Readout," and "Control Recognition."

[0071] Example 2

[0072] This embodiment provides an efficient compressed sensing target recognition method based on discrete cosine transform. See [link to relevant documentation]. Figure 2 The method includes:

[0073] This application proposes an efficient compressed sensing target recognition method based on discrete cosine transform, the specific steps of which are as follows: Figure 9 As shown.

[0074] S1, the system control module 4 powers on and starts up, loads the firmware, and completes a self-test. Then, based on preset configuration parameters or those received from an external host, it initializes and configures the image sensor 1, the discrete cosine transform readout circuit module 2, and the image recognition module 3. These configuration parameters include, but are not limited to, imaging resolution, exposure time, discrete cosine transform block size, and neural network model selection, preparing for subsequent steps.

[0075] S2, after initialization and configuration, upon receiving the start-up trigger signal from the host computer, the image sensor 1 begins operation. Its pixel array 6 responds to the incident light signal to sense light and converts the light signal into an analog electrical signal. Under the precise timing control of the system control module 4, the row drive logic unit 7 and the column drive logic unit 8 work together to complete the row-by-row scanning and readout of the image.

[0076] S3, the pixel-domain data output by image sensor 1 is directly fed into discrete cosine transform readout circuit module 2. This module performs block-wise two-dimensional discrete cosine transform calculations on the input data in real time at the hardware level, converting the spatial domain image data into a discrete cosine transform coefficient data stream in the frequency domain. This step directly outputs discrete cosine transform coefficients representing the frequency domain features of the image, achieving data compression and efficient feature extraction.

[0077] S4, the discrete cosine transform coefficient data stream obtained above is directly transmitted to the image recognition module 3. The neural network processing unit 18 in this module calls a pre-trained neural network model optimized for processing frequency domain features, directly performing forward propagation inference in the discrete cosine transform coefficient space. This process completely avoids the traditional inverse transform stage, greatly reducing computational latency and power consumption. The neural network analyzes and calculates based on the input frequency domain features to obtain preliminary recognition results.

[0078] S5, the image recognition module 3 performs final processing and formatting of the neural network calculation results, generates target recognition results (such as classification labels, bounding box coordinates, etc.), and outputs the results to the host computer or external actuator through its output interface, thereby completing a complete recognition task.

[0079] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high efficient compressive sensing target recognition system based on discrete cosine transform, characterized in that, The system comprises an image sensor, a discrete cosine transform readout circuit module, an image recognition module and a system control module, the image sensor, the discrete cosine transform readout circuit module and the image recognition module are sequentially connected and are connected with the system control module; The image sensor is used for converting incident light signals into pixel domain spatial distribution electric signals, the discrete cosine transform readout circuit module receives the pixel domain spatial distribution electric signals, obtains two-dimensional discrete cosine transform results through two-stage analog vector matrix multiplication circuit, directly outputs compressed discrete cosine transform frequency domain coefficients, the image recognition module directly takes the discrete cosine transform frequency domain coefficients as input to obtain target recognition results, and the system control module is used for realizing control of system and total interaction outside the chip, storing configuration information and controlling imaging time sequence of the image sensor, readout time sequence of the discrete cosine transform readout circuit module and recognition behavior of the image recognition module according to the configuration information. The image sensor comprises a pixel array, a row drive logic unit, a column drive logic unit and a sensor internal system control unit, the sensor internal system control unit is connected with the system control module, the sensor internal system control unit is connected with the row drive logic unit and the column drive logic unit respectively, the row drive logic unit is coupled with row lines of the pixel array and is used for providing row drive state voltage with flexible and adjustable time sequence for the pixel array, the column drive logic unit is coupled with column lines of the pixel array and is used for processing and reading signals in the column direction, under synchronous control of the sensor internal system control unit, working time sequence of the column drive logic unit and row scanning time sequence of the row drive logic unit are accurately matched, thereby realizing parallel reading and processing of column pixel signals in the selected row and obtaining pixel domain original analog data, i.e. pixel domain data, the pixel array is composed of M*N light sensitive devices in a two-dimensional matrix form.

2. The system of claim 1, wherein, The discrete cosine transform readout circuit module comprises a sample and hold circuit, a row direction analog vector matrix multiplication circuit, a column direction analog vector matrix multiplication circuit, an analog-digital conversion circuit, a pipeline unit and a mobile industry processor interface unit one which are sequentially connected, the sample and hold circuit is used for receiving pixel domain data from the image sensor, converting charge value data into voltage value and holding, and performing centering on the value, the row direction analog vector matrix multiplication circuit and the column direction analog vector matrix multiplication circuit respectively complete one-dimensional row vector discrete cosine transform to obtain two-dimensional discrete cosine transform analog voltage, the analog-digital conversion circuit is used for converting the analog voltage into digital signal discrete cosine transform coefficients, the pipeline unit is used for temporarily storing and time sequence reorganizing the discrete cosine transform coefficients, and the mobile industry processor interface unit one is used for packaging data according to a predetermined standard protocol format and then outputting.

3. The system of claim 2, wherein, The row direction analog vector matrix multiplication circuit and the column direction analog vector matrix multiplication circuit are composed of a matrix network composed of a plurality of resistance programmable resistors or capacitance programmable capacitors, the value of the resistance programmable resistor is encoded as the inverse of the cosine base in the discrete cosine transform, and the value of the capacitance programmable capacitor is encoded as the cosine base in the discrete cosine transform.

4. The system of claim 3, wherein, The image recognition module includes a mobile industry processor interface unit two, a neural network processing unit, a static random access memory, and an output packaging unit; wherein the mobile industry processor interface unit two, the neural network processing unit, and the output packaging unit are connected in sequence, and the static random access memory is connected with the neural network processing unit; the mobile industry processor interface unit two is configured to receive data output by the mobile industry processor interface unit one, align with the standard protocol of the discrete cosine transform readout circuit module, and perform preprocessing operations; the neural network processing unit is configured to directly perform an image recognition task in a frequency domain by using a pre-trained neural network model; the output packaging unit is configured to output image recognition result data after communication protocol encapsulation; and the static random access memory is configured to store weight and bias parameters of the neural network.

5. The system of claim 4, wherein, The neural network model includes an input layer, a Zig-Zag coding layer, a superimposed information layer, a same-frequency fusion layer, a cross-band multi-head attention feature extraction layer, and an output layer connected in sequence. The input layer is configured to receive a discrete cosine transform coefficient matrix, the Zig-Zag coding layer is configured to block the discrete cosine transform coefficient matrix and perform Zig-Zag coding, the superimposed information layer is configured to superimpose a quantization quality factor matrix and a classification label on the Zig-Zag coded matrix, the same-frequency fusion layer is configured to integrally fuse information of the same frequency band in different blocks, the cross-band multi-head attention feature extraction layer adopts a Transformer-like architecture and extracts features through 12 layers of encoders, each layer of encoder is composed of a multi-head attention mechanism and a feedforward network, and the output layer obtains a target recognition result through the classification label.

6. The system of claim 5, wherein, The system control module includes an on-chip slave unit, an on-chip host unit, and a top-level control unit, and the on-chip slave unit and the on-chip host unit are connected with the top-level control unit; The on-chip slave unit communicates with an external host using an SPI protocol to configure and monitor working parameters of the system; The on-chip host unit communicates with other functional modules in the system using an APB protocol; The top-level control unit internally includes a register stack, the register stack includes a series of configuration registers and state registers mapped to specific memory addresses, and finally generates three key control instruction signals: "control imaging", "control readout", and "control recognition".

7. A high efficient compressive sensing target recognition method based on discrete cosine transform, characterized in that, The method is implemented based on the system of any one of claims 1-6, and the method includes: Step 1, the system control module is powered on and started, and the image sensor, the discrete cosine transform readout circuit module, and the image recognition module are initialized and configured according to preset or received configuration parameters from an external host; Step 2, the image sensor uses a pixel array to respond to incident light signals, converts the light signals into analog electrical signals, and outputs pixel domain data; Step 3, after receiving the pixel domain data, the discrete cosine transform readout circuit module performs a two-dimensional discrete cosine transform calculation on the input data, converts the pixel domain image data into a discrete cosine transform coefficient data stream in the frequency domain, and transmits the data stream to an image recognition module; Step 4, the image recognition module directly performs an image recognition task in the frequency domain using a pre-trained neural network model, and outputs a recognition result.

8. The method of claim 7, wherein, The neural network model training uses a cross-entropy loss function to perform gradient descent method training.

9. Application of the system of any one of claims 1-6 or the method of any one of claims 7-8 in automatic driving, security monitoring, unmanned aerial vehicle vision, and Internet of Things.

Images