Security enhanced deployment method of YOLOv4-tiny model under FPGA (Field Programmable Gate Array) architecture

Abstract

The invention discloses a security enhanced deployment method of a YOLOv4-Tiny model under an FPGA architecture, and the method comprises the steps: dividing a YOLOv4-Tiny network into four basic units, and setting a security verification breakpoint at a key position; 8-bit quantization weights are stored in an on-chip Flash, and a security flash memory communication module is designed by combining a dynamic key verification mechanism; a three-level buffering mechanism with a data protection function is constructed, and parallel calculation and data loading are achieved; designing a convolution calculation four-stage assembly line integrated with time sequence confusion, and combining three space parallel strategies; enabling the main state machine and the block processing state machine to cooperatively control data flow through a safe handshake mechanism; and performing integrity verification between network layers, and finally performing signature confirmation on a detection result and outputting the detection result. A pure FPGA implementation scheme is adopted, the time sequence characteristics of high safety, low power consumption, strong environmental adaptability and certainty are achieved, and the method is particularly suitable for deploying a target detection system in resource-limited and high-safety application scenes such as military equipment, industrial automation and aerospace.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent hardware, in particular to a secure enhanced deployment method of a YOLOv4-tiny model under an FPGA architecture. BACKGROUND

[0002] YOLOv4-Tiny is a lightweight variant of the YOLO (You Only Look Once) series of object detection algorithms, proposed by Alexey Bochkovskiy, Chien-Yao Wang and Hong-Yuan Mark Liao in 2020. The model simplifies the network structure of YOLOv4, reducing the original version of about 162 layers to about 29 layers, reducing the model size from about 250MB to about 23MB, reducing the calculation amount by about 95%, while sacrificing only about 10-15% of the detection accuracy (mAP), making it particularly suitable for deployment on edge devices with limited computing resources.

[0003] FPGA (Field-Programmable Gate Array) is a semiconductor device that can be reconfigured through a hardware description language, with high parallelism, low latency and deterministic timing characteristics. In the field of artificial intelligence acceleration, FPGA, with its customizable data flow architecture, precise resource allocation capability and flexible bit width optimization, has become an ideal platform for deep learning inference acceleration, especially suitable for real-time application scenarios.

[0004] Compared with traditional SoC (System-on-Chip) solutions, pure FPGA implementation has significant advantages: first, it is more power efficient, reducing energy consumption by 30-70% in typical applications; second, pure FPGA solutions have stronger environmental adaptability, can work stably in an industrial temperature range of -40℃ to 85℃, and have stronger resistance to electromagnetic interference; in addition, the deterministic delay characteristics of pure FPGA implementation ensure the timing stability of the inference process, which is very suitable for critical tasks that require real-time response. These characteristics make FPGA the preferred platform for deploying object detection systems in harsh environments such as military equipment, industrial automation, polar scientific research and aerospace.

[0005] The information disclosed in this BACKGROUND section is only intended to increase the understanding of the general background of the application, and should not be considered as recognition or in any form as suggesting that this information constitutes prior art that is known to those of ordinary skill in the art. SUMMARY

[0006] The technical problem to be solved by the present application is to overcome the above technical defects, and to provide a secure enhanced deployment method of a YOLOv4-tiny model under an FPGA architecture, which significantly reduces resource consumption through an optimized computing architecture and quantization strategy, is implemented by pure FPGA resources, and is free from dependence on an upper computer controller, and a multi-level security mechanism is integrated to effectively prevent algorithm theft and unauthorized access, and to protect core intellectual property rights, and the hardware-based security implementation has extremely low performance overhead and maintains the real-time performance of the algorithm.

[0007] To solve the above problems, the technical scheme of the present application is a secure enhanced deployment method of a YOLOv4-tiny model under an FPGA architecture, which comprises the following steps:

[0008] Step one, according to four basic units of a standard convolution module, a point convolution module (1x1 convolution), a max pooling module and an up-sampling module, the YOLOv4 detection network is divided into three parts of a backbone network, a neck network and a detection head, the standard convolution module is composed of a 3x3 convolution layer and a Leaky ReLU activation function; the point convolution module is composed of a 1x1 convolution layer and an activation function; the max pooling module is used for down-sampling operation; and the up-sampling module is used for feature map size recovery;

[0009] Step two, the quantized weights are stored in an on-chip Flash, a flash communication security module with dynamic key verification is designed, and state verification is performed before weight reading; dynamic encryption processing is performed on the weight index table for fast batch data retrieval; the weight reading module adopts a secure prefetch architecture, so that the verification process and weight loading are executed in parallel, the precision is restored through a dequantization module, the weight data is organized in a hierarchical structure of weight-bias-convolution kernel dimensions, and the computing requirements of different convolution types are adapted;

[0010] Step three, a three-level buffer mechanism is designed, and multiple security functions are integrated: the weight cache uses dynamic XOR transformation technology to preload the next group of convolution kernel parameters; the intermediate result cache saves the calculation results and is associated with the security state, and generates alternative data when unauthorized operation occurs; the memory access layer implements a batch reading strategy and a random offset mechanism; when loading feature map data from SDRAM, dynamic scrambling based on clock cycles is applied, the buffer state is managed through a control signal integrated with the security verification system, and data prefetching and calculation processes are executed in parallel;

[0011] Step four, design a pipeline for convolution calculation that loads weights, loads feature maps, convolution calculation, and result output, while integrating security enhancement mechanisms: the convolution calculation stage adopts a parallel MAC pipeline structure with timing obfuscation, introduces controllable random delay between MAC units based on security level, and realizes a four-stage fine pipeline of multiplication, accumulation, bias addition, and output truncation; adopts three spatial parallel strategies of multiple convolution kernel parallel, multiple output channel parallel, channel grouping parallel, and feature map block parallel, and adds a dynamic selection mechanism for parallel paths; adds a key verification layer to the Leaky ReLU activation function, maximum pooling downsampling, and upsampling operations, and performs security checks during the processing;

[0012] Step five, store the calculation results to the external memory as the next layer input or as the final output, while performing reversible transformation encryption protection on the data, and automatically restoring the original value when reading; the main state machine and the block processing state machine work cooperatively through a secure handshake mechanism to enhance the security of the control flow and ensure the integrity and confidentiality of the data flow;

[0013] Step six, repeat steps two to five in sequence according to the network structure until all 29 layers are executed, while implementing an integrity verification mechanism between network layers, checking the output feature map after each layer is executed; based on the layer execution state, the working key is updated in real time to form a dynamic key chain that is unpredictable; after the calculation is completed, the detection processing module performs signature confirmation on the final result to ensure that the output data is not tampered with, enhancing the overall security of the system;

[0014] After the detection is completed, the prediction result and the label are output together to a display device.

[0015] As preferred, in step one, a multi-level protection mechanism is designed, and a state machine verification system based on dynamic keys is integrated for the SDRAM arbiter and the YOLO network inference control module to prevent unauthorized access and control flow hijacking; security verification breakpoints are set at four key positions of the network, the input layer verification breakpoint ensures the integrity of the initial data, the backbone-neck network transition breakpoint protects the feature extraction process, the neck-detection head transition breakpoint verifies the legality of feature fusion, and the detection head output preservation breakpoint prevents result tampering.

[0016] As preferred, the security verification breakpoints in step one include: input layer verification breakpoint, backbone-neck network transition breakpoint, neck-detection head transition breakpoint, and detection head output preservation breakpoint.

[0017] The 16-bit fixed-point data is scaled to 8-bit fixed-point type [-127, 128], and different scaling factors are allocated according to the data size of each layer, and the quantization formula is:

[0018]

[0019] where a is the minimum value of the quantization range, b is the maximum value of the quantization range, x is the original 16-bit weight value, q is the quantized 8-bit value, and scale_factor is the scaling factor of each parameter.

[0020] Preferably, the space allocation of the on-chip Flash in step two includes a Boot area, a scale factor area, a weight index area, a weight storage area, and a security verification area for storing verification templates and key derivation parameters.

[0021] Preferably, the feature map blocking processing in step three has parameters as follows:

[0022] The P4 layer (26x26) uses a block size of 16x16, and each group has 32 channels.

[0023] The P5 layer (13x13) uses a block size of 8x8, and each group has 64 channels.

[0024] In step three, the SDRAM is divided into a 2MB image storage area and a 6MB inference data storage area, and the access to the two areas is managed by an arbiter integrated with a security mechanism.

[0025] Preferably, the parallel MAC pipeline in step four includes a four-stage fine pipeline.

[0026] The first stage implements data preparation and sliding window generation.

[0027] The second stage implements multiplication operations.

[0028] The third stage performs accumulation operations.

[0029] The fourth stage performs bias addition and activation function calculation.

[0030] The activation function in step four uses Leaky ReLU, whose mathematical expression is as follows:

[0031]

[0032] The system supports two activation modes: act=0 for linear activation and act=1 for Leaky ReLU activation.

[0033] Preferably, the security handshake mechanism in step five includes a three-stage protocol of request-verification-confirmation, and a complete handshake process is required for each state transition.

[0034] As preferred, the detection processing module in the step six comprises a detection result processing module, a bounding box drawing module, a pixel position calculation module, a label display module and a text rendering module; the text rendering module comprises two read-only memories of font ROM and category label ROM.

[0035] As preferred, the dynamic key update adopts a multi-factor derivation mechanism to generate an unpredictable session key based on timing information, execution state and physical characteristics.

[0036] Compared with the prior art, the present application has the following advantages:

[0037] 1. The present application significantly reduces resource consumption through an optimized computing architecture and quantization strategy, and only increases about 2-3% additional resources to achieve comprehensive security protection.

[0038] 2. The present application is implemented using pure FPGA resources, eliminating the dependence on the host controller, simplifying the system architecture, and eliminating the security risks of traditional processors.

[0039] 3. The integrated multi-level security mechanism of the present application effectively prevents algorithm theft and unauthorized access, protecting core intellectual property rights.

[0040] 4. The hardware-based security implementation of the present application has extremely low performance overhead, maintaining the real-time performance of the algorithm.

[0041] 5. The pure FPGA solution of the present application has excellent environmental adaptability and can work stably in an industrial temperature range of -40℃ to 85℃, and has stronger resistance to electromagnetic interference, making it particularly suitable for harsh environment application scenarios such as military, polar and aerospace. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 is a method flowchart for implementing the YOLOv4-Tiny detection network by FPGA according to an embodiment of the present application;

[0043] Figure 2 is a hierarchical structure schematic diagram of a security-enhanced YOLO-v4 network layer according to an embodiment of the present application;

[0044] Figure 3 is a three-level cache architecture schematic diagram according to an embodiment of the present application;

[0045] Figure 4 is a SDRAM arbiter function schematic diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the content of the application easier to be clearly understood, the technical solutions in the embodiments of the application will be clearly and completely described below in combination with the drawings in the embodiments of the application.

[0047] As shown in Figure 1 The application provides a secure enhanced deployment method of a YOLOv4-tiny model under an FPGA architecture, including the following steps:

[0048] Step one, according to four basic units of a standard convolution module (Conv-LeakyReLU), a point convolution module (1×1 convolution), a maximum pooling module and an up-sampling module, the YOLOv4 detection network is divided into three parts of a backbone network, a neck network and a detection head, wherein: the standard convolution module is composed of a 3×3 convolution layer and a Leaky ReLU activation function; the point convolution module is composed of a 1×1 convolution layer and an activation function; the maximum pooling module is used for down-sampling operation; the up-sampling module is used for feature map size recovery; in order to enhance the security of the algorithm, a multi-level protection mechanism is designed, a state machine verification system based on a dynamic key is integrated for an SDRAM arbitrator and a YOLO network inference control module, unauthorized access and control flow hijacking are effectively prevented; security verification breakpoints are set at four key positions of the network, an input layer verification breakpoint ensures the integrity of initial data, a backbone-neck network transition breakpoint protects the feature extraction process, a neck-detection head transition breakpoint verifies the legality of feature fusion, and a detection head output protection breakpoint prevents result tampering;

[0049] Generally, YOLOv4 is mainly composed of a convolution layer, a BN layer, a pooling layer and an activation function, the convolution layer has characteristics of calculation-intensive and weight sharing, and is the calculation core of the entire network. In the weight storage process, the weight is pre-processed by the BN layer calculation, the network no longer needs such a link of the BN layer, and the backbone calculation network only has the convolution layer, the activation layer and the pooling layer;

[0050] As shown in Figure 2 In order to design the state machine, the YOLO module is divided into three modules of a weight loading module, a backbone network execution module and a feature detection module which are executed in turn; a permission control state machine is set to ensure the security of the system. The permission in the control state adopts a four-state design, after initialization, it enters a strict verification state, and after successful verification, it gradually promotes the execution permission. The verification tokens are transmitted between the modules through a secure handshake mechanism, so that only the modules that pass the previous verification can obtain the execution permission;

[0051] The system adopts a multi-factor dynamic key derivation mechanism, generates unpredictable session keys based on timing information, execution state and physical characteristics. The key is automatically updated when the key network layer is converted, forming a forward secure authentication chain, effectively preventing key extraction and replay attacks, and ensuring that the system forms a complete authentication chain between the weight loading, network execution and feature detection three main stages.

[0052] Step two, store the quantized weight in the on-chip Flash, design a flash communication security module with dynamic key verification, perform state verification before weight reading; Dynamic encryption is implemented on the weight index table for fast batch data retrieval; The weight reading module uses a secure prefetch architecture, so that the verification process and weight loading are executed in parallel. Through the weight_dequantizer module to restore the precision, the weight data is organized in a hierarchical structure of weight-bias-convolution kernel dimensions to adapt to the calculation requirements of different convolution types.

[0053] The storage scheme of the weight, after setting the algorithm to 16-bit fixed-point number, in order to further reduce the storage space occupied, design the storage scheme of 8-bit fixed-point weight which can maintain the inference accuracy.

[0054] After getting 16-bit fixed-point weight, scale the 16-bit fixed-point data to 8-bit fixed-point type in [-127, 128], in order to ensure the accuracy of scaling, according to the data size of each layer, allocate different size scaling factors, the minimum value and the maximum value, that is, a represents the minimum value of the quantization range, and b represents the maximum value of the quantization range. Then get the quantization value q by rounding, that is, scale the 16-bit fixed-point number to an integer in the range of [-127, 128] in proportion.

[0055] The quantization formula is:

[0056]

[0057] The formula for dequantizing the read function is:

[0058]

[0059] The storage scheme on the on-chip Flash is designed to allocate space in the Flash chip, including the Boot area, the scaling factor area, the weight index area and the weight storage area. The Boot area stores system and driver files, the scaling factor area stores the scaling factor required for weight dequantization of each layer, the weight index area stores the address data table of all weights, and the weight data area stores the 8-bit weight data of different layers according to the address. The weight dequantization is 16 bits, which is used for function inference.

[0060] In addition to the Boot area, scaling factor area, weight index area and weight storage area, a security verification area is added for storing verification templates and key derivation parameters. The flash communication security module workflow is as follows: first, the current execution phase identifier and dynamic token are input through the state machine before the read operation is triggered; second, the security module calculates the session key and verifies the access authority; after the verification is passed, the time-based decryption transformation is applied to the weight index table, so that the real address mapping is visible; finally, the weight reading and the inverse quantization process are executed in parallel, and the decrypted index information guides the pre-fetch module to extract the weight data from the Flash in batches. The entire verification process is completed in the pre-fetch stage of the weight loading pipeline, and the calculation masking technology is used to ensure that sensitive information is not leaked even in the case of verification failure, and at the same time, the weight data is processed by linear transformation during transmission, and after reading, it is restored to 16-bit fixed-point precision by the inverse quantization module;

[0061] Step three, design a three-level buffer mechanism, integrate multiple security functions: weight cache uses dynamic XOR transformation technology to preload the next group of convolution kernel parameters; intermediate result cache saves calculation results and is associated with security state, generates alternative data when unauthorized operation; memory access layer implements batch reading strategy and random offset mechanism. When loading feature map data from SDRAM, dynamic scrambling based on clock cycles is applied, P4 layer (26x26) uses 16x16 block, 32 channels per group, P5 layer (13x13) uses 8x8 block, 64 channels per group. The buffer state is managed by the control signal integrated with the security verification system, ensuring that data prefetching and calculation processes are executed in parallel;

[0062] The first level cache is composed of Block RAM, used for high-speed calculation and image processing, and integrates dynamic XOR transformation technology to provide data protection when preloading the next group of convolution kernel parameters;

[0063] The second level cache is implemented by FIFO and double-ended RAM, not only communicating the first level cache and slow devices, but also saving intermediate calculation results and associating with security state to ensure that only authorized operations can access real data;

[0064] The third level cache is composed of large-capacity SDRAM, managed by a controller and a security arbiter, implementing batch reading strategy and random offset mechanism to enhance data access security.

[0065] To meet the needs of video transmission and image inference, the 8MB SDRAM is divided into a 2MB image storage block and a 6MB inference data storage block. When loading feature map data from SDRAM, clock cycle-based dynamic scrambling is applied. The P4 layer (26×26) uses a 16×16 block configuration with 32 channels per group, and the P5 layer (13×13) uses an 8×8 block configuration with 64 channels per group. The entire buffer system is managed by control signals integrated with a security verification mechanism, ensuring efficient parallel processing of data prefetching and computation while maintaining data security. The storage architecture is attached. Figure 3 As shown in the attached diagram, the arbitrator logic is as follows: Figure 4 As shown.

[0066] Step 4: Design a pipeline for convolution computation that loads weights → loads feature maps → performs convolution computation → outputs the result, while integrating security enhancement mechanisms: The convolution computation stage adopts a parallel MAC pipeline structure with temporal obfuscation. Controllable random delays based on security levels are introduced between MAC units to achieve a four-stage fine pipeline of multiplication, accumulation, bias addition, and output truncation; three spatial parallelism strategies are adopted: multi-kernel parallelism (simultaneous computation of multiple output channels), channel grouping parallelism (processing input channels in groups), and feature map block parallelism (each block is computed independently), and a dynamic selection mechanism is added to the parallel path; a key verification layer is added to the Leaky ReLU activation function, max pooling downsampling, and upsampling operations to perform security checks during processing;

[0067] The computational load of the YOLOv4 detection network is mainly concentrated in the convolutional layers. A security-enhanced parallel convolutional architecture is designed to maintain high computational efficiency while providing data protection mechanisms. The system implements a complete pipeline from loading weights to loading feature maps, performing convolutional computation, and outputting results, with security verification functions integrated at each stage.

[0068] like Figure 2 As shown, this system fully utilizes the four independences of convolutional computation (independence of weights within the convolutional kernel, independence of different input channels, independence of different convolutional windows of the same feature map, and independence of different output channels) to design an efficient parallel architecture. Each parallel convolutional unit processes four convolutional kernels simultaneously. Controllable random delays based on safety levels are introduced between MAC units to achieve temporal obfuscation without affecting overall throughput. The system employs a grouping computation strategy, dividing the input channels into several groups (Tn=16) and the output channels into several groups (Tm=16). A dynamic selection mechanism is added to the parallel paths, making the data flow pattern unpredictable.

[0069] The convolution kernel calculation adopts a four-stage fine pipeline structure: the first stage realizes data preparation, sliding window generation and security verification; the second stage performs multiplication operation; the third stage completes accumulation operation; and the fourth stage performs bias addition and activation function calculation. Each pipeline stage integrates a targeted security check mechanism while maintaining a one-clock-cycle delay.

[0070] To realize the convolution sliding window, the system uses FIFO and shift registers to build a sliding window circuit. Taking a 3x3 convolution kernel as an example, 3 FIFOs with a depth greater than the width of the feature map and 3x3 shift registers are used to realize the sliding window. For 1x1 convolution, a matrix multiplication calculation is directly performed using a multiplier array, without the need for a sliding window structure.

[0071] The convolution calculation uses DSP resource acceleration, as shown in Figure 2 Each multiplication unit uses a DSP block to realize an efficient multiply-accumulate (MAC) operation, and the use_dsp="yes" instruction is used to explicitly instruct the synthesis tool to use DSP resources. For 16-bit fixed-point multiplication, each DSP block can realize a complete MAC operation, greatly improving the calculation performance.

[0072] The activation function uses Leaky ReLU, and its mathematical expression is as follows:

[0073]

[0074] The system supports two activation modes, which are selected by the act parameter: act=0 for linear activation (no activation), and act=1 for Leaky ReLU activation. The activation function implementation uses efficient conditional selection logic, which only requires a fixed-point multiplier and a comparator.

[0075] Step five, store the calculation results to the external memory as the next layer input or as the final output, and perform reversible transformation encryption protection on the data, automatically restore the original value when reading; the main state machine and the sub-block processing state machine work cooperatively through a secure handshake mechanism, enhancing the security of the control flow and ensuring the integrity and confidentiality of the data flow;

[0076] The main state machine is responsible for combining the image device and the inference module, and its workflow after security enhancement includes: initialization state (establishing security parameters) → display mode (transparent data transmission) → inference module preparation (permission verification) → inference module execution (protecting calculation) → display mode preparation (result verification) → display mode execution (safe output).

[0077] In this process, the calculation result is stored in the external SDRAM, and the reversible transformation based on the current execution state is applied, so that the stored intermediate feature map data presents a meaningless state to unauthorized access, and the original value is automatically restored by reverse transformation when reading the next layer. A three-stage security handshake protocol is designed between the main state machine and the block processing state machine: request-verification-confirmation. A complete handshake process is required for each state transition to prevent illegal state jumping or control flow hijacking.

[0078] The integrity of data flow is guaranteed by the checksum mechanism, and each data block transmission is accompanied by checksum information. The confidentiality is realized by the state-dependent scrambling function, which makes the same data present different storage patterns in different execution stages, effectively preventing pattern analysis and data extraction attacks.

[0079] Step six, repeat steps two to five in the order of network structure until all 29 layers are executed. At the same time, implement the integrity verification mechanism between network layers, and check the output feature map after each layer execution; update the working key based on the layer execution state in real time to form an unpredictable dynamic key chain; after the calculation is completed, the detection processing module performs signature confirmation on the final result to ensure that the output data is not tampered with, enhancing the overall security of the system.

[0080] The display subsystem is located at the back end of the entire YOLOv4 detection network, mainly responsible for real-time visualization of the detection results of the network. The display subsystem is composed of five parts: detection result processing module, bounding box drawing module, pixel position calculation module, label display module and text rendering module.

[0081] The detection result processing module receives the original detection data output from the YOLOv4 network, including target confidence, class probability and bounding box coordinates. This module uses a four-state finite state machine to receive up to four detection results per batch, filters targets with confidence thresholds below the set value, and converts the normalized coordinates output by YOLOv4 to actual pixel coordinates under the resolution of the display. The converted results are passed to the bounding box drawing module through a three-stage pipeline.

[0082] The bounding box drawing module is responsible for real-time superimposition of visual target detection results on the original image. This module uses a pixel-level parallel processing strategy to determine whether the current scanning pixel is located on the edge of a detection box, in the label area or inside. The coordinate point calculation method is as follows, pixel addr box x and box y represent the coordinate position of the detection box, and img width is the number of image rows.

[0083] pixel addr = boxy x img width + box x

[0084] In addition, the module will add a label area above each bounding box, showing the detected object class name and confidence percentage. The pixel position calculation module tracks the current display scanning position in real time, converting VGA / HDMI synchronization signals into a standardized pixel coordinate system. The label display module is responsible for generating text labels for each detected target. This module uses a text display buffer. The text rendering module is responsible for converting ASCII text characters into pixel matrices. This module contains two read-only memories, a font ROM and a category label ROM. The font ROM stores the dot matrix data of standard ASCII characters, and the category label ROM stores the name strings of 80 COCO dataset categories.

[0085] The above describes the present application and its embodiments, which are not limited, and the drawings only show one of the embodiments of the present application. In general, if a person skilled in the art is inspired by it, without departing from the purpose of the present application, without creative design, similar structural ways and embodiments of the technical solution should belong to the protection scope of the present application.

Claims

1. A method for secure enhanced deployment of a YOLOv4-tiny model under an FPGA architecture, characterized in that: Comprise the following steps: Step one, according to the standard convolution module, point convolution module (1x1 convolution), maximum pooling module and four kinds of basic units of upsampling module, YOLOv4 detection network is divided into three parts of main network, neck network and detection head; The four key positions of the network are set security verification breakpoints, the input layer verification breakpoint ensures the integrity of the initial data, the main-neck network transition breakpoint protects the feature extraction process, the neck-detection head transition breakpoint verifies the legality of feature fusion, and the detection head output breakpoint prevents result tampering; Step two, store the quantized weight in the on-chip Flash, design a flash communication security module with dynamic key verification, and perform state verification before weight reading; Step three, design a three-level buffer mechanism, integrate multiple security functions, apply dynamic scrambling based on clock cycles when loading feature map data from SDRAM, and manage the buffer state through the control signal integrated with the security verification system, to ensure that data prefetching and calculation processes are executed in parallel; Step four, design a pipeline for convolution calculation, load weights, load feature maps, convolution calculation, and result output, use multiple convolution kernels in parallel, calculate multiple output channels simultaneously, use channel grouping parallel and feature map block parallel three spatial parallel strategies, and add a dynamic selection mechanism to the parallel path; Step five, store the calculation results to the external memory as the next layer input or as the final output, and perform reversible transformation encryption protection on the data, which automatically restores the original value when reading; The main state machine and the block processing state machine work cooperatively through a secure handshake mechanism; Step six, repeat steps two to five in the order of network structure until the 29th layer is executed completely, and implement an integrity verification mechanism between network layers to check the output feature map after each layer is executed; Based on the real-time update of the layer execution state, the working key is updated, forming an unpredictable dynamic key chain; After the calculation is completed, the detection processing module confirms the signature of the final result to ensure that the output data is not tampered with; After the detection is completed, the prediction results and labels are output to the display device.

2. The method of claim 1, wherein the method is characterized in that: In step one, a multi-level protection mechanism is designed, and a state machine verification system based on dynamic keys is integrated for SDRAM arbiter and YOLO network inference control module to prevent unauthorized access and control flow hijacking.

3. The method of claim 1, wherein the method is characterized in that: The standard convolution module in step one is composed of a 3x3 convolution layer and a Leaky ReLU activation function; The point convolution module is composed of a 1x1 convolution layer and an activation function; The maximum pooling module is used for downsampling operation; The upsampling module is used for feature map size recovery; The security verification breakpoints in step one include: input layer verification breakpoint, main-neck network transition breakpoint, neck-detection head transition breakpoint and detection head output breakpoint: The 16-bit fixed-point data is scaled to 8-bit fixed-point type [-127, 128], different size scaling factors are allocated according to the data size of each layer, and the quantization formula is: Where a is the minimum value of the quantization range, b is the maximum value of the quantization range, x is the original 16-bit weight value, q is the quantized 8-bit value, and scale_factor is the scaling factor of each parameter.

4. The method of claim 1, wherein the method is characterized in that: The step two dynamically encrypts the weight index table for fast batch data retrieval; the weight reading module uses a secure prefetch architecture, enabling the verification process to be performed in parallel with weight loading, the precision is restored through the inverse quantization module, and the weight data is organized in a hierarchical structure according to the weight-bias-convolution kernel dimensions, which adapts to the calculation requirements of different convolution types; the on-chip Flash space allocation in the step two includes a Boot area, a scaling factor area, a weight index area, a weight storage area, and a security verification area, and the security verification area is used to store the verification template and key derivation parameters.

5. The method of claim 1, wherein the method is characterized in that: The step three buffer mechanism includes: the weight buffer uses dynamic XOR transformation technology to preload the next group of convolution kernel parameters; the intermediate result buffer saves the calculation results and is associated with the security state, and generates alternative data when unauthorized operation occurs; the memory access layer implements a batch reading strategy and a random offset mechanism; The step three feature map blocking processing parameters are: The P4 layer (26x26) uses a block size of 16x16, and each group has 32 channels; The P5 layer (13x13) uses a block size of 8x8, and each group has 64 channels; The step three SDRAM is divided into a 2MB image storage area and a 6MB inference data storage area, and the arbitrator integrated with the security mechanism manages the access to the two areas.

6. The method of claim 1, wherein the method is characterized in that: The step four also integrates a security enhancement mechanism: the convolution calculation stage uses a parallel MAC pipeline structure with timing confusion, and a controllable random delay based on security levels is introduced between MAC units to realize a four-stage fine pipeline of multiplication, accumulation, bias addition, and output truncation; The step four parallel MAC pipeline includes a four-stage fine pipeline: The first stage realizes data preparation and sliding window generation; The second stage realizes multiplication operation; The third stage performs accumulation operation; The fourth stage performs bias addition and activation function calculation. The step four activation function uses Leaky ReLU, and its mathematical expression is: The system supports two activation modes: act=0 for linear activation and act=1 for Leaky ReLU activation, and a key verification layer is added for Leaky ReLU activation function, maximum pooling downsampling, and upsampling operation, and security checks are performed during the processing.

7. The method of claim 1, wherein the method further comprises: The step five security handshake mechanism includes a three-stage protocol of request-verification-confirmation, and a complete handshake process is required for each state transition.

8. The method of claim 1, wherein the method is characterized in that: The step six detection processing module includes: a detection result processing module, a bounding box drawing module, a pixel position calculation module, a label display module, and a text rendering module; the text rendering module includes two read-only memories of font ROM and category label ROM.

9. The method of claim 2, wherein the method further comprises: The dynamic key update uses a multi-factor derivation mechanism to generate an unpredictable session key based on timing information, execution state, and physical characteristics.

10. A system for deploying YOLOv4-tiny based on FPGA security enhancement, characterized in that, The system uses the method of any one of claims 1 to 9.

Images