A neural network model compression method based on structured intelligent camera

Abstract

The present invention discloses a structured intelligent camera, comprising a main control unit, a network model processing unit, a network model fine-tuning unit, a network model conversion unit and a network model testing unit, wherein the main control unit realizes management and control with the network model processing unit, the network model fine-tuning unit, the network model conversion unit and the network model testing unit respectively through a plurality of communication transmission units. Under the premise that various hardware devices are already determined, the present invention performs autonomous compression on the basis of an existing neural network model, and can obtain an adaptive structured intelligent camera AI chip network model, effectively solving the problem that the network model cannot exert its own performance due to hardware limitations, reducing the model memory and computational complexity while ensuring the target detection accuracy and detection rate. The entire operation is simple and convenient, and the entire process does not require human supervision and work. Through the setting of the control program and the construction of the environment, the network model has the ability of autonomous training and learning and autonomous update and iterative self-adaptation.

Description

Technical Field

[0001] The present invention belongs to the technical field of structured intelligent cameras and artificial intelligence neural networks, and in particular relates to a neural network model compression method based on structured intelligent cameras. Background Art

[0002] Neural network algorithms are currently the primary method for intelligent video image processing in structured smart cameras. Their primary advantages include high accuracy in object detection and recognition, as well as the ability to quickly and automatically extract relevant target features. However, their drawbacks stem from the need to build deeper and larger neural network models to achieve higher precision and accuracy. However, as models become deeper and larger, the number of parameters and computational complexity increase dramatically, requiring greater storage space, memory usage, and power consumption. Smart terminals and embedded devices cannot meet the stringent hardware requirements of large network models. Therefore, compressing neural networks to suit mobile applications while maintaining accuracy is crucial. Summary of the Invention

[0003] The purpose of the present invention is to overcome the above-mentioned shortcomings, and thus provide a neural network model compression method based on structured intelligent cameras, which establishes a neural network model compression and fine-tuning function module in a structured intelligent camera, so that the neural network model has strong autonomous compression and autonomous update iteration capabilities to adapt to the device hardware, and keeps the accuracy of detection and recognition of the neural network model in the same scenario within an allowable range. The number of parameters and complexity of the neural network model are significantly reduced, and the method is better adapted to the device-related hardware.

[0004] In order to achieve the above-mentioned design purpose, the technical solution adopted by the present invention is: a structured intelligent camera, including a main control unit, a network model processing unit, a network model fine-tuning unit, a network model conversion unit and a network model testing unit. The main control unit realizes management and control with the network model processing unit, the network model fine-tuning unit, the network model conversion unit and the network model testing unit respectively through multiple communication transmission units. The network model processing unit has a model compression module, which is connected to the front-end data receiving device through the communication transmission unit to receive the incoming network model; the network model fine-tuning unit has a model micro-training module; the network model conversion unit has a model parsing module and a quantization module, and the model is transplanted and used in the structured intelligent camera through the network model testing unit.

[0005] A neural network model compression method based on a structured intelligent camera is implemented by the following steps:

[0006] Step 1: Model processing: The structured smart camera obtains the initial network model from the front-end device through the network model processing unit. After the network model compression module compresses the obtained initial model, the compressed model is transferred to the model micro-training module of the network model fine-tuning unit.

[0007] Step 2: Model fine-tuning: The compressed model is fine-tuned through the model micro-training module of the network model fine-tuning unit to improve the precision and accuracy of model detection and recognition. The fine-tuned model is transmitted to the network model conversion unit through the communication transmission unit.

[0008] Step 3: Model quantization: The model parsing module of the network model conversion unit first parses the model to extract relevant data, which is then passed to the quantization module to quantize the model and convert it into a format suitable for the AI ​​chip in the smart camera.

[0009] Step 4: Model testing: The model is converted into the AI ​​chip data format and then transplanted into the AI ​​chip. The network model testing unit performs model testing to test the usage of the AI ​​chip when the compressed model is in use.

[0010] Step 5: Network model iteration: When the performance of the new network model obtained by compression is better than that of the previous network model, the new network model iteratively replaces the previous network model, and the iterative update stops after a certain number of iterations.

[0011] Beneficial effects of the present invention: When the present invention is used, a network model conversion unit is first required to quantize the model. The network model conversion unit is connected to the front-end device through the communication transmission unit to receive the incoming network model. The parsing module and the quantization module in the network model conversion unit parse and quantize the initial model and convert it into the data format of the AI ​​chip. After the conversion, the model is transplanted to the AI ​​chip in the test unit. The performance of the initial network model is tested by the AI ​​chip to obtain the performance data of the network model test and the practical data of the AI ​​chip. In order to compare with the compressed model later, the model is compressed by the network model processing unit and the network model processing unit is used to compress the model. The processing unit receives the incoming initial network model through the front-end device, performs relevant compression processing on the incoming network model through the network model compression module, passes the compressed network model to the fine-tuning unit, fine-tunes the compressed network model, and fine-tunes the network model through the model micro-training module. The number of training rounds and learning rate parameters are set. When the model training reaches a certain threshold, the training is stopped and the new model is passed to the conversion unit. The new model is parsed and quantified by the parsing module and the quantization module, and the converted data format is passed to the test unit. The model is calculated in the AI ​​chip in the test unit to obtain the test results, and the new network model is obtained through the management module. The performance of the previous model is compared. If the performance is better than the previous network model, the previous performance data is replaced, the result is notified to the fine-tuning unit, and the replaced model is transmitted to the network model processing unit for a new round of processing. With continuous iterative replacement, a network model suitable for the structured smart camera will be obtained. In order to make the model have good precision and accuracy in different environmental scenarios, efforts are made to build deeper and larger network models. This will cause the network model to consume a lot of memory and increase the amount of calculation. Due to the limitations of the current hardware development, the needs of the network model cannot be met, which greatly limits the performance of the network model. The present invention is a basic An intelligent adaptive compression method for autonomous adaptation and autonomous updating of network models of structured smart cameras; under the premise that various hardware devices have been determined, autonomous compression is performed on the basis of the existing neural network model to obtain a network model adapted to the AI ​​chip of the structured smart camera. This solves the problem that the network model cannot exert its own performance due to hardware limitations, reduces the memory and computational complexity of the model while ensuring the accuracy and detection rate of target detection. The entire operation is simple and convenient, and the entire process does not require human supervision and work. Through the control program setting and environment construction, the neural network model has the adaptive ability of autonomous compression and autonomous update iteration.Under the premise that various hardware devices have been determined, the present invention performs autonomous compression on the basis of the existing neural network model to obtain a network model that is adapted to the AI ​​chip of the structured intelligent camera, effectively solving the problem that the network model cannot exert its own performance due to hardware limitations, reducing the model memory and computational complexity while ensuring the accuracy and detection rate of target detection. The entire operation is simple and convenient, and the entire process does not require human supervision and work. Through the setting of management and control programs and the establishment of the environment, the network model has the adaptive ability of autonomous training and learning and autonomous update and iteration. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of the hardware architecture of a structured smart camera according to the present invention;

[0013] Figure 2 A schematic diagram of the software architecture of a neural network model compression method based on a structured intelligent camera according to the present invention;

[0014] Figure 3 This is a flow chart of the working principle of a neural network model compression method based on a structured intelligent camera in the present invention. DETAILED DESCRIPTION

[0015] The following describes a detailed description of specific embodiments of the present invention with reference to the accompanying drawings. A structured intelligent camera includes a main control unit, a network model processing unit, a network model fine-tuning unit, a network model conversion unit, and a network model testing unit. The main control unit manages and controls the network model processing unit, the network model fine-tuning unit, the network model conversion unit, and the network model testing unit through multiple communication transmission units. The network model processing unit includes a model compression module that connects to a front-end data receiving device via the communication transmission unit to receive an incoming network model. The network model fine-tuning unit includes a model micro-training module. The network model conversion unit includes a model parsing module and a quantization module. The network model testing unit allows the model to be transplanted and used in the structured intelligent camera.

[0016] A neural network model compression method based on a structured intelligent camera is implemented by the following steps:

[0017] Step 1: Model processing: The structured smart camera obtains the initial network model from the front-end device through the network model processing unit. After the network model compression module compresses the obtained initial model, the compressed model is transferred to the model micro-training module of the network model fine-tuning unit.

[0018] Step 2: Model fine-tuning: The compressed model is fine-tuned through the model micro-training module of the network model fine-tuning unit to improve the precision and accuracy of model detection and recognition. The fine-tuned model is transmitted to the network model conversion unit through the communication transmission unit.

[0019] Step 3: Model quantization: The model parsing module of the network model conversion unit first parses the model to extract relevant data, which is then passed to the quantization module to quantize the model and convert it into a format suitable for the AI ​​chip in the smart camera.

[0020] Step 4: Model testing: The model is converted into the AI ​​chip data format and then transplanted into the AI ​​chip. The network model testing unit performs model testing to test the usage of the AI ​​chip when the compressed model is in use.

[0021] Step 5: Network model iteration: When the performance of the new network model obtained by compression is better than that of the previous network model, the new network model iteratively replaces the previous network model, and the iterative update stops after a certain number of iterations.

[0022] When the present invention is used: Figure 1 As shown, the model is compressed by the network model processing unit, and the compressed network model is passed to the fine-tuning unit. The network model is fine-tuned through the model micro-training module, and the obtained new model is passed to the conversion unit. The new model is parsed and quantized, and the data format is converted and then passed to the testing unit. The model is calculated in the AI ​​chip in the testing unit to obtain the test results, and the performance is compared with the previous model. The model with good performance is sent to the network model processing unit for a new round of processing, followed by continuous iterative replacement;

[0023] exist Figure 2 As shown, the software architecture diagram of the present invention is divided into four parts: network interaction layer, business logic layer, BSP layer, and hardware implementation layer. Network interaction is provided with various communication protocols: http, rtp, and udp; the business logic layer is divided into storage, device management, and communication; the BSP layer is divided into system call layer, operation layer, and API layer; the hardware layer has ARM, DSP, and AI chips. When working, it first calls the relevant protocols in the network interaction layer to contact the client, receives the initial network model, stores the model through the storage layer in the business logic layer, and then parses and quantizes the network model through the BSP layer. The converted model is transplanted to the AI ​​chip of the hardware layer for testing, and the obtained test data is saved. The network model is compressed and fine-tuned in the API of the BSP layer to obtain a new model, which is then parsed and quantified and passed to the AI ​​chip for testing. It is compared with the test data of the initial model. If the performance is better than before, it is replaced and updated, and then the business logic performs the next round of iterative update. Through autonomous compression and autonomous iteration, the network model is finally adapted to the specific hardware.

[0024] exist Figure 3 As shown, the model is compressed by the network model processing unit, which receives the incoming initial network model through the front-end device, and performs relevant compression processing on the incoming network model through the network model compression module, and transmits the compressed network model to the fine-tuning unit, and fine-tunes the compressed network model. The network model is fine-tuned by the model micro-training module, and the number of training rounds and learning rate parameters are set. When the model training reaches a certain threshold, the training is stopped, and the obtained new model is transmitted to the conversion unit, and the new model is parsed and quantized. The converted data format is transmitted to the test unit, and the model is calculated in the AI ​​chip in the test unit to obtain the test results. The performance of the new network model is compared with the previous model through the management module. If the performance is better than the previous network model, the previous performance data is replaced, and the result is notified to the fine-tuning unit. The replaced model is transmitted to the network model processing unit for a new round of processing, and then it is continuously iterated and replaced.

Claims

1. A structured smart camera, characterized by: It includes a main control unit, a network model processing unit, a network model fine-tuning unit, a network model conversion unit and a network model testing unit. The main control unit realizes management and control with the network model processing unit, the network model fine-tuning unit, the network model conversion unit and the network model testing unit respectively through multiple communication transmission units. The network model processing unit has a model compression module, which is connected to the front-end data receiving device through the communication transmission unit to receive the incoming network model; the network model fine-tuning unit has a model micro-training module; The network model conversion unit has a model analysis module and a quantification module, and the model is transplanted and used in a structured intelligent camera through a network model testing unit; A neural network model compression method based on a structured intelligent camera is implemented by the following steps: Step 1: Model processing: The structured smart camera obtains the initial network model from the front-end device through the network model processing unit. After the network model compression module compresses the obtained initial model, the compressed model is transferred to the model micro-training module of the network model fine-tuning unit. Step 2: Model fine-tuning: The compressed model is fine-tuned through the model micro-training module of the network model fine-tuning unit to improve the precision and accuracy of model detection and recognition. The fine-tuned model is transmitted to the network model conversion unit through the communication transmission unit. Step 3: Model quantization: The model parsing module of the network model conversion unit first parses the model to extract relevant data, which is then passed to the quantization module to quantize the model and convert it into a format suitable for the AI ​​chip in the smart camera. Step 4: Model testing: The model is converted into the AI ​​chip data format and then transplanted into the AI ​​chip. The network model testing unit performs model testing to test the usage of the AI ​​chip when the compressed model is in use. Step 5: Network model iteration: When the performance of the new network model obtained by compression is better than that of the previous network model, the new network model iteratively replaces the previous network model, and the iterative update stops after a certain number of iterations.

Images