Neural network operator optimization method, electronic equipment and medium
Through the neural network operator optimization method, including optimization steps in the model training and deployment stage, the problem of high resource consumption and poor robustness of deep learning models on edge devices is solved, and more efficient computing and lower energy consumption is achieved. , improves the accuracy and generalization capabilities of the model.
Patent Information
- Application Number
- CN202411090611.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-13
AI Technical Summary
The deployment of deep learning models on edge devices is limited, has high resource consumption, poor robustness, is susceptible to noise interference, affecting generalization capabilities.
The neural network operator optimization method is adopted, including the model training stage and deployment stage. By converting the .pt file into an onnx model, deleting slice and Concat operators, using opencv to process images, quantizing floating point numbers into integers, and performing normalization processing. Optimize model inference, use CPU for preprocessing, and reduce memory and computing resource requirements.
Improve model performance and generalization capabilities, reduce resource consumption, reduce power consumption, extend equipment battery life, and reduce energy costs.
Smart Images
Figure CN119131547A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network technology, and in particular to a neural network operator optimization method, electronic equipment and medium. Background Art
[0002] With the rapid development of artificial intelligence technology, the scale and complexity of deep learning models continue to rise, resulting in a sharp increase in the demand for computing resources and memory, which not only limits the deployment of models on edge devices, but also significantly increases operating costs and energy consumption. In addition, existing models show poor robustness when processing out-of-distribution or abnormal data and are easily affected by noise, which affects their generalization ability and practical application value. Summary of the invention
[0003] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by this patent application is how to provide a neural network operator optimization method, electronic device and medium with high accuracy, strong stability and wide application range.
[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0005] A neural network operator optimization method includes a model training phase and a model deployment phase. The model training phase includes the following steps: S1: converting a .pt file into an onnx model using yolov5; S2: modifying the onnx model, deleting slice and Concat operators, and directly passing input data to the Conv operator; S3: using opencv functions to process images; S4: quantizing the original floating point numbers into integers through model quantization; S5: normalizing the image, and finally reasoning the newly modified model.
[0006] As an optimization, in step S1, export.py is used to export the onnx file.
[0007] As an optimization, in step S3, the image preprocessing part is all performed on the CPU.
[0008] As an optimization, in step S4, the integer is uint8.
[0009] As an optimization, in step S5, floating point storage (operation) is converted to integer storage (operation) to reduce the memory and computing resources required by the model.
[0010] As an optimization, in step S5, the inferred data is further processed, and the inferred data is screened by non-maximum suppression (NMS) and threshold processing.
[0011] An electronic device, characterized in that the device contains a processor and a memory
[0012] A processor, configured to read instructions in the memory and execute the method according to any one of claims 1 to 6.
[0013] A storage medium, characterized in that a neural network model, an optimization algorithm and related data are stored thereon, and when the content thereon is called by a processor, the processor executes the method as described in any one of claims 1-6.
[0014] The present invention has the following effects:
[0015] This patent effectively improves model performance and reduces resource consumption. It improves the accuracy and generalization ability of the model. By increasing the calculation speed and reducing resource consumption, it can extend the battery life of the device and reduce energy costs by reducing power consumption. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 The present invention is a flowchart of a neural network operator optimization method. DETAILED DESCRIPTION
[0017] The present invention is further described in detail below in conjunction with the accompanying drawings. In the description of the present invention, it should be understood that the directions or positional relationships indicated by directional words such as "upper, lower" and "top, bottom" are usually based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description. Unless otherwise stated, these directional words do not indicate or imply that the devices or components referred to must have a specific direction or be constructed and operated in a specific direction, and therefore cannot be understood as limiting the scope of protection of the present invention; the directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.
[0018] like Figure 1 As shown, a neural network operator optimization method includes a model training stage and a model deployment stage. The model training stage contains the following steps: S1: convert the .pt file into an onnx model using yolov5; S2: modify the onnx model, delete the slice and Concat operators, and directly pass the input data to the Conv operator; S3: use the opencv function to process the image; S4: quantize the original floating point number into an integer through model quantization; S5: normalize the image, and finally infer the newly modified model.
[0019] In this embodiment, in step S1, export.py is used to export the onnx file.
[0020] In this embodiment, in step S3, the image preprocessing part is all performed on the CPU.
[0021] In this embodiment, in step S4, the integer is uint8.
[0022] In this embodiment, in step S5, floating point storage (operation) is converted into integer storage (operation) to reduce the memory and computing resources required for the model.
[0023] In this embodiment, in step S5, the inferred data is further processed by filtering the inferred data through non-maximum suppression (NMS) and threshold processing.
[0024] Specifically, hardware accelerators are important tools for achieving efficient neural network computing. Accelerators take advantage of dedicated hardware. For GPUs, CUDA or OpenCL is used to compile and optimize for specific hardware; for CPUs, SIMD instructions and multi-threaded parallelism are considered. Use appropriate data layout to maximize cache hit rates and reduce memory latency, ensuring that the layout of data in memory matches the cache behavior of the processor. Optimizing the hardware's memory architecture is also critical to improving the performance of neural network operators. Neural network computing involves a large amount of data access and transmission, so an efficient memory access mechanism can significantly reduce data access latency and improve computing efficiency. Using a hierarchical storage structure, making rational use of the cache mechanism, and optimizing data layout and access patterns can effectively improve the performance of neural network operators.
[0025] Specifically, operators can be fused to maximize memory access. For matrix operations, data should be stored in continuous memory blocks to facilitate cache prefetching and reading, reduce frequent memory access, and improve memory access efficiency. Reduce the number of memory allocation and release to reduce memory management overhead. Pre-allocate memory and reuse it to avoid frequent dynamic memory allocation. Use data compression or quantization technology to reduce the storage space of models and data in memory, thereby reducing the cost of memory access.
[0026] An electronic device, characterized in that the device contains a processor and a memory
[0027] A processor, configured to read instructions in the memory and execute the method according to any one of claims 1 to 6.
[0028] A storage medium, characterized in that a neural network model, an optimization algorithm and related data are stored thereon, and when the content thereon is called by a processor, the processor executes the method as described in any one of claims 1-6.
[0029] The present invention has the following effects:
[0030] This patent effectively improves model performance and reduces resource consumption. It improves the accuracy and generalization ability of the model. By increasing the calculation speed and reducing resource consumption, it can extend the battery life of the device and reduce energy costs by reducing power consumption.
[0031] Finally, it should be noted that those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A neural network operator optimization method, It is characterized in that It includes the model training phase and the model deployment phase. The model training phase contains the following steps: S1: Convert the .pt file into an onnx model using yolov5; S2: Modify the onnx model, delete the slice and Concat operators, and pass the input data directly to the Conv operator; S3: Use opencv functions to process the image; S4: Quantize the original floating point number into an integer through model quantization; S5: Normalize the image and finally perform inference on the newly modified model.
2. A neural network operator optimization method, electronic device and medium according to claim 1, It is characterized in that In step S1, export.py is used to export the onnx file.
3. A neural network operator optimization method, electronic device and medium according to claim 2, It is characterized in that In step S3, the image preprocessing part is all performed on the CPU.
4. A neural network operator optimization method, electronic device and medium according to claim 3, It is characterized in that In step S4, the integer is uint8.
5. A neural network operator optimization method, electronic device and medium according to claim 4, It is characterized in that In step S5, floating point storage (operation) is converted to integer storage (operation) to reduce the memory and computing resources required by the model.
6. A neural network operator optimization method, electronic device and medium according to claim 5, It is characterized in that In step S5, the inferred data is further processed by filtering the inferred data through non-maximum suppression (NMS) and threshold processing.
7. An electronic device, It is characterized in that The device contains a processor and a memory A processor, configured to read instructions in the memory and execute the method according to any one of claims 1 to 6.
8. A storage medium, It is characterized in that A neural network model, an optimization algorithm and related data are stored thereon, and when the content thereon is called by a processor, the processor executes a method as described in any one of claims 1-6.