Traffic light detection engineering deployment method based on ResNet network
Through lightweight compression and optimization technology based on ResNet network, the problem of deployment of traffic light detection models on mobile devices and balancing accuracy and speed is solved, and efficient traffic light detection on edge devices is realized, improving detection accuracy and speed.
Patent Information
- Application Number
- CN202411090613.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-13
AI Technical Summary
The existing traffic light detection model has too many parameters, slow inference speed, difficult to deploy on mobile devices, and difficult to achieve balance in detection accuracy and speed, and is easily affected by misdetecting interferences around traffic lights.
Adopt a lightweight compression method based on ResNet network, including model quantization, format conversion, deployment optimization and performance tuning, through linear quantization, operator scheduling optimization, weight data reordering, memory optimization, and using AscendCL API library and CPU parallel computing technology , deployed on edge devices.
It realizes efficient traffic light detection on the edge computing platform, with accurate detection results, with Ap value reaching 35%, the preprocessing time is reduced to 1ms, and the postprocessing time is reduced to 0.7ms, improving detection efficiency and accuracy.
Smart Images
Figure CN119131746A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of traffic light detection and recognition, and in particular to an engineering deployment method for traffic light detection based on a ResNet network. Background Art
[0002] The detection and recognition of traffic lights is of great significance to unmanned driving and assisted driving technologies, and can provide strong protection for road traffic safety. By accurately positioning and identifying the types of traffic lights, it can provide traffic light information to moving vehicles, ensure vehicle driving safety, and reduce the occurrence of traffic accidents.
[0003] In recent years, object detection technology using deep learning has been widely used. Object detection algorithms are mainly divided into two categories: two-stage object detection algorithms and one-stage object detection algorithms. The former first generates candidate object boxes (Region of Interest, ROI) through the region proposal stage, and then classifies and accurately locates these candidate boxes. The latter directly performs dense detection predictions on the image without explicit region extraction steps, and predicts the location and category of the target by applying dense anchors (Anchors) or default boxes (Default Boxes) at different positions and scales.
[0004] Although the performance of traffic light detection has been improved, there are still some problems. First, too many parameters in the model will reduce the inference speed of the model and make it difficult to port it to mobile devices. Second, there are many interference objects around traffic lights, and the model is prone to false detection. Finally, it is difficult for the model to strike a balance between detection accuracy and speed. Summary of the invention
[0005] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by this patent application is how to provide a traffic light detection engineering deployment method based on the ResNet network with lightweight model compression and less time consumption.
[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0007] A traffic light detection engineering deployment method based on a ResNet network includes the following steps: S1: model quantization: low-bit processing of the model's weights and data; S2: format conversion: converting the network model of the open source framework into an offline model supported by the edge device hardware platform; S3: model deployment: deploying the neural network model on the edge device; S4: performance tuning: optimizing and adjusting the deployed model.
[0008] As an optimization, in step S1, the model quantization adopts linear quantization, and its expression is:
[0009]
[0010] Where scale represents the quantization factor, offset represents the offset, and the supported value range is:
[0011]
[0012] offset∈[-128, 127].
[0013] As an optimization, in step S1: whether retraining is needed during the quantization process, the weights in the trained model are quantized from floating point numbers to low-bit integers using post-training quantization, and the model is calibrated and quantized based on the inference process using calibration data, thereby reducing the accuracy loss during the quantization process.
[0014] As an optimization, in step S2: model conversion ensures that it can run effectively on edge devices, and the process will perform operations including but not limited to operator scheduling optimization, weight data rearrangement, and memory usage optimization.
[0015] As an optimization, in step S3, deploying the neural network model on the edge device needs to consider multiple factors, including but not limited to model size, storage occupancy, inference latency, and power consumption. The API library AscendCL is used as the inference framework of the traffic light detection model, and it is run and processed on the Huawei Ascend AI processor.
[0016] As an optimization, in step S4, after the corresponding model is deployed on the board, the software running time needs to meet the requirements, and the time mainly includes the time consumed in pre-processing, reasoning and post-processing.
[0017] As an optimization, in step S4, the pre- and post-processing programs are run on the CPU, and the CPU parallel computing technology is used for optimization; the reasoning is run on the neural network processor, and is achieved by replacing and fusing the time-consuming operators in the model.
[0018] The present invention has the following effects:
[0019] After the above optimization steps, this patent was deployed on the edge computing platform, and the visualization results of the model's detection effect were improved. Traffic lights can be accurately identified in the detection results, and the Ap value in the test set is 35%. In terms of time consumption, after using neon optimization in pre-processing, the time consumption of standardized operations can be reduced to 1ms, and the time consumption of post-processing can be reduced to 0.7ms. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 This is a method flow chart of a method for engineering deployment of traffic light detection based on ResNet network described in the present invention.
[0021] Figure 2 This is a comparison chart of the model before and after quantization, taking quantization to INT8 data type as an example in step S1.
[0022] Figure 3 The following is a flowchart for model conversion.
[0023] Figure 4 This is the logic block diagram of AscendCL. DETAILED DESCRIPTION
[0024] The present invention is further described in detail below in conjunction with the accompanying drawings. In the description of the present invention, it should be understood that the directions or positional relationships indicated by directional words such as "upper, lower" and "top, bottom" are usually based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description. Unless otherwise stated, these directional words do not indicate or imply that the devices or components referred to must have a specific direction or be constructed and operated in a specific direction, and therefore cannot be understood as limiting the scope of protection of the present invention; the directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.
[0025] like Figure 1-4 As shown, a traffic light detection engineering deployment method based on the ResNet network includes the following steps: S1: model quantization: low-bit processing of the model's weights and data; S2: format conversion: converting the network model of the open source framework into an offline model supported by the edge device hardware platform; S3: model deployment: deploying the neural network model on the edge device; S4: performance tuning: optimizing and adjusting the deployed model.
[0026] In this embodiment, in step S1, the model quantization adopts a linear quantization method, and its expression is:
[0027]
[0028] Where scale represents the quantization factor, offset represents the offset, and the supported value range is:
[0029]
[0030] offset∈[-128, 127].
[0031] In this embodiment, in step S1: whether retraining is needed in the quantization process, the weights in the trained model are quantized from floating point numbers to low-bit integers by using post-training quantization, and the model is calibrated and quantized based on the inference process through calibration data, thereby reducing the accuracy loss in the quantization process.
[0032] In this embodiment, in step S2: model conversion ensures that it can run effectively on the edge device, and operations including but not limited to operator scheduling optimization, weight data rearrangement, and memory usage optimization will be performed during the process.
[0033] Specifically, during the conversion, the tensor compiler ATC is used to convert the model, and the conversion process includes the conversion of AI CPU operators and AI GPU operators.
[0034] In this embodiment, in step S3, deploying the neural network model on the edge device needs to consider multiple factors, including but not limited to model size, storage occupancy, inference delay and power consumption. The API library AscendCL is used as the inference framework of the traffic light detection model, and it is run and processed on the Huawei Ascend AI processor.
[0035] Specifically, the advantage of AscendCL is that it uses only one API for operator compilation, loading, and execution. Compared with one API for each operator, AscendCL significantly reduces the number of APIs. At the same time, the AscendCL interface can unify application codes and is not aware of the chip model.
[0036] In this embodiment, in step S4, after the corresponding model is deployed on the board, the running time of the software needs to meet the requirements, and the time mainly includes the time consumed in pre-processing, reasoning and post-processing.
[0037] In this embodiment, in step S4, the pre- and post-processing are programs running on the CPU, and CPU parallel computing technology is used for optimization; reasoning is running on the neural network processor, and is achieved by replacing and fusing the time-consuming operators in the model.
[0038] Specifically, the CPU parallel computing technology may adopt the NEON instruction set, or adopt multi-threading technology.
[0039] Beneficial Effects
[0040] The present invention has the following effects:
[0041] After the above optimization steps, this patent was deployed on the edge computing platform, and the visualization results of the model's detection effect were improved. Traffic lights can be accurately identified in the detection results, and the Ap value in the test set is 35%. In terms of time consumption, after using neon optimization in pre-processing, the time consumption of standardized operations can be reduced to 1ms, and the time consumption of post-processing can be reduced to 0.7ms.
[0042] The comparison data before and after optimization is shown in the following table:
[0043] time consuming Before optimization After optimization Pre-processing (standardization only) 5ms 1.2ms Reasoning time 24ms 27ms Post-processing 15ms 0.7ms
[0044] As can be seen from the table, the pre-processing normalization phase time is reduced from 5ms to 1.2ms, the post-processing phase is reduced from 15ms to 0.7ms, and the inference phase time is increased by 3ms. This is because the topK operator is added to the model inference to reduce the amount of model output data. Although the inference phase time is increased, the overall time is reduced from 44ms to 29ms, which speeds up the efficiency of image detection.
[0045] Finally, it should be noted that those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A traffic light detection engineering deployment method based on ResNet network, characterized in that: The following steps are involved: S1: Model quantization: low-bit processing of model weights and data; S2: Format conversion: convert the network model of the open source framework into an offline model supported by the edge device hardware platform; S3: Model deployment: Deploy the neural network model on edge devices; S4: Performance tuning: Optimize and adjust the deployed model.
2. According to claim 1, a traffic light detection engineering deployment method based on ResNet network is characterized in that: In step S1, the model quantization adopts linear quantization, and its expression is: Where scale represents the quantization factor, offset represents the offset, and the supported value range is: offset∈[-128, 127].
3. According to claim 2, a traffic light detection engineering deployment method based on ResNet network is characterized in that: In step S1: whether retraining is needed during the quantization process, the weights in the trained model are quantized from floating point numbers to low-bit integers using post-training quantization, and the model is calibrated and quantized based on the inference process using calibration data, thereby reducing the accuracy loss during the quantization process.
4. According to claim 3, a traffic light detection engineering deployment method based on ResNet network is characterized in that: In step S2: model conversion ensures that it can run effectively on edge devices. During the process, operations including but not limited to operator scheduling optimization, weight data rearrangement, and memory usage optimization will be performed.
5. According to claim 4, a traffic light detection engineering deployment method based on ResNet network is characterized in that: In step S3, deploying the neural network model on the edge device requires considering multiple factors, including but not limited to model size, storage usage, inference latency, and power consumption. The API library AscendCL is used as the inference framework of the traffic light detection model, and it is run and processed on the Huawei Ascend AI processor.
6. According to claim 5, a traffic light detection engineering deployment method based on ResNet network is characterized in that: In step S4, after the corresponding model is deployed on the board, the software running time needs to meet the requirements, and the time mainly includes pre-processing, reasoning and post-processing time.
7. The engineering deployment method for traffic light detection based on ResNet network according to claim 6 is characterized in that: In step S4, the pre- and post-processing are programs running on the CPU, and the CPU parallel computing technology is used for optimization; the reasoning is running on the neural network processor, and is achieved by replacing and fusing the time-consuming operators in the model.