A traffic sign recognition multi-task model inference optimization method based on model quantization optimization and a related device thereof

Abstract

The application discloses a traffic road sign recognition multi-task model inference optimization method based on model quantization optimization and related equipment thereof. The method comprises the following steps: deploying a simplified model obtained by performing INT8 quantization and simplification processing on a traffic road sign multi-task recognition model based on precision control at a user end, the model comprising a backbone network and a plurality of task branches; acquiring a traffic road condition image and inputting the image into the simplified model; performing feature extraction on the traffic road condition image through the backbone network to obtain a traffic road sign global feature map; for each task branch, performing a task of each task branch based on the traffic road sign global feature map, and outputting a task result of each task branch. As can be seen, by performing INT8 quantization and simplification on the traffic road sign multi-task recognition model based on precision control, the model can reduce power consumption and improve inference speed while ensuring precision in the inference process, thereby improving traffic road sign multi-task recognition efficiency.

Description

Technical Field

[0001] This application relates to the field of traffic sign recognition technology, and more specifically, to a multi-task model inference optimization method for traffic sign recognition based on model quantization optimization and related equipment. Background Technology

[0002] With the rapid development of embedded artificial intelligence and edge machine vision technologies, low-power embedded platforms, thanks to their low cost and integrated AI acceleration capabilities, are widely used in traffic sign recognition tasks in scenarios such as intelligent vehicles, robot navigation, and intelligent inspection. Currently, user-side road sign recognition needs to simultaneously complete multiple tasks such as detection, classification, localization, and distance estimation to meet the actual needs of comprehensive perception and real-time decision-making in traffic scenarios. Multi-task collaborative recognition has become the core development direction of edge-side intelligent traffic perception.

[0003] Existing edge-side road sign multi-task recognition generally adopts a multi-model independent deployment scheme. The model structure is complex and the number of parameters is large, which makes it impossible to run efficiently on embedded microcontrollers. After simplifying the model structure, the recognition accuracy of each task is greatly reduced, making it difficult to simultaneously meet the requirements of accuracy, real-time performance and low power consumption for multi-task recognition of traffic road signs.

[0004] How to balance the requirements of accuracy, real-time performance, and low power consumption in the process of multi-task recognition of traffic signs is an issue that needs attention. Summary of the Invention

[0005] In view of the above problems, this application provides a traffic sign recognition multi-task model inference optimization method and related equipment based on model quantization optimization, so as to simultaneously take into account the requirements of accuracy, real-time performance and low power consumption in the process of traffic sign multi-task recognition.

[0006] To achieve the above objectives, the following specific solutions are proposed: A traffic sign recognition multi-task model inference optimization method based on model quantization optimization is applied to the user terminal. The user terminal deploys a traffic sign recognition multi-task simplified model, which is obtained by INT8 quantization simplification processing based on precision control after pre-training the traffic sign recognition multi-task model. The traffic sign recognition multi-task simplified model includes a backbone network and multiple task branches. The method includes: Acquire traffic condition images and input the traffic condition images into the traffic sign multi-task recognition simplified model; The traffic condition image is used to extract features through the backbone network to obtain a global feature map of traffic signs. For each of the task branches, based on the global feature map of traffic signs, the task of each task branch is executed, and the task result of each task branch is output.

[0007] Optionally, the traffic sign multi-task recognition model is a floating-point model, and the user terminal has a built-in ESP32-S3 embedded microcontroller. The process of obtaining the simplified traffic sign multi-task recognition model by performing INT8 quantization simplification on the traffic sign multi-task recognition model includes: Based on the computing power parameters of the ESP32-S3 embedded microcontroller, and with the model accuracy loss being lower than the first preset accuracy loss as a constraint, some convolutional kernels and some fully connected layer neurons of the traffic sign multi-task recognition model are removed to obtain a preliminary simplified model of traffic sign multi-task recognition. The simplified model for multi-task recognition of traffic signs is quantized and calibrated using INT8 to obtain the simplified model for multi-task recognition of traffic signs.

[0008] Optionally, the preliminary simplified model for multi-task recognition of traffic signs is subjected to INT8 quantization and calibration to obtain the simplified model for multi-task recognition of traffic signs, including: By performing INT8 quantization on the model weights and activation values ​​of the preliminary simplified model for multi-task recognition of traffic signs, the model weights and activation values ​​are mapped to the interval [-128, 127], thus obtaining the quantized model for multi-task recognition of traffic signs. A road sign image calibration set is constructed, which contains multiple road sign images of different traffic scenarios. The road sign image calibration set is then input into the traffic road sign multi-task recognition quantization model, and the percentage of accuracy loss of the traffic road sign multi-task recognition quantization model compared to the traffic road sign multi-task recognition model is calculated. If the percentage of accuracy loss is not greater than the second preset accuracy loss, then the accuracy calibration of the traffic sign multi-task recognition quantization model is completed, and the traffic sign multi-task recognition quantization model is determined to be the traffic sign multi-task recognition calibration model; otherwise, the model parameters of the traffic sign multi-task recognition quantization model are adjusted until the percentage of accuracy loss is not greater than the second preset accuracy loss. Based on the model adaptation format of the ESP32-S3 embedded microcontroller, the traffic sign multi-task recognition calibration model is converted to obtain the traffic sign multi-task recognition simplified model.

[0009] Optionally, the method further includes: Based on the storage alignment order of the ESP32-S3 embedded microcontroller, the model weights of the simplified traffic sign multi-task recognition model are arranged to obtain the simplified traffic sign multi-task recognition model with weighted arrangement.

[0010] Optionally, the method further includes: The model weights of the simplified multi-task recognition model for traffic signs are stored in a fixed-point number format.

[0011] Optionally, the method further includes: The calculation operations corresponding to a model weight value of 0 in the simplified multi-task traffic sign recognition model are filtered out to obtain a simplified multi-task traffic sign recognition model after invalid calculations are removed.

[0012] Optionally, the multiple task branches include a detection branch, a classification branch, a localization branch, and a distance-angle calculation branch; For each of the task branches, based on the global feature map of traffic signs, the task of each task branch is executed, and the task result of each task branch is output, including: Based on the global feature map of the traffic sign, the detection branch detects the bounding box coordinates of the target traffic sign and outputs the traffic sign region of the traffic sign image. The category of traffic instruction information corresponding to the target road sign is output through the classification branch; An image coordinate system is constructed through the positioning branch, and the position coordinates of the target road sign are output in the image coordinate system. Based on the position coordinates and the camera parameters of the vision module, the scene position of the target road sign in the traffic scene of the traffic image is output. The distance and angle calculation branch calculates and outputs the shooting distance and shooting angle of the target road sign based on the scene location, the camera parameters, and the prior information of the target road sign's size.

[0013] A traffic sign recognition multi-task model inference optimization device based on model quantization optimization is applied to the user terminal. The user terminal is equipped with a traffic sign multi-task recognition simplified model. The traffic sign multi-task recognition simplified model is obtained by INT8 quantization simplification processing based on precision control after a pre-trained traffic sign multi-task recognition model. The traffic sign multi-task recognition simplified model includes a backbone network and multiple task branches. The device includes: A traffic condition image input unit is used to acquire traffic condition images and input the traffic condition images into the traffic sign multi-task recognition simplified model. The feature extraction unit is used to extract features from the traffic condition image through the backbone network to obtain a global feature map of traffic signs; The task result output unit is used to execute the task of each task branch based on the global feature map of traffic signs, and output the task result of each task branch.

[0014] Optionally, the traffic sign multi-task recognition model is a floating-point model, and the user terminal has a built-in ESP32-S3 embedded microcontroller. The device also includes: The model is initially simplified by removing some convolutional kernels and some fully connected layer neurons of the traffic sign multi-task recognition model based on the computing power parameters of the ESP32-S3 embedded microcontroller and with the model accuracy loss being lower than the first preset accuracy loss. The quantization and calibration unit is used to perform INT8 quantization and calibration on the preliminary simplified model of the multi-task recognition of traffic signs to obtain the simplified model of the multi-task recognition of traffic signs.

[0015] Optionally, the quantization calibration unit includes: The INT8 quantization unit is used to perform INT8 quantization on the model weights and activation values ​​of the preliminary simplified model of the traffic sign multi-task recognition, and to map the model weights and activation values ​​to the interval [-128, 127] to obtain the traffic sign multi-task recognition quantization model. The accuracy loss percentage calculation unit is used to construct a road sign image calibration set containing multiple road sign images of different traffic scenarios, and to calculate the accuracy loss percentage of the traffic road sign multi-task recognition quantization model compared to the traffic road sign multi-task recognition model by inputting the road sign image calibration set into the traffic road sign multi-task recognition quantization model. The model accuracy calibration unit is used to complete the accuracy calibration of the traffic sign multi-task recognition quantization model and determine the traffic sign multi-task recognition quantization model as a traffic sign multi-task recognition calibration model if the accuracy loss percentage is not greater than the second preset accuracy loss; otherwise, the model parameters of the traffic sign multi-task recognition quantization model are adjusted until the accuracy loss percentage is not greater than the second preset accuracy loss. The format conversion unit is used to convert the traffic sign multi-task recognition calibration model according to the model adaptation format of the ESP32-S3 embedded microcontroller to obtain the traffic sign multi-task recognition simplified model.

[0016] Optionally, the device may also include: The weighting unit is used to arrange the model weights of the traffic sign multi-task recognition simplified model according to the storage alignment order of the ESP32-S3 embedded microcontroller, so as to obtain the weighted traffic sign multi-task recognition simplified model.

[0017] Optionally, the device may also include: The fixed-point number format weight storage unit is used to store the model weights of the simplified traffic sign multi-task recognition model in fixed-point number format.

[0018] Optionally, the device may also include: An invalid computation operation filtering unit is used to filter computation operations with a model weight value of 0 in the simplified traffic sign multi-task recognition model, so as to obtain a simplified traffic sign multi-task recognition model after invalid computation is removed.

[0019] Optionally, the multiple task branches include a detection branch, a classification branch, a localization branch, and a distance-angle calculation branch; The task result output unit includes: The road sign region output unit is used to detect the bounding box coordinates of the target road sign based on the global feature map of the traffic road sign through the detection branch, and output the road sign region of the traffic road sign image; A traffic instruction information category output unit is used to output the traffic instruction information category corresponding to the target road sign through the classification branch; The scene location output unit is used to construct an image coordinate system through the positioning branch, output the location coordinates of the target road sign in the image coordinate system, and output the scene location of the target road sign in the traffic scene of the traffic condition image according to the location coordinates and the camera parameters of the vision module. The distance and angle output unit is used to calculate and output the shooting distance and shooting angle of the target road sign based on the scene position, the camera parameters and the prior information of the size of the target road sign through the distance and angle calculation branch.

[0020] A traffic sign recognition multi-task model inference optimization device based on model quantization optimization, including a memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the multi-task model inference optimization method for traffic sign recognition based on model quantization optimization as described above.

[0021] A storage medium storing a computer program, which, when executed by a processor, implements the steps of the multi-task model inference optimization method for traffic sign recognition based on model quantization optimization as described above.

[0022] Using the above technical solution, this application obtains a simplified traffic sign multi-task recognition model by deploying a traffic sign multi-task recognition model on the user end and performing INT8 quantization simplification based on precision control. This model includes a backbone network and multiple task branches. Traffic condition images are acquired and input into the simplified model. The backbone network extracts features from the traffic condition images to obtain a global feature map of traffic signs. For each task branch, based on the global feature map, the task of each branch is executed, and the task result of each branch is output. Therefore, by performing INT8 quantization simplification based on precision control on the traffic sign multi-task recognition model, the model can reduce power consumption and increase inference speed while maintaining accuracy during inference, thereby improving the efficiency of traffic sign multi-task recognition. Attached Figure Description

[0023] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating an embodiment of this application for inference optimization of a multi-task model for traffic sign recognition based on model quantization optimization; Figure 2 A flowchart illustrating the INT8 quantization and calibration of a preliminary simplified model for multi-task recognition of traffic signs, provided in an embodiment of this application; Figure 3 This is a schematic diagram of a model architecture for implementing multi-task recognition of traffic signs provided in an embodiment of this application; Figure 4 A schematic diagram of a multi-task process based on a traffic sign multi-task recognition model architecture provided in this application embodiment; Figure 5 A schematic diagram of a device structure for implementing inference optimization of a multi-task model for traffic sign recognition based on model quantization optimization, provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a device for implementing multi-task model inference optimization for traffic sign recognition based on model quantization optimization, provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] This application's solution can be implemented based on a terminal with data processing capabilities. This terminal can be a user terminal, which can have a built-in embedded microcontroller with an ESP32-S3 main control module, a dual-core Tensilica LX7 processor with a 240MHz clock speed, 512KB of built-in SRAM, and external 8MB of PSRAM and 16MB of Flash. It supports ESP-NN hardware acceleration and ESP-DSP digital signal processing. The user terminal deploys a simplified traffic sign multi-task recognition model and a vision module. The simplified traffic sign multi-task recognition model can be obtained by simplifying a pre-trained traffic sign multi-task recognition model through INT8 quantization based on accuracy control. The simplified traffic sign multi-task recognition model is burned into the ESP32-S3's Flash memory via a model file. Compared to the traffic sign multi-task recognition simplified model, the simplified model can improve inference speed and reduce power consumption by adapting to the low computing power characteristics of the ESP32-S3, while maintaining an accuracy loss of ≤5%.

[0026] The simplified multi-task traffic sign recognition model can include a backbone network and multiple task branches. The backbone network can employ a lightweight convolutional neural network (such as a lightweight improved version of MobileNetV2 or a custom lightweight CNN) to adapt to the low computational constraints of the ESP32-S3. The backbone network extracts road sign features from images acquired by the vision module and outputs a global feature map. This global feature map provides unified feature support for subsequent task branches. Multiple task branches can be connected to the output of the backbone network. Each task branch can share the global feature map output by the backbone network while performing its own task, and can refine its own task characteristics based on the global feature map.

[0027] Furthermore, to address the computational constraints of ESP32-S3, the backbone network structure can be lightweighted and improved. This includes reducing the number of convolutional layers, for example, simplifying the original 19 convolutional layers of MobileNetV2 to 12 layers and removing redundant bottleneck structures; adjusting the number of convolutional kernels, for example, reducing the number of convolutional kernels in each layer by 30%, such as adjusting the number of convolutional kernels in the first layer from 32 to 22, thus reducing the number of parameters; and using depthwise separable convolutions in all convolutional layers to further reduce the number of parameters and computational cost, keeping the model parameter size within 1.2MB to adapt to the memory constraints of ESP32-S3.

[0028] The vision module can be an OV2640 camera module with a resolution of 640×480, a frame rate of 30fps, and support for autofocus. It connects to the ESP32-S3 via an I2C interface. The vision module can be used to acquire / capture images of real-time traffic conditions. Therefore, after the vision module acquires / captures images, the images can be directly fed into the traffic sign multi-task recognition model. Simultaneously, the camera parameters can also be fed into the traffic sign multi-task recognition model for inference.

[0029] In addition, the user terminal includes auxiliary modules: a 3.7V lithium battery (2000mAh) for power supply; a buzzer and LED indicator for local alarms; and a Wi-Fi / BLE module (built into the ESP32-S3) for data upload and linkage control. The user terminal can connect to the test equipment PC (for model training and parameter debugging). The test equipment includes a road sign test set (containing 20 common road signs such as speed limits, no entry, straight ahead, and turns, totaling 5000 images, covering scenarios such as backlight, nighttime, slight obstruction, and long distance).

[0030] Next, combined Figure 1 The traffic sign recognition multi-task model inference optimization method based on model quantization optimization of this application may include the following steps: Step S110: Obtain traffic condition images and input them into the traffic sign multi-task recognition simplified model.

[0031] Specifically, a vision module deployed on the user end can acquire visible light images of road scenes in real time. Further preprocessing of the acquired raw images includes size normalization, grayscale correction, noise reduction filtering, adaptive illumination enhancement, and gamma correction, scaling the images to the resolution required by the model input. After preprocessing, the image data is converted into a tensor format acceptable to the model, removing blurry, overexposed, and underexposed frames to ensure the clarity and usability of the input image, providing a stable and high-quality image data source for subsequent feature extraction and multi-task recognition. The preprocessed traffic image is input into a simplified traffic sign multi-task recognition model. Following the simplified model's preset input specifications, the standardized traffic image is passed to the model input in tensor form, completing data format alignment, channel arrangement conversion, and numerical normalization to match the image data with the model weight parameters. Before being fed into the backbone network, image data is cached and validated to ensure no data loss and no format errors, providing compliant and reliable input data for the simplified model to stably perform feature extraction and multi-task parallel inference, ensuring the continuous and efficient operation of subsequent recognition processes.

[0032] Understandably, adaptive illumination enhancement and noise suppression reduce the interference of complex environments on recognition, improving the model's robustness in backlit, low-light, and occluded scenarios. Image normalization and resolution standardization ensure standardized model input, reducing computational anomalies caused by inconsistent sizes. Invalid frame filtering reduces unnecessary computation, saving computing power and memory overhead on the edge. High-quality image input improves the accuracy of backbone network feature extraction, providing a reliable feature foundation for subsequent multi-task branches such as detection, classification, localization, and distance estimation, thus improving overall recognition accuracy and inference stability, and ensuring real-time recognition performance on the edge.

[0033] Step S120: Extract features from traffic condition images using a backbone network to obtain a global feature map of traffic signs.

[0034] Specifically, standardized preprocessed traffic images can be input into a lightweight backbone network. Through multiple layers of depthwise separable convolution, pooling, and activation operations, edge, texture, contour, and semantic information in the image are extracted layer by layer. While filtering out redundant background information, the network focuses on key features of road sign regions, ultimately outputting a unified feature map containing both global semantics and local details.

[0035] Step S130: For each task branch, based on the global feature map of traffic landmarks, execute the task of each task branch and output the task result of each task branch.

[0036] Specifically, the unified global feature map output by the backbone network can be synchronously distributed to each task branch. Each branch performs targeted feature refinement and task inference based on the shared features, independently completing its own preset recognition and calculation tasks. Each branch can execute the inference logic in parallel or partially in parallel, making full use of the feature sharing mechanism to reduce redundant computation, achieving collaborative optimization while ensuring task independence, and finally outputting stable and reliable results for each task in parallel.

[0037] Understandably, sharing the same set of features across branches enhances consistency and synergy in the recognition results of different tasks, effectively reducing biases and errors caused by independent task optimization. Simultaneously, the single-model, multi-branch structure reduces the number of models and parameters, lowers memory and storage requirements, and is more suitable for embedded deployments. This approach balances task independence with feature uniformity, improving recognition accuracy and stability while achieving lightweight, low-power, multi-task integrated recognition.

[0038] This embodiment provides a traffic sign recognition multi-task model inference optimization method based on model quantization optimization. The simplified traffic sign recognition multi-task model is obtained by deploying a traffic sign multi-task recognition model on the user end and performing INT8 quantization simplification processing based on precision control. This model includes a backbone network and multiple task branches. Traffic condition images are acquired and input into the simplified model. The backbone network extracts features from the traffic condition images to obtain a global feature map of traffic signs. For each task branch, based on the global feature map, the task of each branch is executed, and the task result of each branch is output. Therefore, by performing INT8 quantization simplification based on precision control on the traffic sign multi-task recognition model, the model can reduce power consumption and increase inference speed while maintaining accuracy during inference, thereby improving the efficiency of traffic sign multi-task recognition.

[0039] In some embodiments of this application, the training process of the traffic sign multi-task recognition model mentioned in the foregoing embodiments is described, and the training process may include: S1. Construct a sample set of road sign images.

[0040] The road sign image sample set can contain multiple road sign images labeled with information tags.

[0041] For example, 4,000 images of common road signs can be collected and preprocessed, including image scaling, normalization, random flipping, and illumination enhancement. These images are then labeled with information, including at least the bounding box, category, actual location, and distance.

[0042] S2. Using the road sign image sample set as training samples and information labels as sample labels, the multi-branch recognition model is trained under the constraint of the joint loss function to obtain the traffic road sign multi-task recognition model.

[0043] The multi-branch recognition model can include an initialization backbone network and multiple initialization task branches connected to the output of the initialization backbone network. These initialization task branches include a detection branch, a classification branch, a localization branch, and a distance / angle calculation branch. The learning rate is 0.001, the Adam optimizer is used, and the training epochs are 100. The joint loss function is:

[0044] in, For the total loss, To optimize the accuracy of bounding box detection, the bounding box loss is used for branch detection. The cross-entropy loss of the classification branch is used to optimize the accuracy of road sign classification. The coordinate loss of the positioning branch is used to optimize the deviation of the positioning coordinates. The regression loss for the distance and angle branches is calculated to optimize the estimation accuracy of distance and angle. , , and These are the loss balance coefficients for the corresponding branch losses.

[0045] During the training of the multi-branch recognition model, a road sign image validation set (such as containing 500 common road sign images labeled with information tags) can be constructed. After each preset training round, the multi-branch recognition model can be validated using the road sign image validation set to obtain validation results, and the loss balance coefficients in the joint loss function can be adjusted based on the validation results.

[0046] For example, after every 10 training rounds, the multi-branch recognition model trained to the current stage can be validated using a road sign image validation set to obtain the model's recognition accuracy, and the loss balance coefficient can be adjusted based on the recognition accuracy. , , and .

[0047] Furthermore, a road sign image test set (500 images covering various complex scenes) can be constructed to test the performance of the traffic road sign multi-task recognition model that has been trained and exported in TFLite format. The test results are as follows: Recognition accuracy: Road sign detection accuracy is 96.2%, classification accuracy is 93.8%, average positioning deviation is 2.3 pixels, and average distance estimation error is 4.2%.

[0048] Inference performance: The inference time per frame is approximately 65ms, and the inference frame rate is approximately 15.4fps, which meets the requirements for real-time recognition.

[0049] Hardware usage: The model occupies approximately 0.8MB of Flash memory, approximately 280KB of SRAM during inference, and approximately 500KB of PSRAM. No memory overflow occurred.

[0050] Power consumption test: In battery-powered mode, the power consumption is about 80mA during inference and about 5mA during standby. The 2000mAh lithium battery can work continuously for about 25 hours, meeting the low power consumption requirements.

[0051] Scene adaptability: The recognition accuracy remains above 90% in backlight, nighttime, slightly obstructed, and long-distance (5-15 meters) scenarios, demonstrating good adaptability.

[0052] In some embodiments of this application, the process of performing INT8 quantization simplification on the traffic sign multi-task recognition model mentioned in the foregoing embodiments is described. Specifically, simplification can be performed on the data format. The traffic sign multi-task recognition model before simplification can be a floating-point model, and the user terminal has a built-in ESP32-S3 embedded microcontroller. The INT8 quantization simplification process can specifically include: S1. Based on the computing power parameters of the ESP32-S3 embedded microcontroller, and with the model accuracy loss being lower than the first preset accuracy loss as a constraint, some convolutional kernels and some fully connected layer neurons of the traffic sign multi-task recognition model are removed to obtain a preliminary simplified model of traffic sign multi-task recognition.

[0053] Specifically, based on hardware parameters such as the main frequency, SRAM capacity, PSRAM capacity, Flash storage space of the ESP32-S3 embedded microcontroller, and the computing power of the ESP-NN hardware acceleration unit of the ESP32-S3, combined with constraints on memory usage, computational overhead, and power consumption control for edge-side inference, a structured pruning and simplification of the traffic sign multi-task recognition model is performed with the strict constraint that the overall accuracy loss of the model is less than a first preset accuracy loss (≤2%). While retaining the core feature extraction links of the shared backbone network and ensuring the basic expressive capabilities of the four tasks—detection, classification, localization, and distance / angle calculation—the contribution of convolutional kernel weights and the intensity of neuron activation responses are statistically analyzed layer by layer. Redundant convolutional kernels with weight amplitudes approaching zero and low feature contributions are removed, as are redundant neurons in fully connected layers that have a weak impact on task output and bring additional computational burden. At the same time, the topological structure of the shared backbone and multi-task branches is maintained intact, without disrupting the feature sharing and task collaboration mechanisms. Finally, a preliminary simplified model for traffic sign multi-task recognition is obtained with significantly reduced parameters, greatly reduced computational load, and controllable accuracy loss.

[0054] S2. Perform INT8 quantization and calibration on the preliminary simplified model of traffic sign multi-task recognition to obtain the simplified model of traffic sign multi-task recognition.

[0055] Understandably, performing INT8 quantization and calibration on the preliminary simplified model for multi-task recognition of traffic signs can further compress the model size, reduce computational load and power consumption, and better adapt it to the low computing power hardware constraints of the ESP32-S3, while ensuring that the loss of accuracy is controllable.

[0056] Specifically, the process of performing INT8 quantization and calibration on the initial simplified model for multi-task traffic sign recognition to obtain the simplified model can be found in [reference needed]. Figure 2 Specifically, it can include: Step S210: By performing INT8 quantization on the model weights and activation values ​​of the preliminary simplified model for multi-task recognition of traffic signs, the model weights and activation values ​​are mapped to the interval [-128, 127] to obtain the quantized model for multi-task recognition of traffic signs.

[0057] Specifically, by uniformly performing INT8 symmetric quantization on the model weights and activation values ​​of the preliminary simplified model for multi-task traffic sign recognition, and based on preset quantization rules and statistically obtained dynamic scaling factors and offsets, the original floating-point type (FP32 / FP16) weight data and activation feature data generated by forward inference are linearly mapped from the floating-point numerical space to the integer range [-128, 127], completing the conversion from floating-point to fixed-point numerical format. During the quantization process, minimizing quantization error is the goal, and quantization constraints are applied to the weights and activations of the shared backbone network and each task branch, preserving the numerical distribution features that play a crucial role in feature extraction and task output, and suppressing accuracy degradation caused by minor numerical noise. After quantization, all model calculations are converted from floating-point operations to integer operations, directly matching the ESP32-S3 ESP-NN hardware acceleration unit and vector instruction set, reducing computational overhead and memory usage.

[0058] Step S220: Construct a road sign image calibration set containing multiple road sign images of different traffic scenarios, and input the road sign image calibration set into the traffic road sign multi-task recognition quantization model to calculate the percentage of accuracy loss of the traffic road sign multi-task recognition quantization model compared to the traffic road sign multi-task recognition model.

[0059] Specifically, a road sign image calibration set can be constructed covering various typical traffic scenarios, including normal lighting, backlighting, nighttime, slight occlusion, long distance, and complex roads. The calibration set should contain at least 1000 images, including all categories of road signs such as speed limits, no-entry, instructions, and warnings, ensuring that the scene and category distribution closely matches the actual deployment environment. The road sign image calibration set is then batch-input into a traffic road sign multi-task recognition quantization model that has already undergone INT8 quantization. Complete forward inference is performed, and four indicators—detection accuracy, classification accuracy, positioning deviation, and distance / angle calculation error—are simultaneously statistically analyzed. Using the inference results of the original, unquantized traffic road sign multi-task recognition model on the same calibration set as a benchmark, the differences in indicators before and after quantization are compared item by item. The overall percentage of accuracy loss of the quantized model relative to the original model across the four tasks is calculated to form a quantization error evaluation result.

[0060] Step S230: If the percentage of accuracy loss is not greater than the second preset accuracy loss, then complete the accuracy calibration of the traffic sign multi-task recognition quantization model and determine the traffic sign multi-task recognition quantization model as the traffic sign multi-task recognition calibration model; otherwise, adjust the model parameters of the traffic sign multi-task recognition quantization model until the percentage of accuracy loss is not greater than the second preset accuracy loss.

[0061] Specifically, the calculated overall accuracy loss percentage can be compared with the second preset accuracy loss (≤5%). If the overall accuracy loss percentage of the quantized model is less than or equal to this threshold, it indicates that INT8 quantization has not caused significant performance degradation to the four tasks of detection, classification, localization, and distance / angle calculation, meeting the accuracy requirements for ESP32-S3 edge deployment. The accuracy calibration process is then completed directly, and the current quantized model is identified as the traffic sign multi-task recognition calibration model. If the accuracy loss percentage exceeds the preset threshold, an iterative optimization mechanism is initiated to adjust the quantization parameters, including optimizing the scaling factors of weights and activation values, calibration intervals, and quantization points. A local floating-point retention strategy is adopted for key layers, or calibration samples are added to enhance scene coverage. Quantization and accuracy verification are re-executed, and iterative adjustments are made repeatedly until the overall accuracy loss is controlled within the allowable range, ultimately obtaining a calibration model that meets the accuracy requirements and is compatible with edge hardware.

[0062] Step S240: Based on the model adaptation format of the ESP32-S3 embedded microcontroller, the traffic sign multi-task recognition calibration model is converted to obtain a simplified traffic sign multi-task recognition model.

[0063] Specifically, considering the hardware architecture, memory addressing rules, development environment, and compatibility requirements of the ESP32-S3 embedded microcontroller's edge inference framework, and based on its specified model storage alignment and operator support specifications, a standardized format conversion is performed on the traffic sign multi-task recognition calibration model that has completed accuracy calibration. After conversion, the model is ensured to be directly loaded and parsed by the ESP32-S3's AI acceleration unit, supporting vector instruction calls and dual-core parallel inference. The result is a small, fast-loading, and stable simplified traffic sign multi-task recognition model that can be directly burned and deployed to achieve low-latency, low-power edge inference.

[0064] Based on this, the model weights of the simplified traffic sign multi-task recognition model can be arranged according to the storage alignment order of the ESP32-S3 embedded microcontroller to obtain the simplified traffic sign multi-task recognition model with weighted arrangement.

[0065] Specifically, based on the on-chip memory architecture, SRAM / PSRAM addressing rules, and byte alignment specifications of the ESP32-S3 embedded microcontroller, the INT8 weight data of the simplified multi-task recognition model for traffic signs can be ordered and normalized according to the hardware-specified data read bit width and address alignment requirements. The weight parameters of each task branch and the shared backbone network are arranged in segments according to the inference execution order, ensuring that the starting address of each weight segment meets the storage alignment constraints of the ESP32-S3, eliminating the waiting cycles caused by non-aligned accesses.

[0066] It is understandable that arranging model weights according to the ESP32-S3 storage alignment rules can significantly improve weight loading speed and inference stability. Aligned storage can avoid hardware misalignment access anomalies, reduce CPU wait cycles and data error correction overhead, and enable hardware acceleration units to read weights more efficiently, thereby improving operator execution efficiency.

[0067] Based on this, the model weights of the simplified model for multi-task recognition of traffic signs can also be stored in a fixed-point number format.

[0068] Specifically, the simplified model for multi-task recognition of traffic signs, after quantization, calibration, and format conversion, can have all model weights uniformly converted from floating-point format and stored in INT8 fixed-point format. The values ​​are strictly normalized according to the fixed-point arithmetic specifications supported by the ESP32-S3 hardware, removing the exponent and mantissa bits required for floating-point arithmetic and retaining only the valid integer bits and sign bits. This makes the weight data compatible with the fixed-point computing unit of ESP32-S3 and the ESP-NN acceleration library, realizing fixed-point formatting throughout the entire process from storage to inference, and further reducing the data bit width and storage volume.

[0069] Understandably, storing weights in fixed-point format reduces model storage footprint and read overhead, decreasing size by more than 75% compared to FP32 format. This conserves ESP32-S3's Flash and SRAM resources, while enabling integer operations throughout inference, eliminating the need for floating-point / fixed-point format conversion, reducing CPU load and power consumption, and improving operator execution speed. Fixed-point format directly matches ESP32-S3 vector instructions and hardware acceleration units, improving inference efficiency, ensuring low-latency, low-power stable operation, and adapting to edge-side embedded deployment requirements.

[0070] Based on this, the calculation operations corresponding to a model weight value of 0 in the simplified traffic sign multi-task recognition model can be filtered out to obtain a simplified traffic sign multi-task recognition model after invalid calculations are removed.

[0071] Specifically, based on the simplified traffic sign multi-task recognition model that has completed INT8 quantization and weight arrangement, all convolutional kernels and neuron weights of the backbone network and each task branch can be traversed to filter and locate the positions of parameters with a weight value of 0. The calculation operations such as convolution, multiplication and accumulation, and activation bound to such zero weights can be blocked, and invalid calculation processes that do not contribute to the actual work can be eliminated. The effective feature calculation links can be retained to form a simplified traffic sign multi-task recognition model with further reduced computation and more efficient inference process.

[0072] Furthermore, the simplified traffic sign multi-task recognition model, after undergoing simplification, can be exported in TFLite format, with a size less than 1MB, thus adapting to the ESP32-S3's Flash storage. Moreover, the simplified traffic sign multi-task recognition model can also employ piecewise loading, dividing the simplified model into four pieces, each approximately 0.2MB in size, which are loaded sequentially into SRAM. During the inference phase, the memory of the current piece is released after inference is completed, avoiding memory overflow caused by loading all at once. Simultaneously, due to the uniformity of the global feature maps, feature maps between different task branches can be reused, thereby reducing memory consumption.

[0073] In some embodiments of this application, the multiple task branches mentioned in the foregoing embodiments are further described. Specifically, the multiple task branches may include a detection branch, a classification branch, a localization branch, and a distance and angle calculation branch.

[0074] The detection branch can employ a lightweight YOLOv5-nano detection head structure, outputting the x, y, w, and h coordinates of the bounding box and its confidence score for filtering road sign target regions. The classification branch can use two fully connected layers plus a Softmax activation function to output the classification probabilities of multiple road sign classes. The input to the classification head is the road sign region features filtered by the detection branch. The localization branch can use one convolutional layer plus one fully connected layer to output the center coordinates (x0, y0) of the road sign in the image coordinate system. Combined with the intrinsic parameters of the OV2640 camera (e.g., focal length f=3.6mm, sensor size 3.2mm×2.4mm), it is converted into localization information in the real scene. The distance and angle calculation branch can calculate the distance d and relative angle θ using geometric formulas based on the center coordinates (x0, y0) of the positioning branch, the actual size of the road sign (e.g., the diameter of a speed limit sign is 30cm), and the camera parameters: d = (f×D) / (w), θ=arctan((x0 - W / 2)×s / f), where D is the actual size of the road sign, w is the width of the road sign in the image, W is the image width, and s is the sensor pixel size; the distance and angle calculation branch also uses a fully connected layer to output the predicted distance and angle values.

[0075] The structural relationship between each task branch and the backbone network can be referenced. Figure 3The backbone network output is connected to detection branches, classification branches, localization branches, and distance and angle calculation branches, enabling each task branch to share a unified global feature map of traffic signs.

[0076] Based on this, the multi-task recognition accuracy model for traffic signs mentioned in the previous embodiment is described as follows: Features are extracted from traffic images through a backbone network to obtain a global feature map of traffic signs. For each task branch, based on the global feature map of traffic signs, the task of each task branch is executed, and the task result of each task branch is output. (Refer to...) Figure 4 The process may include: Step S310: Extract features from traffic condition images using a backbone network to obtain a global feature map of traffic signs.

[0077] Specifically, step S310 is the same as step S120 in the aforementioned embodiment. For a detailed description of the process, please refer to the aforementioned embodiment, which will not be repeated here.

[0078] Step S320: Detect the bounding box coordinates of the target road sign based on the global feature map of traffic signs through the detection branch, and output the road sign region of the traffic sign image.

[0079] Specifically, the detection branch receives the global feature map of traffic signs extracted by the backbone network. It first performs target perception and region filtering on the feature map to identify potential areas in the image containing traffic signs. Then, through bounding box regression and confidence judgment, it accurately calculates the position, width, and height of each traffic sign in the image, forming standardized bounding box coordinates. After completing target detection and position calibration, it extracts valid traffic sign areas from the original traffic image based on the bounding box coordinates, filters out irrelevant background interference, and outputs the clean traffic sign area features and image information simultaneously.

[0080] Step S330: Output the traffic instruction information category corresponding to the target road sign through the classification branch.

[0081] Specifically, the classification branch can use the target road signs detected by the detection branch in step S320 as the processing object. Simultaneously, based on the global feature map extracted from the backbone network, it can perform targeted feature refinement and in-depth analysis of the road sign region, extracting distinguishable features such as road sign patterns, shapes, text, and symbols. Through classification decisions and probability calculations, the traffic instruction type to which the road sign belongs is determined, outputting confidence scores for each category. The category with the highest probability is selected as the final recognition result, forming a standardized traffic instruction information category output.

[0082] Step S340: Construct an image coordinate system through the positioning branch, output the position coordinates of the target road sign in the image coordinate system, and output the scene position of the target road sign in the traffic scene of the traffic condition image according to the position coordinates and the camera parameters of the vision module.

[0083] In this step, the localization task does not depend on any information output by the classification task in step S330. Therefore, step S340 can be executed in parallel with step S330, that is, the localization branch can be executed in parallel with the classification branch.

[0084] Specifically, an image coordinate system is constructed through the localization branch, and the position coordinates of the target road sign are output in the image coordinate system. Based on the position coordinates and the camera parameters of the vision module, the scene position of the target road sign in the traffic scene of the traffic image is output. The localization branch establishes a standard image coordinate system based on the shared global feature map and the road sign area determined by the detection branch, and accurately calculates the pixel coordinates of the road sign center and key points. Combining the camera parameters built into the vision module, such as focal length, sensor size, and pixel pitch, the image coordinates are converted into physical location information in the real traffic scene, completing the mapping from pixel-level localization to scene-level localization, and outputting stable and reliable scene position results.

[0085] Step S350: Calculate and output the shooting distance and shooting angle of the target road sign based on the scene position, camera parameters, and prior information on the size of the target road sign through the distance and angle calculation branch.

[0086] Specifically, the distance and angle calculation branch can be based on the scene position output by the positioning branch in step S340, combined with the inherent camera parameters such as the focal length, sensor size, and pixel pitch of the vision module, and introduce prior data on the standard sizes of different types of road signs in the real world. Numerical solutions are then performed using geometric projection relationships and triangulation principles. First, the shooting distance is calculated based on the ratio of the pixel width occupied by the target road sign in the image to its actual size. Then, the relative shooting angle is calculated based on the horizontal offset of the target road sign in the image and the camera's field of view. Finally, stable and high-precision distance and angle values ​​are output, providing quantitative spatial reference data for obstacle avoidance, path planning, and scene perception, thereby improving the reliability and practicality of overall traffic perception.

[0087] The apparatus for implementing multi-task model inference optimization for traffic sign recognition based on model quantization optimization, provided in the embodiments of this application, is described below. The apparatus for implementing multi-task model inference optimization for traffic sign recognition based on model quantization optimization described below can be referred to in correspondence with the method for implementing multi-task model inference optimization for traffic sign recognition based on model quantization optimization described above.

[0088] See Figure 5 , Figure 5This is a schematic diagram of a device for implementing multi-task model inference optimization for traffic sign recognition based on model quantization optimization, as disclosed in an embodiment of this application.

[0089] like Figure 5 As shown, the device may include: Traffic condition image input unit 11 is used to acquire traffic condition images and input the traffic condition images into the traffic sign multi-task recognition simplified model. Feature extraction unit 12 is used to extract features from the traffic condition image through the backbone network to obtain a global feature map of traffic signs; The task result output unit 13 is used to execute the task of each task branch based on the global feature map of traffic signs, and output the task result of each task branch.

[0090] Optionally, the traffic sign multi-task recognition model is a floating-point model, and the user terminal has a built-in ESP32-S3 embedded microcontroller. The device also includes: The model is initially simplified by removing some convolutional kernels and some fully connected layer neurons of the traffic sign multi-task recognition model based on the computing power parameters of the ESP32-S3 embedded microcontroller and with the model accuracy loss being lower than the first preset accuracy loss. The quantization and calibration unit is used to perform INT8 quantization and calibration on the preliminary simplified model of the multi-task recognition of traffic signs to obtain the simplified model of the multi-task recognition of traffic signs.

[0091] Optionally, the quantization calibration unit includes: The INT8 quantization unit is used to perform INT8 quantization on the model weights and activation values ​​of the preliminary simplified model of the traffic sign multi-task recognition, and to map the model weights and activation values ​​to the interval [-128, 127] to obtain the traffic sign multi-task recognition quantization model. The accuracy loss percentage calculation unit is used to construct a road sign image calibration set containing multiple road sign images of different traffic scenarios, and to calculate the accuracy loss percentage of the traffic road sign multi-task recognition quantization model compared to the traffic road sign multi-task recognition model by inputting the road sign image calibration set into the traffic road sign multi-task recognition quantization model. The model accuracy calibration unit is used to complete the accuracy calibration of the traffic sign multi-task recognition quantization model and determine the traffic sign multi-task recognition quantization model as a traffic sign multi-task recognition calibration model if the accuracy loss percentage is not greater than the second preset accuracy loss; otherwise, the model parameters of the traffic sign multi-task recognition quantization model are adjusted until the accuracy loss percentage is not greater than the second preset accuracy loss. The format conversion unit is used to convert the traffic sign multi-task recognition calibration model according to the model adaptation format of the ESP32-S3 embedded microcontroller to obtain the traffic sign multi-task recognition simplified model.

[0092] Optionally, the device may also include: The weighting unit is used to arrange the model weights of the traffic sign multi-task recognition simplified model according to the storage alignment order of the ESP32-S3 embedded microcontroller, so as to obtain the weighted traffic sign multi-task recognition simplified model.

[0093] Optionally, the device may also include: The fixed-point number format weight storage unit is used to store the model weights of the simplified traffic sign multi-task recognition model in fixed-point number format.

[0094] Optionally, the device may also include: An invalid computation operation filtering unit is used to filter computation operations with a model weight value of 0 in the simplified traffic sign multi-task recognition model, so as to obtain a simplified traffic sign multi-task recognition model after invalid computation is removed.

[0095] Optionally, the multiple task branches include a detection branch, a classification branch, a localization branch, and a distance-angle calculation branch; The task result output unit includes: The road sign region output unit is used to detect the bounding box coordinates of the target road sign based on the global feature map of the traffic road sign through the detection branch, and output the road sign region of the traffic road sign image; A traffic instruction information category output unit is used to output the traffic instruction information category corresponding to the target road sign through the classification branch; The scene location output unit is used to construct an image coordinate system through the positioning branch, output the location coordinates of the target road sign in the image coordinate system, and output the scene location of the target road sign in the traffic scene of the traffic condition image according to the location coordinates and the camera parameters of the vision module. The distance and angle output unit is used to calculate and output the shooting distance and shooting angle of the target road sign based on the scene position, the camera parameters and the prior information of the size of the target road sign through the distance and angle calculation branch.

[0096] The device for inference optimization of a multi-task model for traffic sign recognition based on model quantization optimization provided in this application embodiment can be applied to devices for inference optimization of a multi-task model for traffic sign recognition based on model quantization optimization, such as the user terminal side. Optionally, Figure 6 This diagram illustrates the hardware structure of a device for inference optimization of a multi-task model for traffic sign recognition based on model quantization optimization. (Refer to...) Figure 6 The hardware structure of the device for traffic sign recognition multi-task model inference optimization based on model quantization optimization may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4. In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4; Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device; The memory stores a program, which the processor can call. The program is used for: Acquire traffic condition images and input the traffic condition images into a simplified multi-task traffic sign recognition model; The traffic condition image is used to extract features through a backbone network to obtain a global feature map of traffic signs. For each task branch, based on the global feature map of traffic signs, the task of each task branch is executed, and the task result of each task branch is output.

[0097] Optionally, the refined and extended functions of the program can be found in the description above.

[0098] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor, the program being used for: Acquire traffic condition images and input the traffic condition images into a simplified multi-task traffic sign recognition model; The traffic condition image is used to extract features through a backbone network to obtain a global feature map of traffic signs. For each task branch, based on the global feature map of traffic signs, the task of each task branch is executed, and the task result of each task branch is output.

[0099] Optionally, the refined and extended functions of the program can be found in the description above.

[0100] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0101] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0102] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-task model inference optimization method for traffic sign recognition based on model quantization optimization, characterized in that, The application is applied to the user terminal, which deploys a traffic sign multi-task recognition simplified model. The traffic sign multi-task recognition simplified model is obtained by INT8 quantization simplification processing based on precision control after a pre-trained traffic sign multi-task recognition model. The traffic sign multi-task recognition simplified model includes a backbone network and multiple task branches. The method includes: Acquire traffic condition images and input the traffic condition images into the traffic sign multi-task recognition simplified model; The traffic condition image is used to extract features through the backbone network to obtain a global feature map of traffic signs. For each of the task branches, based on the global feature map of traffic signs, the task of each task branch is executed, and the task result of each task branch is output.

2. The method according to claim 1, characterized in that, The traffic sign multi-task recognition model is a floating-point model, and the user terminal has a built-in ESP32-S3 embedded microcontroller. The process of obtaining the simplified traffic sign multi-task recognition model by performing INT8 quantization simplification on the traffic sign multi-task recognition model includes: Based on the computing power parameters of the ESP32-S3 embedded microcontroller, and with the model accuracy loss being lower than the first preset accuracy loss as a constraint, some convolutional kernels and some fully connected layer neurons of the traffic sign multi-task recognition model are removed to obtain a preliminary simplified model of traffic sign multi-task recognition. The simplified model for multi-task recognition of traffic signs is quantized and calibrated using INT8 to obtain the simplified model for multi-task recognition of traffic signs.

3. The method according to claim 2, characterized in that, The preliminary simplified model for multi-task recognition of traffic signs is subjected to INT8 quantization and calibration to obtain the simplified model for multi-task recognition of traffic signs, including: By performing INT8 quantization on the model weights and activation values ​​of the preliminary simplified model for multi-task recognition of traffic signs, the model weights and activation values ​​are mapped to the interval [-128, 127], thus obtaining the quantized model for multi-task recognition of traffic signs. A road sign image calibration set is constructed, which contains multiple road sign images of different traffic scenarios. The road sign image calibration set is then input into the traffic road sign multi-task recognition quantization model, and the percentage of accuracy loss of the traffic road sign multi-task recognition quantization model compared to the traffic road sign multi-task recognition model is calculated. If the percentage of accuracy loss is not greater than the second preset accuracy loss, then the accuracy calibration of the traffic sign multi-task recognition quantization model is completed, and the traffic sign multi-task recognition quantization model is determined to be the traffic sign multi-task recognition calibration model; otherwise, the model parameters of the traffic sign multi-task recognition quantization model are adjusted until the percentage of accuracy loss is not greater than the second preset accuracy loss. Based on the model adaptation format of the ESP32-S3 embedded microcontroller, the traffic sign multi-task recognition calibration model is converted to obtain the traffic sign multi-task recognition simplified model.

4. The method according to claim 2, characterized in that, The method also includes: Based on the storage alignment order of the ESP32-S3 embedded microcontroller, the model weights of the simplified traffic sign multi-task recognition model are arranged to obtain the simplified traffic sign multi-task recognition model with weighted arrangement.

5. The method according to claim 2, characterized in that, The method also includes: The model weights of the simplified multi-task recognition model for traffic signs are stored in a fixed-point number format.

6. The method according to claim 2, characterized in that, The method also includes: The calculation operations corresponding to a model weight value of 0 in the simplified multi-task traffic sign recognition model are filtered out to obtain a simplified multi-task traffic sign recognition model after invalid calculations are removed.

7. The method according to any one of claims 1-6, characterized in that, The multiple task branches include a detection branch, a classification branch, a localization branch, and a distance / angle calculation branch; For each of the task branches, based on the global feature map of traffic signs, the task of each task branch is executed, and the task result of each task branch is output, including: Based on the global feature map of the traffic sign, the detection branch detects the bounding box coordinates of the target traffic sign and outputs the traffic sign region of the traffic sign image. The category of traffic instruction information corresponding to the target road sign is output through the classification branch; An image coordinate system is constructed through the positioning branch, and the position coordinates of the target road sign are output in the image coordinate system. Based on the position coordinates and the camera parameters of the vision module, the scene position of the target road sign in the traffic scene of the traffic image is output. The distance and angle calculation branch calculates and outputs the shooting distance and shooting angle of the target road sign based on the scene location, the camera parameters, and the prior information of the target road sign's size.

8. A traffic sign recognition multi-task model inference optimization device based on model quantization optimization, characterized in that, The application is applied to the user terminal, which deploys a traffic sign multi-task recognition simplified model. The traffic sign multi-task recognition simplified model is obtained by INT8 quantization simplification processing based on precision control after a pre-trained traffic sign multi-task recognition model. The traffic sign multi-task recognition simplified model includes a backbone network and multiple task branches. The device includes: A traffic condition image input unit is used to acquire traffic condition images and input the traffic condition images into the traffic sign multi-task recognition simplified model. The feature extraction unit is used to extract features from the traffic condition image through the backbone network to obtain a global feature map of traffic signs; The task result output unit is used to execute the task of each task branch based on the global feature map of traffic signs, and output the task result of each task branch.

9. A traffic sign recognition multi-task model inference optimization device based on model quantization optimization, characterized in that, Including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the multi-task model inference optimization method for traffic sign recognition based on model quantization optimization as described in any one of claims 1-7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the multi-task model inference optimization method for traffic sign recognition based on model quantization optimization as described in any one of claims 1-7.

Images