An operator sensitivity driven mot mixed precision quantization method
By employing an operator-sensitivity-driven hybrid precision quantization method, this approach identifies and improves the precision of key operators for multi-target tracking tasks, resolving the cross-frame identity consistency issue caused by quantization and enabling efficient and reliable deployment at the edge.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF ELECTRONICS SCI & TECH OF CHINA
- Filing Date
- 2026-03-10
- Publication Date
- 2026-06-05
AI Technical Summary
In existing technologies for multi-target tracking tasks, numerical perturbations caused by quantization lead to shifts in the position, confidence, or feature vector direction of the detection box, affecting cross-frame identity consistency and state estimation stability. In particular, the probability of incorrect matching increases in complex scenarios, resulting in frequent breaks in the tracking trajectory and ID jumps, making it difficult to achieve real-time inference on resource-constrained edge devices.
By constructing a quantized baseline and a set of observation points, a set of sensitive candidate operators is generated, and the minimum set principle is used for optimization to improve the accuracy of only the key operators. Cosine similarity is used as the core criterion to generate a mixed accuracy configuration, thus avoiding a full increase in model size and energy consumption.
It significantly restores over 90% of the IDF1 score, reduces IDSwitch, and maintains tracking stability close to INT16, making it suitable for resource-constrained edge scenarios and reducing the cost and cycle of quantitative deployment.
Smart Images
Figure CN122154751A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of edge multi-target tracking model deployment technology, specifically involving an operator-sensitivity driven MOT hybrid precision quantization method. Background Technology
[0002] Deploying deep learning models at the edge typically employs whole-model INT8 quantization to reduce model size and improve inference speed. However, this uniform quantization strategy faces unique challenges in multi-object tracking (MOT) tasks. MOT systems rely not only on single-frame detection accuracy but also on cross-frame identity consistency and state estimation stability. Numerical perturbations caused by quantization can lead to shifts in bounding box position, confidence, or feature vector direction. These shifts are amplified frame-by-frame through data association (such as the Hungarian algorithm and IoU matching) and Kalman filter prediction, manifesting as increased ID switching and decreased IDF1 (IDF1-Score), thus affecting the reliability of decisions and control based on tracking results. Especially in complex scenarios such as occlusion, rapid movement, and multi-object interaction, the perturbation of the association cost matrix by quantization errors can cause a sharp increase in the probability of incorrect matches, resulting in frequent track breakage or ID jumps, severely impacting the reliability of downstream applications (such as vehicle interception and behavior analysis).
[0003] Existing mixed precision quantization methods often rely on experience or use a single error metric (such as MSE / MAE) to evaluate sensitive layers, making it difficult to characterize the nonlinear impact of "feature orientation perturbation" on association costs in MOT. Furthermore, they tend to introduce too many high-precision nodes, leading to increased model size and energy consumption, making it difficult to achieve real-time inference on resource-constrained edge devices. Summary of the Invention
[0004] To address the above problems, this invention proposes an operator-sensitivity-driven MOT hybrid precision quantization method.
[0005] The technical solution of this invention is: an operator-sensitivity-driven MOT hybrid precision quantization method comprising the following steps:
[0006] S1. Construct a quantized baseline and set of observation points for the MOT model to be deployed, and generate a set of sensitive candidate operators;
[0007] S2. Optimize the sensitive candidate operator set using the minimum set principle to obtain the minimum subset.
[0008] Furthermore, S1 includes the following sub-steps:
[0009] S11. Construct a quantization baseline for the MOT model to be deployed and generate a set of observation points;
[0010] S12. Determine the output tensor or statistic for each observation point in the observation point set;
[0011] S13. Calculate the quantization error index for each observation point based on the output tensor or statistics of each observation point.
[0012] S14. Construct waveform curves based on the quantization error index of each observation point;
[0013] S15. Generate a set of sensitive candidate operators based on the waveform curve.
[0014] Furthermore, in S11, the set of observation points includes key convolutional or residual units of the detection backbone network, upsampling and fusion nodes of the feature pyramid, classification and regression output nodes of the detection head, and ReID feature extraction branches.
[0015] Furthermore, in S14, the quantification error metrics include cosine similarity, mean absolute error, and relative Euclidean distance.
[0016] Furthermore, in S14, the cosine similarity of each observation point is arranged according to the operator index to form a sequence, and the sequence is subjected to segmented probability statistics, kernel density estimation or histogram densityization to obtain the waveform curve.
[0017] Furthermore, in S15, observation points in the waveform curve that exhibit waveform dips, frequent crossovers, enhanced local oscillations, or the formation of anomalous clusters are selected as a set of sensitive candidate operators.
[0018] The waveform concavity is specifically as follows: ,in, This represents the mean cosine similarity of all observation points. The standard deviation of the cosine similarity of all observation points. Represents the cosine similarity value of the observation points;
[0019] Frequent crossovers specifically refer to: comparing the waveform curve with the baseline waveform point by point using a sliding window of length W, with the sign of the difference between the waveform curve and the baseline waveform changing no less than 2 times within the sliding window;
[0020] The enhancement of local oscillations specifically refers to the standard deviation of the cosine similarity within a local window formed by several consecutive observation points satisfying the following condition: ,in, The standard deviation of cosine similarity within a local window. Represents a constant;
[0021] Anomaly clusters are defined as follows: after kernel density estimation of a cosine similarity sequence, a secondary peak independent of the main distribution appears, and the probability density quality covered by this secondary peak accounts for a proportion of the total not less than a set threshold.
[0022] Furthermore, in S2, a greedy strategy, a budget constraint strategy, or a multi-objective scoring strategy is used to optimize the set of sensitive candidate operators to obtain the minimum subset.
[0023] Furthermore, the greedy strategy is as follows: construct a sensitivity scoring function for each candidate node in the sensitive candidate operator set, sort the candidate nodes according to the scores, and generate a mixed precision set using the minimum set principle.
[0024] Furthermore, in the sensitivity scoring function, the cosine similarity deviation is taken as the main term, the mean absolute error and relative Euclidean distance are taken as auxiliary terms, and the number of parameters, computational cost, or bandwidth usage of candidate nodes are taken as cost terms. Its expression is:
[0025] ;
[0026] in, The weights representing the cosine similarity deviation, The weights representing the mean absolute error. The weights represent the relative Euclidean distances. Indicates the first Cosine similarity deviation of each node Indicates the first The mean absolute error of each node Indicates the first The relative Euclidean distance between the nodes This represents a cost item.
[0027] The beneficial effects of this invention are:
[0028] (1) This invention focuses on the accurate location and repair of MOT association stability. It uses cosine similarity to characterize the impact of "feature direction perturbation" on the association cost matrix, which is more in line with the ID consistency loss mechanism in MOT scenarios than using only MAE / MSE magnitude error. Through sensitive node location and minimum set principle, it can recover more than 90% of IDF1 and significantly reduce IDSwitch by improving the accuracy of only a few key operators, thus avoiding model expansion and inference efficiency reduction caused by full INT16.
[0029] (2) The present invention is resource-constrained and engineering-reusable: it achieves tracking stability close to INT16 under the premise of near INT8 model size and inference latency, which is particularly suitable for edge resource-constrained scenarios; the output operator-level precision mapping configuration can be directly used by mainstream inference engines, supports automated tuning and version iteration, and significantly reduces the manual cost and cycle of quantitative deployment. Attached Figure Description
[0030] Figure 1 A flowchart of the operator-sensitivity-driven MOT hybrid precision quantization method;
[0031] Figure 2 A schematic diagram of an operator-sensitivity-driven hybrid precision quantization method;
[0032] Figure 3 This is a waveform analysis diagram of the cosine similarity probability.
[0033] Figure 4 A ranking chart of quantization error indices and sensitivity of key operator nodes;
[0034] Figure 5 A performance comparison chart of different quantization schemes. Detailed Implementation
[0035] The embodiments of the present invention will be further described below with reference to the accompanying drawings.
[0036] like Figure 1 As shown, this invention provides an operator-sensitivity-driven MOT hybrid precision quantization method, comprising the following steps:
[0037] S1. Construct a quantized baseline and set of observation points for the MOT model to be deployed, and generate a set of sensitive candidate operators;
[0038] S2. Optimize the sensitive candidate operator set using the minimum set principle to obtain the minimum subset.
[0039] This invention addresses the issue that quantization errors in MOT tasks are amplified through the association cost matrix, state prediction, and matching processes, leading to a decline in ID consistency. It uses the cosine similarity of key tensors before and after quantization as the primary criterion to automatically or semi-automatically locate "quantization-sensitive operator nodes." Subsequently, based on the "minimum set principle," only the sensitive operator nodes are precision-enhanced, forming a hybrid precision configuration. Regression validation is performed using metrics such as IDF1 / MOTA / IDs and model size to output the optimal hybrid precision quantization scheme and operator precision list. This significantly improves ID consistency-related performance while maintaining a near-INT8 model size and inference efficiency.
[0040] This invention aims to transform quantization optimization from "trial and error based on experience" into "an interpretable, reproducible, and automated process" by using an operator-sensitive localization method with cosine similarity as the core criterion and combining it with the minimum set principle to generate mixed-precision quantization configurations. This achieves a better trade-off between model size, inference latency, and tracking accuracy (especially ID consistency), providing efficient and reliable quantization toolchain support for edge MOT deployment.
[0041] In this embodiment of the invention, S1 includes the following sub-steps:
[0042] S11. Construct a quantization baseline for the MOT model to be deployed and generate a set of observation points;
[0043] S12. Determine the output tensor or statistic for each observation point in the observation point set;
[0044] S13. Calculate the quantization error index for each observation point based on the output tensor or statistics of each observation point.
[0045] S14. Construct waveform curves based on the quantization error index of each observation point;
[0046] S15. Generate a set of sensitive candidate operators based on the waveform curve.
[0047] This invention provides an operator sensitivity localization method based on cosine similarity probability waveform analysis. For example... Figure 2 As shown, for the MOT model to be deployed, a full INT8 quantization baseline is first established, and observation points are set for predetermined operator nodes in the compilation or inference chain. The corresponding output tensors or statistics before and after quantization under the same input are then exported. For each observation point, a quantization error index is calculated, including at least the cosine similarity. Mean absolute error Relative Euclidean distance This forms an error sequence arranged according to operator nodes. Further... The results are subjected to probability statistics, segmented aggregation, or probability density processing to construct a "probability waveform" of cosine similarity. Based on the waveform's crossover frequency, oscillation amplitude, and abnormal cluster distribution, the set of sensitive candidate operators is located and quantified. .
[0048] In this embodiment of the invention, in S11, the set of observation points includes key convolutional or residual units of the detection backbone network, upsampling and fusion nodes of the feature pyramid, classification and regression output nodes of the detection head, and ReID feature extraction branches.
[0049] In this embodiment of the invention, in S14, the quantization error index includes cosine similarity, mean absolute error, and relative Euclidean distance.
[0050] In this embodiment of the invention, in S14, the cosine similarity of each observation point is arranged according to the operator index to form a sequence, and the sequence is subjected to segmented probability statistics, kernel density estimation or histogram densityization to obtain a waveform curve.
[0051] In this embodiment of the invention, in S15, observation points in the waveform curve that show waveform dips, frequent crossovers, enhanced local oscillations, or the formation of abnormal clusters are taken as a set of sensitive candidate operators.
[0052] The waveform concavity is specifically as follows: ,in, This represents the mean cosine similarity of all observation points. The standard deviation of the cosine similarity of all observation points. Represents the cosine similarity value of the observation points;
[0053] Frequent crossovers specifically refer to: comparing the waveform curve with the baseline waveform point by point using a sliding window of length W, with the sign of the difference between the waveform curve and the baseline waveform changing no less than 2 times within the sliding window;
[0054] The enhancement of local oscillations specifically refers to the standard deviation of the cosine similarity within a local window formed by several consecutive observation points satisfying the following condition: ,in, The standard deviation of cosine similarity within a local window. Represents a constant;
[0055] The anomalous cluster is defined as follows: after kernel density estimation of the cosine similarity sequence, a secondary peak independent of the main distribution appears, and the probability density quality covered by the secondary peak accounts for a proportion of the total not less than a set threshold, with the set threshold ranging from 5% to β to 20%.
[0056] In this embodiment of the invention, in S2, a greedy strategy, a budget constraint strategy, or a multi-objective scoring strategy is used to optimize the set of sensitive candidate operators to obtain the minimum subset.
[0057] This invention provides a method / scheme for hybrid precision configuration generation and regression verification closed-loop optimization based on the minimum set principle. For example... Figure 2 As shown, in the candidate set Based on this, the minimum subset is selected using the minimum set principle. Only for Internal operator nodes improve computational precision, while other nodes retain INT8, to generate a mixed-precision configuration. (Select) The strategies can be: a greedy strategy (adding parameters gradually from high to low sensitivity and performing regression verification, stopping once the objective is met), a budget-constrained strategy (maximizing ID consistency improvement within the model increment / latency increment budget), and a multi-objective scoring strategy (combining sensitivity scores with cost priorities such as operator parameter count, computational cost, and bandwidth usage). Regression verification is then performed on the standard MOT dataset or target scenario data. Evaluation metrics should include at least MOT task metrics such as IDF1, MOTA, and IDs, as well as resource overhead metrics such as model size, inference latency, and power consumption. If the preset objective is not met, adjustments are made. The process iterates and optimizes until both accuracy and resource constraints are met. The final output can be directly used as an operator-level precision mapping configuration file, a sensitive node analysis report, and a mixed precision strategy list for the inference engine / compiler, enabling engineering implementation.
[0058] In this embodiment of the invention, the greedy strategy is as follows: construct a sensitivity scoring function for each candidate node of the sensitive candidate operator set, sort the candidate nodes according to the scores, and generate a mixed precision set using the minimum set principle.
[0059] Under the premise of meeting the above accuracy targets and resource constraints. The number of nodes included should be as small as possible, that is, for To perform reduction verification, try removing the lowest-ranked node one by one. If the constraint is still satisfied after removal, the removal is confirmed. Continue until no more nodes can be removed, and finally obtain the minimum subset. .
[0060] In this embodiment of the invention, the sensitivity scoring function uses cosine similarity deviation as the main term, mean absolute error and relative Euclidean distance as auxiliary terms, and the number of parameters, computational cost, or bandwidth usage of candidate nodes as cost terms. Its expression is as follows:
[0061] ;
[0062] in, The weights representing the cosine similarity deviation, The weights representing the mean absolute error. The weights represent the relative Euclidean distances. Indicates the first Cosine similarity deviation of each node Indicates the first The mean absolute error of each node Indicates the first The relative Euclidean distance between the nodes This represents a cost item.
[0063] For the first The cosine similarity deviation of each node is used as the main scoring item to reflect the feature direction perturbation caused by quantization. For the first The average absolute error of each node is used as an auxiliary term; For the first The relative Euclidean distances between the nodes are used as an auxiliary term; For cost items, the parameter values of candidate nodes are taken. Calculation workload (FLOPs) or bandwidth usage One or a weighted combination thereof, normalized to the [0,1] interval before use; Let be the weighting coefficients for each error term, and , ; This is a cost penalty index used to control the inhibitory effect of cost items on the score.
[0064] Finally, press Candidate nodes are sorted from largest to smallest, and nodes with high scores (high benefit, low cost) are prioritized to be upgraded to high precision (INT16), gradually building a mixed precision set.
[0065] In a MOT instance, a quantization baseline is first established for the MOT model to be deployed, and a set of observation points covering key sub-modules is set. The observation points should at least cover: key convolutional / residual units of the detection backbone network, upsampling and fusion nodes of the feature pyramid, classification and regression output nodes of the detection head, and optional ReID feature extraction branches. On the same input sequence or the same batch of samples, both the high-precision version and the INT8 quantization version are run, and the output tensors or statistics of each observation point are collected to provide comparable data for subsequent error analysis.
[0066] Subsequently, error indices such as cosine similarity, mean absolute error, and relative Euclidean distance are calculated for each observation point, and a "probabilistic waveform" representation is constructed with cosine similarity as the core. Specifically, the cosine similarity values of each observation point can be arranged into a sequence according to the operator index, and the sequence is subjected to piecewise probability statistics, kernel density estimation, or histogram densityization to obtain the waveform curve; then, the high-precision baseline is compared and analyzed with the waveform of the INT8 version. Figure 3 As shown, when certain operator nodes exhibit obvious waveform dips, frequent crossovers, enhanced local oscillations, or the formation of anomalous clusters under INT8, they can be identified as quantization-sensitive candidate nodes. This embodiment embodies the technical innovation of the present invention: by directly linking feature direction perturbations with MOT association stability through "cosine similarity probability waveforms," the location of sensitive nodes is more closely aligned with the ID consistency degradation mechanism, rather than relying solely on amplitude errors.
[0067] Based on the candidate sensitive node set, multiple indicators are further integrated to form a sensitivity ranking and interpretable selection criteria. For example... Figure 4 As shown, a sensitivity scoring function can be constructed for each candidate node, for example, by... Deviation as the main term, combined with and As an auxiliary factor, the number of parameters, computational cost, or bandwidth usage of the operator node is introduced as a cost item to obtain a unified "benefit-cost" metric. After sorting the candidate nodes according to this score, a mixed-precision set is generated using the minimum set principle. Prioritize upgrading nodes with high ranking and "high benefit at low cost" to high accuracy, and perform regression verification after each adjustment to confirm whether the benefit is significant; stop scaling when the preset tracking stability target is reached and resource consumption does not exceed the budget. This process achieves the goal of "significant ID consistency recovery with a small improvement in the accuracy of key nodes", avoiding unacceptable model size, latency, or power consumption caused by a full improvement in accuracy.
[0068] Finally, the overall effects of different quantification strategies are compared, and deployable configurations are output. For example... Figure 5 As shown, the full INT8, full INT16, and mixed-precision schemes generated by this invention can be compared. Evaluation dimensions include: ID consistency metrics (such as IDF1, IDs), overall tracking metrics (such as MOTA), and resource metrics (such as model size, inference latency, and power consumption). After meeting the target constraints, a configuration file containing metadata such as model identification information, operator node precision mapping, and effective range is output, and a sensitive node waveform analysis report can be attached. This allows the compiler or inference engine to directly load and generate the corresponding mixed-precision execution graph, enabling rapid deployment without modifying the model structure and training process.
[0069] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.
Claims
1. An operator-sensitivity-driven hybrid precision quantization method for MOT, characterized in that, Includes the following steps: S1. Construct a quantized baseline and set of observation points for the MOT model to be deployed, and generate a set of sensitive candidate operators; S2. Optimize the sensitive candidate operator set using the minimum set principle to obtain the minimum subset.
2. The operator-sensitivity-driven MOT hybrid precision quantization method according to claim 1, characterized in that, S1 includes the following sub-steps: S11. Construct a quantization baseline for the MOT model to be deployed and generate a set of observation points; S12. Determine the output tensor or statistic for each observation point in the observation point set; S13. Calculate the quantization error index for each observation point based on the output tensor or statistics of each observation point. S14. Construct waveform curves based on the quantization error index of each observation point; S15. Generate a set of sensitive candidate operators based on the waveform curve.
3. The operator-sensitivity-driven MOT hybrid precision quantization method according to claim 2, characterized in that, In S11, the set of observation points includes key convolutional or residual units of the detection backbone network, upsampling and fusion nodes of the feature pyramid, classification and regression output nodes of the detection head, and ReID feature extraction branches.
4. The operator-sensitivity-driven MOT hybrid precision quantization method according to claim 2, characterized in that, In S14, the quantification error indicators include cosine similarity, mean absolute error, and relative Euclidean distance.
5. The operator-sensitivity-driven MOT hybrid precision quantization method according to claim 2, characterized in that, In step S14, the cosine similarity of each observation point is arranged according to the operator index to form a sequence, and the sequence is subjected to segmented probability statistics, kernel density estimation or histogram densityization to obtain the waveform curve.
6. The operator-sensitivity-driven MOT hybrid precision quantization method according to claim 2, characterized in that, In S15, observation points in the waveform curve that show waveform concavity, frequent crossovers, enhanced local oscillations, or the formation of abnormal clusters are used as a set of sensitive candidate operators. The specific meaning of the waveform concavity is: ,in, This represents the mean cosine similarity of all observation points. The standard deviation of the cosine similarity of all observation points. Represents the cosine similarity value of the observation points; The frequent crossover specifically means that the waveform curve and the baseline waveform are compared point by point using a sliding window of length W, and the sign of the difference between the waveform curve and the baseline waveform changes no less than 2 times within the sliding window. The local oscillation enhancement specifically refers to the standard deviation of the cosine similarity within a local window formed by several consecutive observation points satisfying... ,in, The standard deviation of cosine similarity within a local window. Represents a constant; The anomalous cluster is specifically defined as follows: after kernel density estimation of the cosine similarity sequence, a secondary peak independent of the main distribution appears, and the probability density quality covered by the secondary peak accounts for a proportion of the total not less than a set threshold.
7. The operator-sensitivity-driven MOT hybrid precision quantization method according to claim 1, characterized in that, In step S2, a greedy strategy, a budget constraint strategy, or a multi-objective scoring strategy is used to optimize the set of sensitive candidate operators to obtain the minimum subset.
8. The operator-sensitivity-driven MOT hybrid precision quantization method according to claim 7, characterized in that, The greedy strategy is as follows: construct a sensitivity scoring function for each candidate node in the sensitive candidate operator set, sort the candidate nodes according to the scores, and generate a mixed precision set using the minimum set principle.
9. The operator-sensitivity-driven MOT hybrid precision quantization method according to claim 8, characterized in that, The sensitivity scoring function In this approach, cosine similarity deviation is taken as the main term, mean absolute error and relative Euclidean distance are taken as auxiliary terms, and the number of parameters, computational cost or bandwidth usage of candidate nodes are taken as cost terms. Its expression is: ； in, The weighting coefficients representing the cosine similarity deviation. The weighting coefficients represent the mean absolute error. The weighting coefficients represent the relative Euclidean distance. Indicates the first Cosine similarity deviation of each node Indicates the first The mean absolute error of each node Indicates the first The relative Euclidean distance between the nodes This represents a cost item.