Anti-interference visual tracking system, method and intelligent driving system oriented to expected functional safety

Abstract

The application discloses an anti-interference visual tracking system and method for expected functional safety, and an intelligent driving system, and relates to the technical field of intelligent driving. The system comprises a dynamic template library for storing main stem template features and auxiliary structure template features corresponding to a current effective target template. A feature extraction network processes a current search area image to obtain main stem search features and auxiliary structure search features. A semantic calibration unit determines a structure consistency matrix between the auxiliary structure template features and the auxiliary structure search features. A multi-head attention calculation module injects the structure consistency matrix into attention score calculation as a bias term to obtain calibrated attention features. A double-factor gate discriminator performs target position regression and classification on the calibrated attention features to obtain classification confidence and feature response intensity, and outputs a dynamic template library update signal when the classification confidence and the feature response intensity meet preset conditions. The application can improve expected functional safety and robustness.

Description

Technical Field

[0001] This invention relates to the fields of intended functional safety and visual target tracking technology, and in particular to an anti-interference visual tracking system, method and intelligent driving system for intended functional safety. Background Technology

[0002] Within the Expected Functional Safety (SOTIF) framework, autonomous driving perception systems are prone to performance degradation in scenarios such as nighttime and strong light sources, representing a typical performance limitation and triggering condition issue. Existing Transformer-based visual trackers use a self-attention mechanism for target matching and rely on classification confidence to update the template library. However, in practical deployments, strong light interference can easily cause attention to be hijacked by bright backgrounds, and the template update mechanism lacks effective verification of perception quality, potentially leading the system into an unsafe state.

[0003] Currently, mainstream Transformer tracking frameworks (such as ProContEXT) employ self-attention mechanisms for feature interaction between the target and the search region. Furthermore, these methods typically rely solely on classification confidence scores to determine whether to update the target features of the current frame to the dynamic template library.

[0004] Existing tracking systems have the following shortcomings that are directly related to intended functional safety: (1) The problem of light spot misdirection in attention mechanism: Traditional self-attention mechanism is sensitive to pixel intensity. Bright background noise gets higher attention weight, which leads to target loss. This is a known unsafe scenario for perception system. (2) Confidence spoofing problem in template update: relying solely on classification confidence, template updates with high confidence and low quality are prone to occur under strong light interference, leading to error accumulation and causing the perception system to remain in a state of functional inadequacy in subsequent frames.

[0005] The disclosure of the above background technical content is only for the purpose of assisting in understanding the concept and technical solution of this application, and does not necessarily provide technical instruction. Summary of the Invention

[0006] The purpose of this invention is to provide an anti-interference visual tracking system, method, and intelligent driving system with expected functional safety. This invention can improve expected functional safety and reduce the risk of insufficient functionality of the perception system in nighttime scenarios.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: An interference-resistant visual tracking system with intended functional safety includes: A dynamic template library, configured to store the main template features and auxiliary structure template features corresponding to the currently valid target template; A feature extraction network is configured to process the current search region image to obtain backbone search features and auxiliary structure search features; A semantic calibration unit is configured to determine a structural consistency matrix between the auxiliary structural template features and the auxiliary structural search features; The multi-head attention calculation module is configured to use the backbone template features as the target reference, the backbone search features as the object to be matched, and inject the structural consistency matrix as a bias term into the attention score calculation to obtain the calibrated attention features. The attention score calculation is a matching calculation between the current search region image and the current effective target template. A two-factor gated discriminator is configured to perform target location regression and classification on calibrated attention features to obtain classification confidence and deep feature vectors, and to calculate the feature response intensity of the deep feature vectors. The two-factor gated discriminator is also configured to output an update signal when it is determined that the classification confidence and feature response intensity meet preset conditions, so that the features corresponding to the tracked target in the current frame search region image are updated to the dynamic template library.

[0008] Furthermore, based on any one or a combination of the aforementioned technical solutions, if the classification confidence level... and characteristic response intensity The two-factor gated discriminator outputs an update signal if the following dual-gating logic conditions are met: and or satisfy and If the dynamic template library is updated, then it is updated; otherwise, it is not updated. The preset first confidence threshold, The second confidence threshold is preset. The preset baseline activity threshold, For the preset strict structural threshold, .

[0009] Furthermore, following any one or a combination of the aforementioned technical solutions, the semantic calibration unit is configured to calculate the structural consistency matrix between the auxiliary structural template features and the auxiliary structural search features based on a cosine similarity algorithm. ): ; in, Representing the structural consistency matrix The One element, Indicating the first feature of the auxiliary structure template i The template branch vector corresponding to each spatial location Indicating the first auxiliary structure search feature j The search branch vector corresponding to each spatial location Representing vectors Norm.

[0010] Furthermore, following any one or a combination of the aforementioned technical solutions, the semantic calibration unit includes a structure matcher, which includes a feature normalization layer and a similarity measurement operator. The feature normalization layer performs a normalization operation on the auxiliary structure template features to obtain a template branch vector, and performs a normalization operation on the auxiliary structure search features to obtain a search branch vector. The similarity metric operator is configured to calculate the structural consistency matrix between the template branch vector and the search branch vector based on the cosine similarity algorithm.

[0011] Furthermore, following any or a combination of the aforementioned technical solutions, the semantic calibration unit further includes a bias injection interface, the input of which is connected to the output of the structure matcher, and the output of which is connected to the matrix addition node in the multi-head attention calculation module.

[0012] Furthermore, following any one or a combination of the aforementioned technical solutions, the multi-head attention calculation module superimposes the structural consistency matrix onto the image features to obtain calibrated attention features in the following manner. : ; in, This represents the activation function operation, and T represents the matrix transpose. Indicates the adaptive adjustment factor. Let Q be the structural consistency matrix, Q be the query vector generated based on the backbone search features, K be the key vector generated based on the backbone template features, V be the value vector generated based on the backbone template features, and d be the feature dimension scalar in the image features.

[0013] Furthermore, based on any one or a combination of the aforementioned technical solutions, the formulas for calculating Q, K, and V are as follows: ； ； ； in, This is the query projection parameter matrix within the multi-head attention calculation module. This refers to the key projection parameter matrix within the multi-head attention calculation module. This is the value projection parameter matrix within the multi-head attention calculation module.

[0014] Furthermore, based on any or a combination of the aforementioned technical solutions, the dual-factor gated discriminator includes a tracking head module; The tracking head module is configured to perform target location regression and classification on the calibrated attention features to obtain classification confidence and deep feature vectors, and to calculate the feature response intensity of the deep feature vectors in the following manner. : ;in, For deep feature vectors, This represents the total number of channels in the deep feature vector. For the deep feature vector at the th The activation value on each channel.

[0015] Furthermore, following any one or a combination of the aforementioned technical solutions, the dual-factor gated discriminator further includes a logic verifier, which is configured to receive the classification confidence and feature response intensity output by the tracking head module, and perform asymmetric dual-threshold gated logic judgment on the classification confidence and feature response intensity to determine whether to update the dynamic template library.

[0016] Furthermore, based on any or a combination of the aforementioned technical solutions, the feature extraction network includes a Transformer backbone network and an auxiliary feature mapping module; The Transformer backbone network is configured to receive and process the current search region image to obtain the backbone search features, and the Transformer backbone network is also configured to output the intermediate layer image features output by its encoder to the auxiliary feature mapping module. The auxiliary feature mapping module is configured to take the intermediate layer image features as input, obtain auxiliary structure search features based on the multilayer perceptron projection network, and output them to the semantic calibration unit.

[0017] According to another aspect of the present invention, an interference-resistant visual tracking method for intended functional safety is provided, based on any or a combination of the above-described technical solutions, comprising the following steps: A feature extraction network is used to extract backbone search features and auxiliary structure search features from the current search region image; A semantic calibration unit is used to determine the structural consistency matrix between the auxiliary structural template features and the auxiliary structural search features; A multi-head attention calculation module is adopted, using the backbone template features as the target reference and the backbone search features as the objects to be matched. The structural consistency matrix is ​​injected as a bias term into the attention score calculation to obtain the calibrated attention features. The attention score calculation is a matching calculation between the current search region image and the current effective target template. A two-factor gated discriminator is used to perform target location regression and classification on the calibrated attention features to obtain classification confidence and deep feature vectors, and to calculate the feature response intensity of the deep feature vectors. The two-factor gated discriminator is also configured to output an update signal when it is determined that the classification confidence and feature response intensity meet preset conditions, so that the features corresponding to the tracked target in the current frame search region image are updated to the dynamic template library.

[0018] According to another aspect of the present invention, an intelligent driving system is provided, which is an interference-resistant visual tracking system for intended functional safety based on any or a combination of the above technical solutions.

[0019] The beneficial effects of the technical solution provided by this invention are as follows: a. Enhance expected functional safety capabilities: effectively suppress attention drift caused by strong light source interference, and reduce the risk of insufficient functionality of the perception system in nighttime scenarios; b. Enhance template update reliability: Introducing feature response intensity as an orthogonal verification dimension can avoid high-confidence noise contaminating the template library and support SOTIF's requirements for system status monitoring; c. Improve recall rate for faint targets: Maintain tracking capability for targets with complete structure but blurry appearance, and expand the safe operating range of the perception system. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 An architecture diagram of an interference-resistant visual tracking system for intended functional safety is provided as an exemplary embodiment of the present invention. Figure 2 A flowchart illustrating the operation of the backbone network of an anti-interference visual tracking system, provided as an exemplary embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0024] To address the shortcomings of existing technologies, such as attention inaccuracies in perception systems under strong light interference, resulting in the inability to distinguish semantic targets from highlighted backgrounds, and the lack of independent verification of perception quality in template update strategies, which easily leads to irreversible tracking drift, this application introduces geometric structure priors through a Structural Semantic Calibration Unit (SCU) to forcibly correct the attention distribution and improve the robustness of the perception system in interference scenarios. Furthermore, by introducing feature response intensity as a confidence-independent quality evaluation index through a Two-Factor Gated Discriminator (DFGD), it can avoid high-confidence noise contaminating the template library, thereby enhancing the expected functional safety capabilities of the system.

[0025] This invention constructs an anti-interference tracking system (hereinafter referred to as "tracker") oriented towards Safety for Expected Functional Principles (SOTIF). The tracker is not a simple single component. This application obtains a tracker composed of a feature extraction network, a multi-head attention calculation module, a semantic calibration unit, and a two-factor gated discriminator by embedding a semantic calibration unit inside the Transformer and a two-factor gated discriminator in the post-processing stage.

[0026] In one embodiment of the present invention, an interference-resistant visual tracking system oriented towards intended functional safety is provided, see [link to relevant documentation]. Figure 1 and Figure 2 The system includes: A dynamic template library, configured to store the backbone template features corresponding to the currently valid target template. and auxiliary structural template features; A feature extraction network is configured to process the image of the current search region to obtain backbone search features. and auxiliary structure search features; A semantic calibration unit is configured to determine a structural consistency matrix between the auxiliary structural template features and the auxiliary structural search features; The multi-head attention calculation module is configured to use the backbone template features as the target reference, the backbone search features as the object to be matched, and inject the structural consistency matrix as a bias term into the attention score calculation to obtain the calibrated attention features. The attention score calculation is a matching calculation between the current search region image and the current effective target template. A two-factor gated discriminator is configured to perform target location regression and classification on calibrated attention features to obtain classification confidence and deep feature vectors, and to calculate the feature response intensity of the deep feature vectors. The two-factor gated discriminator is also configured to output an update signal when it is determined that the classification confidence and feature response intensity meet preset conditions, so that the features corresponding to the tracked target in the current frame search region image are updated to the dynamic template library.

[0027] In existing technologies, the dynamic template library is used to store currently recognized target template features, providing target references for the image to be processed and enabling continuous tracking of the same visual target. Compared to a static template library, a continuously updated dynamic template library has the following advantages: on the one hand, by receiving reliable target template features from the current frame, it can better adapt to changes in target pose, scale, and lighting; on the other hand, by maintaining the original valid template when the current frame result is unreliable, it can avoid erroneous features entering the template library and causing subsequent tracking drift. Existing tracking technologies typically store template features of the tracked target in the current frame or historical frames, and mainly determine whether to update the current frame target features to the template library based on classification confidence. This mechanism allows the template to be updated according to changes in target appearance, but in scenarios such as strong light at night or backlight, background glare may generate high-confidence, low-quality erroneous features, thereby polluting the template library and leading to error accumulation.

[0028] In this embodiment, the dynamic template library is configured to store the backbone template features and auxiliary structure template features corresponding to the currently valid target. When the update condition is met, the two types of template features are updated with the new features corresponding to the reliably tracked target in the current frame; when the update condition is not met, the two types of template features remain unchanged. For the current frame, the feature extraction network directly processes the current search region image to obtain the backbone search features and auxiliary structure search features; the two types of template features are not used as input or reference to the feature extraction network, but participate in subsequent attention matching and semantic calibration respectively after feature extraction is completed.

[0029] The dynamic template library in this application still uses target template features as the core storage object, but its key improvement lies in the admission mechanism of template features: only target template features of the current frame that simultaneously meet the requirements of classification confidence and feature response strength are allowed to be updated to the dynamic template library. Therefore, this application does not simply expand the storage content of the template library, but controls the update quality of the template library through two-factor gating.

[0030] Existing attention computation only relies on and The inner product. However, in low light, the real... Matching can be overwhelmed by noise. Therefore, this application introduces a set of specially trained auxiliary features capable of perceiving dark contours as a structure guide.

[0031] The feature extraction network adds an auxiliary feature mapping module to the existing Transformer backbone network. The Transformer backbone network is configured to receive and process the current search region image to obtain the backbone search features; that is, the backbone search features are the output features of the Transformer backbone network. The Transformer backbone network is also configured to output the intermediate layer image features from its encoder to the auxiliary feature mapping module.

[0032] The auxiliary feature mapping module is configured to take the intermediate layer image features as input, obtain auxiliary structure search features based on a multilayer perceptron projection network, and output them to the semantic calibration unit. Specifically, a lightweight multilayer perceptron (MLP) projection network is used to construct an MLP bypass on the Transformer backbone network to serve as the auxiliary feature mapping module. The auxiliary feature mapping module is based on a multilayer perceptron, takes the intermediate layer image features as input, and outputs auxiliary structure search features that have been stripped of absolute brightness information and retain only the target edges and geometric topological attributes.

[0033] The training of the auxiliary feature mapping module is as follows: During the offline training phase of the model, a geometric consistency constraint loss is applied separately to the auxiliary feature mapping module for targeted fine-tuning. This training method forces the network to align features in low-light scenes with the real physical contours of the target, making the final output auxiliary features insensitive to environmental interference such as local bright spots, thus providing reliable geometric input to overcome the system perception limitations in SOTIF.

[0034] The Semantic Calibration Unit (SCU) consists of two core sub-modules: a structure matcher and a bias injection interface. This unit is physically embedded in the Transformer's Multi-Head Attention module.

[0035] The structure matcher is the core computational component of the semantic calibration unit (SCU). Its input is connected to the auxiliary feature output interface of the feature extraction network to receive the auxiliary structure search features v, which represent the geometric topology of the target, output by the backbone network.

[0036] Furthermore, the structure matcher is also configured to receive the auxiliary structural template features. Specifically, the output of the dynamic template library can be connected to the input of the feature extraction network, which receives the backbone template features and the auxiliary structural template features, then outputs the backbone template features to the subsequent multi-head attention calculation module and outputs the auxiliary structural search features to the semantic calibration unit. Alternatively, the output of the dynamic template library can be connected to the input of the semantic calibration unit and the input of the multi-head attention calculation module, respectively, with the dynamic template library outputting the backbone template features to the multi-head attention calculation module and the auxiliary structural search features to the semantic calibration unit.

[0037] The structure matcher consists of a feature normalization layer and a similarity measurement operator. The structure matcher is used to calculate the geometric similarity between the auxiliary structure template features and the auxiliary structure search features in the auxiliary semantic space, which is the enhanced dark-light feature space.

[0038] The feature normalization layer includes an auxiliary geometric feature calculation template branch (Target Template) and a search region branch (Search Region). The auxiliary geometric feature calculation template branch performs a normalization operation on the auxiliary structural template features to obtain the template branch vector. The search region branch performs a normalization operation on the search features of the auxiliary structure to obtain the search branch vector. The similarity metric operator calculates the structural consistency matrix between the template branch vector and the search branch vector based on the cosine similarity algorithm. The calculation formula is as follows: Structural Consistency Matrix The The elements are: ; in, This represents the template branch vector corresponding to the i-th spatial location (i.e., the i-th token) in the auxiliary structure template features. : Represents the search branch vector corresponding to the j-th spatial location (i.e., the j-th token) in the auxiliary structure search features. Representing vectors Norm (i.e., the Euclidean modulus of a vector).

[0039] Because auxiliary features preserve the geometric texture of the target, the target region The value will be very high, while cluttered background noise (such as light spots), although bright, does not have the same geometric structure as the template. The value is low or even negative. This forces attention to focus on the real target area, thereby solving the limitations of perception algorithms and attention biases caused by triggering conditions such as strong light at night and backlight in SOTIF (Expected Functional Safety).

[0040] Furthermore, this application creatively employs the cosine similarity algorithm to measure the geometric structural consistency of auxiliary features, and through... The norm normalizes the auxiliary feature vector, making the calculation formula focus only on the directional consistency of features in the auxiliary semantic space, thus eliminating the interference of absolute illumination intensity (i.e., vector magnitude). Based on directional consistency (cosine similarity), it is insensitive to illumination intensity and can effectively distinguish targets with complete structures from bright noise.

[0041] The output of the structure matcher is connected to the input of the bias injection interface, and is used to output a two-dimensional structure consistency matrix to the bias injection interface. The bias injection interface is connected to the matrix addition node before the existing standard normalization function Softmax within the feature extraction network (Transformer backbone architecture). It is used to add the calculated structural correlation matrix (i.e., the structural consistency matrix representing geometric similarity output by the structural matcher) to the matrix. The feature matching distribution map is superimposed onto the original attention map (which is the feature matching distribution map obtained by performing a dot product between the existing query vector Q and the key vector K of the Transformer).

[0042] Existing techniques (such as the conventional Transformer self-attention calculation mechanism) rely solely on the inner product of the query vector Q and the key vector K to calculate attention weights. In extreme scenarios defined by Expected Functional Safety (SOTIF), such as nighttime bright light and backlighting, the pixel intensity of high-brightness background noise (such as streetlights and oncoming vehicle headlights) is extremely high. This high-frequency noise can lead to... Inner product operations generate abnormally high response values ​​in the interference region, causing the system's attention to be hijacked by semantically meaningless light spots, making it unable to focus on the real, faint target, thus causing severe tracking drift and target loss.

[0043] In this application, the multi-head attention calculation module receives a bias structure consistency matrix input by the bias injection interface, as well as backbone search features output by the feature extraction network and backbone search features output by the dynamic template library.

[0044] In one implementation of cross-attention, the multi-head attention computation module uses the backbone template features output by the dynamic template library ( Using ) as the target reference, the current frame backbone search features ( The multi-head attention calculation module generates a query vector Q based on the backbone search features, and generates a key vector K and a value vector V based on the backbone template features. ； ； ； Among them, among them, This is the query projection parameter matrix within the multi-head attention calculation module. This refers to the key projection parameter matrix within the multi-head attention calculation module. This is the value projection parameter matrix within the multi-head attention computation module. These three parameter matrices map the corresponding input features to the feature space containing the query, key, and value. They are internal trainable parameters of the multi-head attention computation module, do not constitute storage content in the dynamic template library, and do not constitute a new module independent of the multi-head attention computation module.

[0045] The multi-head attention calculation module matches the current frame search region with the current valid target template using the query vector, key vector, and value vector; and processes the structural consistency matrix output by the semantic calibration unit. The attention score is injected as a bias term, resulting in calibrated attention features before Softmax normalization: ; in, This represents the activation function operation, and T represents the matrix transpose. Indicates the adaptive adjustment factor. Let V be the structural consistency matrix, Q be the query vector generated based on the backbone search features, K be the key vector generated based on the backbone template features, and V be the value vector generated based on the backbone template features.d is the feature dimension scalar in the image features.

[0046] This application introduces a structural consistency matrix into the standard attention mechanism. As an explicit bias term, due to The calculation is not based on grayscale values, but rather on the directional consistency (cosine similarity) of deep feature vectors in the auxiliary semantic space. Its physical characteristic is that it is completely insensitive to absolute illumination intensity, thus forcing attention back to a target with the correct geometric structure. This imposes a priori physical constraint on the geometric structure for self-attention calculation. Specifically, even if the bright spot is in the original... A falsely high score obtained in the calculation may also be due to the lack of the true geometric topology of the target (leading to its corresponding...) (Values ​​that are extremely low or even negative) are significantly weakened and suppressed by this bias term at matrix addition nodes; conversely, real targets that appear dim but have complete geometric edges receive a significant boost from this bias term. This mechanism breaks away from the traditional dependence of attention on pixel brightness at the underlying computational logic, solves the problem of attention diffusion in complex lighting environments, and can greatly improve the robustness of autonomous driving perception systems.

[0047] Existing technologies generally assume that high confidence equals good target quality. However, while strong nighttime light noise lacks semantic information, it can generate artificially high confidence through high-frequency signals. To address this technical issue, this application introduces Feature Response Intensity (FRI) as a second discriminant dimension and proposes an orthogonal verification method based on signal energy and confidence. The physical basis is that real objects have stable semantic activation in deep networks, manifested as high feature vector magnitude (energy); while pure background light noise, after layers of convolution and cross-layer constraints, has its feature vector suppressed, manifested as a lower magnitude.

[0048] The dual-factor gating discriminator is configured to perform orthogonal verification based on signal energy and confidence level. The dual-factor gating discriminator (DFGD) is located at the tracker's decision output and includes a tracking head module and a logic verifier.

[0049] The tracking head module includes a target position regression unit, a target classification unit, a feature output unit, and an intensity calculator. The target position regression unit performs target position regression on the calibrated attention features; the target classification unit performs target classification on the calibrated attention features and outputs a classification confidence score (factor 1); and the feature output unit outputs the deep feature vector of the target in the current frame. The intensity calculator takes the deep feature vector as input and calculates the feature response intensity (factor 2) using the following formula: ; in, This refers to the feature response intensity. To track the deep feature vector output by the head module, This represents the total number of channels in the deep feature vector. For this deep feature vector at the th The activation value on each channel.

[0050] The output of the tracking head module is electrically connected to the input of the logic verifier, and the tracking head module outputs the classification confidence score to the logic verifier. The output of the intensity calculator is connected to the input of the logic verifier, and the intensity calculator outputs the feature response intensity to the logic verifier. This application will use vectors The norm (Euclidean modulus) calculation method is applied to measure the physical energy of deep semantic signals to characterize the structural integrity of targets under extreme lighting conditions in SOTIF.

[0051] The logic verifier is internally configured with dual threshold comparison logic circuitry to output an update / non-update Boolean signal to control the dynamic template library's update / non-update. This mechanism constructs the perceived quality verification link in SOTIF at the algorithmic level, avoiding system function degradation caused by template contamination.

[0052] Specifically, the logic verifier includes a noise suppression gate and a faint target acquisition gate, which together constitute the core decision circuit inside the two-factor gated discriminator (DFGD). In terms of connection, the inputs of these two gated units are simultaneously connected in parallel to the classification confidence output of the tracking head module and the energy output of the intensity calculator (i.e., simultaneously receiving...). and The outputs of these two gates are combined via logical OR gates and then connected to the system's dynamic template library control interface. Logically, these two gates operate in parallel and mutually exclusive order, with no specific sequence required. The system simultaneously feeds input data into both gates for evaluation in real time. If the pass condition of either gate is met, the system outputs True, allowing the update; otherwise, it outputs False.

[0053] To address the issue of erroneous updates caused by high-confidence noise at night, such as that from streetlights, this invention designs an asymmetric dual-threshold gating logic based on confidence intervals. The specific decision logic is as follows: (1) Noise Rejection Gate: When the classification confidence level is high... Very high (e.g.) ), but characteristic response strength Very low (e.g.) When this occurs, the system determines that the current issue is high-resolution light spot interference and forcibly prohibits template updates. (2) Dim Target Capture Gate: When the classification confidence is moderate ( This feature response intensity is typically ignored by traditional algorithms. However, if the feature response intensity is extremely high at this point, it indicates that although the classifier is uncertain, the feature extractor has detected the complete target structure, and the system determines it as a valid target, allowing the template to be updated.

[0054] Preferably, four hyperparameters are preset in the system: a high threshold for classification confidence. With low threshold And two energy thresholds: the basic activity threshold. With strict structure threshold And satisfy The four hyperparameters above can be set empirically, but are more preferably determined in the following way: statistical analysis of the energy probability density distribution of real target features and pure background light spot noise features in a large number of real night scene datasets (such as NAT2021) during the offline stage, and the obtained empirical prior constants are used as the four hyperparameters above.

[0055] The two-factor gated discriminator is configured to control the updating of the dynamic template library according to the following piecewise decision logic function: in, This is the final Boolean control signal output by the logic checker (if True, an instruction to perform an update is sent to the dynamic template library; if False, the update is rejected). This is the logical AND operator.

[0056] When the update conditions are met, the logic verifier outputs an update instruction, determines the target position based on the reliable tracking results of the current frame, obtains the corresponding target image, extracts the backbone template features and auxiliary structure template features through the feature extraction network, and updates the dynamic template library using the two types of features; when the update conditions are not met, the two types of template features remain unchanged to avoid erroneous features from unreliable frames from entering the dynamic template library.

[0057] The interference-resistant visual tracking system for intended functional safety provided by this invention has the following advantages: (1) Enhance expected functional safety capabilities: effectively suppress attention drift caused by strong light source interference and reduce the risk of insufficient functionality of the perception system in nighttime scenarios; (2) Enhance the reliability of template updates: Introducing feature response intensity as an orthogonal verification dimension can avoid high confidence noise from polluting the template library and support SOTIF’s requirements for system status monitoring; (3) Improve the recall rate of faint targets: maintain the tracking ability for targets with complete structure but blurry appearance, and expand the safe operating range of the perception system.

[0058] In one embodiment of the present invention, an interference-resistant visual tracking method for intended functional safety is provided, based on the interference-resistant visual tracking system for intended functional safety described in any or a combination of the above embodiments, including the following steps: A feature extraction network is used to extract backbone search features and auxiliary structure search features from the current search region image; A semantic calibration unit is used to determine the structural consistency matrix between the auxiliary structural template features and the auxiliary structural search features; A multi-head attention calculation module is adopted, using the backbone template features as the target reference and the backbone search features as the objects to be matched. The structural consistency matrix is ​​injected as a bias term into the attention score calculation to obtain the calibrated attention features. The attention score calculation is a matching calculation between the current search region image and the current effective target template. A two-factor gated discriminator is used to perform target location regression and classification on the calibrated attention features to obtain classification confidence and deep feature vectors, and to calculate the feature response intensity of the deep feature vectors. The two-factor gated discriminator is also configured to output an update signal when it is determined that the classification confidence and feature response intensity meet preset conditions, so that the features corresponding to the tracked target in the current frame search region image are updated to the dynamic template library.

[0059] According to another aspect of the present invention, an intelligent driving system is provided, including an interference-resistant visual tracking system for intended functional safety as described in any of the above embodiments.

[0060] It should be noted that the anti-interference visual tracking method embodiment for expected functional safety, the intelligent driving system embodiment, and the above-mentioned anti-interference visual tracking system embodiment for expected functional safety are based on the same inventive concept. All contents of the anti-interference visual tracking system embodiment for expected functional safety are incorporated into the anti-interference visual tracking method embodiment and the intelligent driving system embodiment for expected functional safety by reference.

[0061] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0062] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An interference-resistant visual tracking system with intended functional safety, characterized in that, include: A dynamic template library, configured to store the main template features and auxiliary structure template features corresponding to the currently valid target template; A feature extraction network is configured to process the current search region image to obtain backbone search features and auxiliary structure search features; A semantic calibration unit is configured to determine a structural consistency matrix between the auxiliary structural template features and the auxiliary structural search features; The multi-head attention calculation module is configured to use the backbone template features as the target reference, the backbone search features as the object to be matched, and inject the structural consistency matrix as a bias term into the attention score calculation to obtain the calibrated attention features. The attention score calculation is a matching calculation between the current search region image and the current effective target template. A two-factor gated discriminator is configured to perform target location regression and classification on calibrated attention features to obtain classification confidence and deep feature vectors, and to calculate the feature response intensity of the deep feature vectors. The two-factor gated discriminator is also configured to output an update signal when it is determined that the classification confidence and feature response intensity meet preset conditions, so that the features corresponding to the tracked target in the current frame search region image are updated to the dynamic template library.

2. The interference-resistant visual tracking system for intended functional safety according to claim 1, characterized in that, If classification confidence and characteristic response intensity The two-factor gated discriminator outputs an update signal if the following dual-gating logic conditions are met: and or satisfy and If the dynamic template library is updated, then it is updated; otherwise, it is not updated. The preset first confidence threshold, The second confidence threshold is preset. The preset baseline activity threshold, For the preset strict structural threshold, .

3. The interference-resistant visual tracking system for intended functional safety according to claim 1, characterized in that, The semantic calibration unit is configured to calculate the structural consistency matrix between the auxiliary structural template features and the auxiliary structural search features based on a cosine similarity algorithm. ): ； in, Representing the structural consistency matrix The One element, Indicating the first feature of the auxiliary structure template i The template branch vector corresponding to each spatial location Indicating the first auxiliary structure search feature j The search branch vector corresponding to each spatial location Representing vectors Norm.

4. The interference-resistant visual tracking system for intended functional safety according to claim 3, characterized in that, The semantic calibration unit includes a structure matcher, which includes a feature normalization layer and a similarity measurement operator. The feature normalization layer performs a normalization operation on the auxiliary structure template features to obtain a template branch vector, and performs a normalization operation on the auxiliary structure search features to obtain a search branch vector. The similarity metric operator is configured to calculate the structural consistency matrix between the template branch vector and the search branch vector based on the cosine similarity algorithm.

5. The interference-resistant visual tracking system for intended functional safety according to claim 2, characterized in that, The semantic calibration unit further includes a bias injection interface, the input of which is connected to the output of the structure matcher, and the output of which is connected to the matrix addition node in the multi-head attention calculation module.

6. The interference-resistant visual tracking system for intended functional safety according to claim 1, characterized in that, The multi-head attention calculation module superimposes the structural consistency matrix onto the image features to obtain calibrated attention features in the following manner. : ； in, This represents the activation function operation, and T represents the matrix transpose. Indicates the adaptive adjustment factor. Let Q be the structural consistency matrix, Q be the query vector generated based on the backbone search features, K be the key vector generated based on the backbone template features, V be the value vector generated based on the backbone template features, and d be the feature dimension scalar in the image features.

7. The interference-resistant visual tracking system for intended functional safety according to claim 6, characterized in that, The formulas for calculating Q, K, and V are as follows: ； ； ； in, This is the query projection parameter matrix within the multi-head attention calculation module. This refers to the key projection parameter matrix within the multi-head attention calculation module. This is the value projection parameter matrix within the multi-head attention calculation module.

8. The interference-resistant visual tracking system for intended functional safety according to claim 1, characterized in that, The two-factor gated discriminator includes a tracking head module; The tracking head module is configured to perform target location regression and classification on the calibrated attention features to obtain classification confidence and deep feature vectors, and to calculate the feature response intensity of the deep feature vectors in the following manner. : ;in, For deep feature vectors, This represents the total number of channels in the deep feature vector. For the deep feature vector at the th The activation value on each channel.

9. The interference-resistant visual tracking system for intended functional safety according to claim 1, characterized in that, The two-factor gated discriminator further includes a logic verifier, which is configured to receive the classification confidence and feature response intensity output by the tracking head module, and perform asymmetric dual-threshold gated logic judgment on the classification confidence and feature response intensity to determine whether to update the dynamic template library.

10. The interference-resistant visual tracking system for intended functional safety according to claim 1, characterized in that, The feature extraction network includes a Transformer backbone network and an auxiliary feature mapping module; The Transformer backbone network is configured to receive and process the current search region image to obtain the backbone search features, and the Transformer backbone network is also configured to output the intermediate layer image features output by its encoder to the auxiliary feature mapping module. The auxiliary feature mapping module is configured to take the intermediate layer image features as input, obtain auxiliary structure search features based on the multilayer perceptron projection network, and output them to the semantic calibration unit.

11. A robust visual tracking method for intended functional safety, characterized in that, The interference-resistant visual tracking system based on the intended functional safety as described in claim 1 includes the following steps: A feature extraction network is used to extract backbone search features and auxiliary structure search features from the current search region image; A semantic calibration unit is used to determine the structural consistency matrix between the auxiliary structural template features and the auxiliary structural search features; A multi-head attention calculation module is adopted, using the backbone template features as the target reference and the backbone search features as the objects to be matched. The structural consistency matrix is ​​injected as a bias term into the attention score calculation to obtain the calibrated attention features. The attention score calculation is a matching calculation between the current search region image and the current effective target template. A two-factor gated discriminator is used to perform target location regression and classification on the calibrated attention features to obtain classification confidence and deep feature vectors, and to calculate the feature response intensity of the deep feature vectors. The two-factor gated discriminator is also configured to output an update signal when it is determined that the classification confidence and feature response intensity meet preset conditions, so that the features corresponding to the tracked target in the current frame search region image are updated to the dynamic template library.

12. An intelligent driving system, characterized in that, Based on the interference-resistant visual tracking system for intended functional safety as described in claim 1.

Images