A method, device and equipment for identifying a target object in an automatic driving scene

Abstract

The application provides a kind of identification method, device and equipment of target object in automatic driving scene, belong to information processing technical field.The method comprises: obtaining image data under the automatic driving scene;Feature processing is carried out to the image data, and feature data set is obtained;Feature data set is input into target object identification model and is handled, and the identification result of target object in image data is obtained;Wherein, target object identification model will training data set be input into the feature separation layer of preset network model and be handled with feature separation, and first robust feature data and first non-robust feature data are obtained, and according to first robust feature data and first non-robust feature data, training feature data is obtained;Training feature data is input into the target object identification layer of preset network model and is trained to obtain.The scheme provided in the application can improve the performance of the model with less calculation amount, and facilitate end-to-end deployment and application in the automatic driving scene.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a method, apparatus and device for identifying target objects in an autonomous driving scenario. Background Technology

[0002] Deep neural networks have made significant progress in fields such as computer vision, natural language processing, and autonomous driving, impacting a wide range of real-world applications. Due to the widespread use of neural network-based classifiers, ensuring their reliability in practice is crucial. Existing deep neural networks are vulnerable to adversarial attacks—attacks that deliberately create and input carefully crafted images to trick the model into making incorrect decisions or predictions. These attacks typically target visual perception systems, such as those used in autonomous driving scenarios for image recognition or classification tasks based on deep neural network models, but can also be applied to other types of machine learning models. To combat the interference of adversarial examples, numerous defensive techniques have been proposed, including adversarial example detection, input denoising, moving target defense, ensemble model defense, and adversarial training. However, for these methods, small adversarial perturbations at the pixel level can accumulate to a greater extent in the intermediate feature space, corrupting the model's final output. Therefore, designing a method with better robustness while achieving a better trade-off between robustness and accuracy has become a research hotspot. Currently, there is no method to evaluate the effectiveness of robustness enhancement techniques in balancing robustness and accuracy. Summary of the Invention

[0003] This invention provides a method, apparatus, and device for identifying target objects in autonomous driving scenarios, which solves the problem that small adversarial perturbations at the pixel level in image data accumulate and amplify in the intermediate feature space during the identification process of target objects in autonomous driving, leading to the final identification failure.

[0004] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0005] A method for identifying target objects in an autonomous driving scenario includes:

[0006] Acquire image data in autonomous driving scenarios;

[0007] The image data is subjected to feature processing to obtain a feature dataset;

[0008] The feature dataset is input into the target object recognition model for processing to obtain the recognition result of the target object in the image data;

[0009] The target object recognition model inputs the training dataset into the feature separation layer of the preset network model for feature separation processing to obtain first robust feature data and first non-robust feature data, and obtains training feature data based on the first robust feature data and the first non-robust feature data; the training feature data is then input into the target object recognition layer of the preset network model for training.

[0010] Optionally, the image data is processed to obtain a feature dataset, including:

[0011] The image data is processed to obtain a feature dataset containing natural samples;

[0012] The dataset containing natural samples is perturbed or noise is added to obtain a feature dataset of adversarial samples.

[0013] Optionally, the training dataset is input into the feature separation layer of a predefined network model for feature separation processing to obtain first robust feature data and first non-robust feature data, including:

[0014] The training dataset is input into the feature separation layer of the preset network model for feature separation processing to obtain intermediate layer feature data.

[0015] The intermediate layer feature data is decomposed element-wise based on the robustness score to obtain the first robust feature data and the first non-robust feature data.

[0016] Optionally, training feature data is obtained based on the first robust feature data and the first non-robust feature data, including:

[0017] The first non-robust feature data is compressed to obtain the second non-robust feature data;

[0018] The first non-robust feature data is transformed to obtain the third non-robust feature data;

[0019] The first robust feature data is aligned to obtain the second robust feature data;

[0020] The second non-robust feature data, the third non-robust feature data, and the second robust feature data are processed to obtain training feature data.

[0021] Optionally, the first non-robust feature data is compressed to obtain the second non-robust feature data, including:

[0022] Obtain the weight data of the auxiliary layer of the target object recognition model;

[0023] The weighted data is multiplied by the first non-robust feature data to obtain the second non-robust feature data.

[0024] Optionally, the first non-robust feature data is transformed to obtain the third non-robust feature data, including:

[0025] Based on the training feature data, the mask data is obtained;

[0026] Through formula f r+ = (1-mask)×f nr The third non-robust feature data is obtained; where f r+ The third non-robust feature data, mask is the mask data, f nr This is the first non-robust feature data.

[0027] Optionally, the first robust feature data is aligned to obtain the second robust feature data, including:

[0028] The first robust feature data is subjected to global average pooling to obtain a representation vector;

[0029] The channel dimension of the representation vector is normalized to obtain the feature similarity matrix;

[0030] Based on the feature similarity matrix, determine the difference data of the similarity matrix;

[0031] Based on the difference data, the second robust feature data is obtained.

[0032] Optionally, the feature dataset is input into a target object recognition model for processing to obtain the recognition result of the target object in the image data, including:

[0033] The feature dataset is input into the target object recognition model for processing to obtain feature vectors;

[0034] The feature vectors are compressed, transformed, and aligned to obtain the recognition result of the target object in the image data.

[0035] Embodiments of the present invention also provide a target object recognition device in an autonomous driving scenario, the device comprising:

[0036] The acquisition module is used to acquire image data in autonomous driving scenarios;

[0037] The processing module is used to perform feature processing on the image data to obtain a feature dataset;

[0038] The generation module is used to input the feature dataset into the target object recognition model for processing, and obtain the recognition result of the target object in the image data;

[0039] The target object recognition model inputs the training dataset into the feature separation layer of the preset network model for feature separation processing to obtain first robust feature data and first non-robust feature data, and obtains training feature data based on the first robust feature data and the first non-robust feature data; the training feature data is then input into the target object recognition layer of the preset network model for training.

[0040] Optionally, the image data is processed to obtain a feature dataset, including:

[0041] The image data is processed to obtain a feature dataset containing natural samples;

[0042] The dataset containing natural samples is perturbed or noise is added to obtain a feature dataset of adversarial samples.

[0043] Optionally, the training dataset is input into the feature separation layer of a predefined network model for feature separation processing to obtain first robust feature data and first non-robust feature data, including:

[0044] The training dataset is input into the feature separation layer of the preset network model for feature separation processing to obtain intermediate layer feature data.

[0045] The intermediate layer feature data is decomposed element-wise based on the robustness score to obtain the first robust feature data and the first non-robust feature data.

[0046] Optionally, training feature data is obtained based on the first robust feature data and the first non-robust feature data, including:

[0047] The first non-robust feature data is compressed to obtain the second non-robust feature data;

[0048] The first non-robust feature data is transformed to obtain the third non-robust feature data;

[0049] The first robust feature data is aligned to obtain the second robust feature data;

[0050] The second non-robust feature data, the third non-robust feature data, and the second robust feature data are processed to obtain training feature data.

[0051] Optionally, the first non-robust feature data is compressed to obtain the second non-robust feature data, including:

[0052] Obtain the weight data of the auxiliary layer of the target object recognition model;

[0053] The weighted data is multiplied by the first non-robust feature data to obtain the second non-robust feature data.

[0054] Optionally, the first non-robust feature data is transformed to obtain the third non-robust feature data, including:

[0055] Based on the training feature data, the mask data is obtained;

[0056] Through formula f r+ = (1-mask)×f nr The third non-robust feature data is obtained; where f r+ The third non-robust feature data, mask is the mask data, f nr This is the first non-robust feature data.

[0057] Optionally, the first robust feature data is aligned to obtain the second robust feature data, including:

[0058] The first robust feature data is subjected to global average pooling to obtain a representation vector;

[0059] The channel dimension of the representation vector is normalized to obtain the feature similarity matrix;

[0060] Based on the feature similarity matrix, determine the difference data of the similarity matrix;

[0061] Based on the difference data, the second robust feature data is obtained.

[0062] Optionally, the feature dataset is input into a target object recognition model for processing to obtain the recognition result of the target object in the image data, including:

[0063] The feature dataset is input into the target object recognition model for processing to obtain feature vectors;

[0064] The feature vectors are compressed, transformed, and aligned to obtain the recognition result of the target object in the image data.

[0065] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the above-described method.

[0066] The technical solution of the present invention has at least the following effects:

[0067] The present invention provides a method for identifying target objects in an autonomous driving scenario, comprising: acquiring image data in an autonomous driving scenario; performing feature processing on the image data to obtain a feature dataset; inputting the feature dataset into a target object recognition model for processing to obtain the recognition result of the target object in the image data; wherein, the target object recognition model inputs the training dataset into the feature separation layer of a preset network model for feature separation processing to obtain first robust feature data and first non-robust feature data, and obtains training feature data based on the first robust feature data and the first non-robust feature data; and inputs the training feature data into the target object recognition layer of the preset network model for training. The scheme proposed in this invention can significantly improve the performance of the model with less computation, facilitating end-to-end deployment and application in autonomous driving scenarios. Attached Figure Description

[0068] Figure 1 This is a flowchart of a target object recognition method in an autonomous driving scenario provided by an embodiment of the present invention;

[0069] Figure 2 This is a schematic diagram illustrating the feature separation principle of the target object recognition method in the autonomous driving scenario provided in this embodiment of the invention;

[0070] Figure 3 This is a schematic diagram of the dual-stream feature processing framework for the target object recognition method in an autonomous driving scenario provided by an embodiment of the present invention;

[0071] Figure 4 This is a schematic diagram of the feature compression network for the target object recognition method in an autonomous driving scenario provided in this embodiment of the invention;

[0072] Figure 5 This is a schematic diagram of the feature transformation network of the target object recognition method in the autonomous driving scenario provided by the embodiments of the present invention;

[0073] Figure 6 This is a schematic diagram of feature alignment processing in the target object recognition method in an autonomous driving scenario provided by an embodiment of the present invention;

[0074] Figure 7 This is a system processing flowchart of the target object recognition method in an autonomous driving scenario provided by an embodiment of the present invention;

[0075] Figure 8 This is a structural diagram of the target object recognition device in an autonomous driving scenario provided in an embodiment of the present invention;

[0076] Figure 9 This is a schematic diagram of the structure of the computing device provided in an embodiment of the present invention. Detailed Implementation

[0077] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0078] like Figure 1 As shown, embodiments of the present invention propose a method for identifying target objects in an autonomous driving scenario, comprising:

[0079] Step 11: Acquire image data in the autonomous driving scenario;

[0080] Step 12: Perform feature processing on the image data to obtain a feature dataset;

[0081] Step 13: Input the feature dataset into the target object recognition model for processing to obtain the recognition result of the target object in the image data;

[0082] The target object recognition model inputs the training dataset into the feature separation layer of the preset network model for feature separation processing to obtain first robust feature data and first non-robust feature data, and obtains training feature data based on the first robust feature data and the first non-robust feature data; the training feature data is then input into the target object recognition layer of the preset network model for training.

[0083] In this embodiment, image data from actual autonomous driving scenarios is first collected, and then the image data is processed to obtain a feature dataset containing natural samples. The dataset containing natural samples is perturbed or noise is added to obtain a feature dataset containing adversarial samples. The feature dataset is then input into a target object recognition model for processing to obtain the recognition result of the target object in the image data. This robust enhancement method can improve the performance of the model with less computation, making it convenient for end-to-end deployment and application in autonomous driving scenarios.

[0084] In an optional embodiment of the present invention, step 13 may include:

[0085] Step 131: Input the feature dataset into the target object recognition model for processing to obtain feature vectors;

[0086] Step 132: Compress, transform, and align the feature vector to obtain the recognition result of the target object in the image data.

[0087] In this embodiment, the feature dataset is first input into the target object recognition model for processing to obtain feature vectors; the feature vectors are then compressed, transformed, and aligned to obtain the recognition result of the target object in the image data.

[0088] In an optional embodiment of the present invention, the training process of the input target object recognition model may include:

[0089] Step 1311: Input the training dataset into the feature separation layer of the preset network model for feature separation processing to obtain intermediate layer feature data;

[0090] Step 1312: Decompose the intermediate layer feature data element-wise based on the robustness score to obtain the first robust feature data and the first non-robust feature data.

[0091] Step 1313: Compress the first non-robust feature data to obtain the second non-robust feature data;

[0092] Step 1314: The first non-robust feature data is transformed to obtain the third non-robust feature data;

[0093] Step 1315: Align the first robust feature data to obtain the second robust feature data;

[0094] Step 1316: Process the second non-robust feature data, the third non-robust feature data, and the second robust feature data to obtain training feature data;

[0095] Step 1317: Input the training feature data into the target object recognition layer of the preset network model to train and obtain the input target object recognition model.

[0096] In this embodiment, a feature separation network is first trained for the features of any intermediate layer of the deep neural network. A feature separation loss is added to the loss function, and the intermediate layer features are decomposed element-wise into robust features f and non-robust features f based on the robustness score. nr The process of compressing the first non-robust feature data to obtain the second non-robust feature data includes: obtaining the weight data of the auxiliary layer of the target object recognition model; performing matrix multiplication of the weight data and the first non-robust feature data to obtain the second non-robust feature data; and transforming the first non-robust feature data to obtain the third non-robust feature data, including: obtaining mask data based on the training feature data; and using formula f... r+ = (1-mask)×f nr The third non-robust feature data is obtained; where f r+ The third non-robust feature data, mask is the mask data, fnr The process involves: firstly, non-robust feature data; aligning the first robust feature data to obtain second robust feature data, including: performing global average pooling on the first robust feature data to obtain a representation vector; normalizing the channel dimension of the representation vector to obtain a feature similarity matrix; determining the difference data of the similarity matrix based on the feature similarity matrix; obtaining the second robust feature data based on the difference data; processing the second non-robust feature data, the third non-robust feature data, and the second robust feature data to obtain training feature data, including: performing global average pooling on the first robust feature data to obtain a representation vector; normalizing the channel dimension of the representation vector to obtain a feature similarity matrix; determining the difference data of the similarity matrix based on the feature similarity matrix; obtaining the second robust feature data based on the difference data; performing weighted summation on the second non-robust feature data, the third non-robust feature data, and the second robust feature data to obtain training feature data; and inputting the training feature data into the target object recognition layer of a preset network model for training to obtain an input target object recognition model.

[0097] A specific embodiment of the target object recognition method in an autonomous driving scenario provided by this invention is as follows:

[0098] First, the training dataset is input into a deep neural network. A feature separation network is trained for the features of any intermediate layer of the deep neural network. A feature separation loss is added to the loss function, decomposing the intermediate layer features element-wise into first robust feature data f based on the robustness score. r and the first non-robust feature data f nr ,like Figure 2 As shown.

[0099] Based on robustness scoring, the feature vector of any intermediate layer of the model is decomposed into robust and non-robust features element-wise. One method to achieve this goal is to apply a binary mask generated based on the robustness score to the feature map. To ensure the differentiability and continuity of the process of separating non-robust features, a binary mask is obtained using Gumbel-softmax, with the following formula:

[0100]

[0101] Where r is the robustness mapping, σ is the sigmoid function used to normalize the robustness mapping, g1 and g2 are samples extracted from the Gumbel distribution, and τ is a hyperparameter used to control the influence of g1 and g2. To facilitate the randomness of the extraction, g1 and g2 are fixed at -log(-log(u c )), where u c =RC×H×W The expected value is a uniformly distributed value.

[0102] It should be noted that Gumbel-softmax is a proprietary name, referring to a reparameterization technique, which can also be called Gumbel softmax.

[0103] Definition: Robust features Non-robust features It can be assumed that f r The correct classification of the model is positively correlated with f. nr It is negatively correlated with the model's correct classification.

[0104] Design a function based on f r f nr auxiliary layer as input in Composed of an average pooling layer and a fully connected layer, it can output predicted scores p corresponding to two types of features. + and p - Using the cross-entropy loss function L sep Training the separate network loss:

[0105]

[0106] Where N is the number of categories, y is the true label, and y′ is the label value with the highest score in the non-maximum prediction. Since f r positively correlated with the true label y, f nr It is negatively correlated with the true label y, therefore f r The separation loss is y·log(y + ), f nr The separation loss is y′·log(y - By training the auxiliary layer Obtain f related to the model's correct prediction r After being attacked, the model is prone to making incorrect predictions. nr .

[0107] Design a dual-stream feature processing framework to process robust features f separately. r Non-robust features f nr Perform separate processing, such as Figure 3 As shown.

[0108] f nr After processing by the feature compression network (P), it becomes the second non-robust feature data f. nr * First, it retains and fully utilizes the useful discriminative information in non-robust features, ensuring that these features still provide useful clues for correct model predictions even under adversarial attacks. Second, it uses a feature transformation network to transform the non-robust features fnr Transform into third non-robust feature data f r+ This expands the range of robust features that aid model decision-making; through robust feature alignment operations, it reduces the f-value of adversarial examples. r f with natural samples r Perform alignment processing and output the aligned second robust feature data f. r * To enhance robustness, a robust sample alignment loss is added, further improving the model's robustness and accuracy. Finally, the intermediate layer training feature data of the model after the dual-stream feature processing framework is used.

[0109] To avoid the accumulation of adversarial perturbations to the model in subsequent convolutional layers, f nr The feature vectors in the last layer of the neural network are compressed to retain useful information in non-robust features, while compressing the feature space that is vulnerable to attacks, thereby improving the model's classification robustness. This includes the following sub-steps:

[0110] Step 21, to extract f nr Based on information useful for model training and inference, a feature compression network P was designed to obtain f. nr The compression coefficient of each element in the feature vector of the last layer of the neural network, such as Figure 4 As shown. The method for obtaining this coefficient is to introduce an auxiliary MLP layer on the last layer of the neural network, using the weights W obtained from the MLP layer. n As a calculation f nr The magnitude of the correlation with the correct prediction result.

[0111] Step 22, place the W n with f nr Perform matrix multiplication to obtain the compressed feature f. nr * To prevent overfitting, the selection of n during training does not use the label value, but rather the predicted value. A feature compression loss is added, emphasizing f... nr This provides useful information for the model training and inference process.

[0112]

[0113] Among them, W n The compression factor is obtained by introducing an auxiliary MLP layer on the last layer and calculating f. nr l The magnitude of the correlation between the prediction results, This is for matrix multiplication. For an N-class classification task, the W = [w1, w2, ..., w] obtained during the training of the MLP layer is... n ]∈RN To prevent overfitting during MLP layer training, the following is used: The chosen value of n. The cross-entropy loss L for training the MLP layer. comp Recorded as:

[0114]

[0115] in, It is the MLP layer that uses f nr The predicted maximum value, y i The labels are the true class labels. Using a non-robust feature compression network can prevent the discarding of non-robust features, thus effectively utilizing the discriminative information they contain.

[0116] To broaden the range of robust features and thus improve the classification accuracy of the model, a feature transformation network T is introduced, such as... Figure 5 As shown. f nr Through two-dimensional convolution processing, two-dimensional batch normalization processing, and nonlinear activation processing, it is transformed into a new robust feature, including the following sub-steps:

[0117] Step 31: Apply the obtained mask to the feature transformation network, and then... nr Perform a matrix multiplication operation with 1-mask to obtain the new robust feature f. r+ The formula is:

[0118]

[0119] Step 32, in order to achieve the goal of transforming non-robust features back into robust features, thereby increasing the coverage of robust features and helping the model make correct decisions, in f nr Then add an auxiliary layer. Add feature transformation loss and calculate transformation loss L. tranf as follows:

[0120]

[0121] in, For f nr The prediction score of the T network as input, after passing through the T network, is transformed into f. r+ This still provides clues for the model's correct prediction, therefore y i The network is trained using cross-entropy loss to obtain the true class labels. The original robust features f are then added element-wise. r and the transformed robust features f r+ The new robust feature is obtained as f r +f r+ .

[0122] To address the opposing trends in robustness and accuracy, and to achieve a better trade-off between model robustness and natural sample detection accuracy in adversarial training, this study focuses on f. r Alignment operations were performed to make the robust features of adversarial examples more closely resemble the distribution of robust features of natural examples. For example... Figure 6 As shown, to make the robust feature distribution of adversarial examples approximate the robust feature distribution of natural examples, the following sub-steps are included:

[0123] Step 41: Input adversarial examples and natural samples into the model simultaneously. Extract robust feature components of the adversarial examples and natural samples from the feature space respectively, and penalize the difference between the two feature matrices. This allows the model to learn consistent latent feature representations, thereby improving the model's robustness and accuracy. The robust feature representation of natural samples is f. r (x), the robust features of the adversarial example are represented as f r (x adv To improve computational efficiency, the robust feature f is first analyzed. r (x), f r (x adv Perform global average pooling to obtain the representation vector:

[0124]

[0125]

[0126] Where c is the number of channels. To improve the stability of network training and the convenience of similarity calculation, F... c (x), F c (x adv The channel dimension is L2 normalized to obtain the normalized representation:

[0127]

[0128]

[0129] Step 42, calculate the feature similarity matrices M(x) and M(x) for natural samples and adversarial samples respectively. adv ):

[0130] M(x) = Q c (x)·Q c (x) T

[0131] M(x adv ) = Q c (x adv )·Q c (x adv ) T

[0132] Where M(x) and M(x) adv All are symmetric matrices. To penalize the difference between the similarity matrices of the two classes of samples, the cross-entropy loss function of the feature alignment operation g is defined as follows:

[0133]

[0134] Where B is the input batch size.

[0135] The feature output of the (L+1)th layer in the neural network after the above steps is denoted as f. new :

[0136]

[0137] This method can work in any intermediate layer of a deep neural network, where the g operation is for processing the robust feature portion of the intermediate layer features, and the T and P networks are for processing the non-robust feature portion of the intermediate layer features.

[0138] This method can be combined with standard adversarial training strategies, and the loss function can be expressed as:

[0139]

[0140] Where r is the training loss for the robust feature part, nr is the training loss for the non-robust feature part, s is the separation loss for sample features, and α, β, and δ are the control losses for L. align L tranf L comp The hyperparameter of the weight, L cls This represents the classification loss from normal adversarial training.

[0141] This embodiment also proposes an evaluation metric for quantifying the trade-off between robustness and accuracy: Defense Efficiency (DEM). With numerous robustness enhancement methods available, comparing different defense methods under the same perturbation norm, attack step size, and attack method is not an easy task, as these methods differ in robustness and accuracy metrics. The evaluation metric proposed in this embodiment, Defense Efficiency (DEM), quantifies the numerical values ​​of different robustness and accuracy into a unified standard, facilitating the comparison of the effectiveness of different methods in weighing the trade-offs under the same attack and attack intensity. The formula is as follows:

[0142]

[0143] in, Compared to the standard model, represents the percentage increase in model robustness. The percentage decrease in accuracy of the corresponding model on clean samples compared to the ordinary model. An effective strategy to counter adversarial attacks should satisfy the following condition: the increase in model robustness should significantly outweigh the decrease in accuracy on natural samples. This is because if a method to improve robustness comes at the cost of significant accuracy loss, it is considered unacceptable and will be abandoned in practical applications. Therefore, this embodiment designs a logarithmic function to narrow the range between data points, reducing the impact of extreme values ​​on the overall evaluation.

[0144] This embodiment also designs an intuitive comparison method: using the size of the face enclosed by the DEM under different attacks as a method to measure the effectiveness of the trade-off between robustness and accuracy of different methods under attacks.

[0145] The coordinate axes represent different attack methods, and the points on the axes represent the Data Image (DEM) of different robustness enhancement methods corresponding to each attack. Under different attack methods, the area enclosed by the DEM is used to measure the effectiveness of the trade-off between robustness and accuracy in the defense method. The larger the coverage area enclosed by the DEM, the better the trade-off between robustness and accuracy, i.e., the higher the effectiveness of the trade-off. This indicates that the model can maintain high classification accuracy while resisting various attacks, and also shows strong robustness. When adding other attacks and robustness enhancement methods, simply add coordinate axes and use the corresponding DEM points on the coordinate axes to enclose a new graph to conveniently and intuitively display the effectiveness of the trade-off between different methods. The processing flow of this method is as follows: Figure 7 As shown.

[0146] The present invention provides a method for identifying target objects in an autonomous driving scenario, which has the following features:

[0147] Beneficial effects:

[0148] (1) This invention proposes an easy-to-plug-in method that can work at any layer of a deep neural network model without changing the original model. The entire model is trained in an end-to-end manner and can be easily combined with other robust enhancement methods to improve the robustness and accuracy of image classification in autonomous driving scenarios.

[0149] (2) The solution proposed in this invention can significantly improve the performance of the model with less computation, making it convenient for end-to-end deployment and application in autonomous driving scenarios;

[0150] (3) This invention proposes an evaluation metric that quantifies the trade-off between robustness and accuracy, and provides a convenient and intuitive method for evaluating the effectiveness of the trade-off.

[0151] like Figure 8As shown, this embodiment of the invention also provides a target object recognition device 80 in an autonomous driving scenario, the device 80 comprising:

[0152] The acquisition module 81 is used to acquire image data in autonomous driving scenarios;

[0153] Processing module 82 is used to perform feature processing on the image data to obtain a feature dataset;

[0154] The generation module 83 is used to input the feature dataset into the target object recognition model for processing to obtain the recognition result of the target object in the image data;

[0155] The target object recognition model inputs the training dataset into the feature separation layer of the preset network model for feature separation processing to obtain first robust feature data and first non-robust feature data, and obtains training feature data based on the first robust feature data and the first non-robust feature data; the training feature data is then input into the target object recognition layer of the preset network model for training.

[0156] Optionally, the image data is processed to obtain a feature dataset, including:

[0157] The image data is processed to obtain a feature dataset containing natural samples;

[0158] The dataset containing natural samples is perturbed or noise is added to obtain a feature dataset of adversarial samples.

[0159] Optionally, the training dataset is input into the feature separation layer of a predefined network model for feature separation processing to obtain first robust feature data and first non-robust feature data, including:

[0160] The training dataset is input into the feature separation layer of the preset network model for feature separation processing to obtain intermediate layer feature data.

[0161] The intermediate layer feature data is decomposed element-wise based on the robustness score to obtain the first robust feature data and the first non-robust feature data.

[0162] Optionally, training feature data is obtained based on the first robust feature data and the first non-robust feature data, including:

[0163] The first non-robust feature data is compressed to obtain the second non-robust feature data;

[0164] The first non-robust feature data is transformed to obtain the third non-robust feature data;

[0165] The first robust feature data is aligned to obtain the second robust feature data;

[0166] The second non-robust feature data, the third non-robust feature data, and the second robust feature data are processed to obtain training feature data.

[0167] Optionally, the first non-robust feature data is compressed to obtain the second non-robust feature data, including:

[0168] Obtain the weight data of the auxiliary layer of the target object recognition model;

[0169] The weighted data is multiplied by the first non-robust feature data to obtain the second non-robust feature data.

[0170] Optionally, the first non-robust feature data is transformed to obtain the third non-robust feature data, including:

[0171] Based on the training feature data, the mask data is obtained;

[0172] Through formula f r+ = (1-mask)×f nr The third non-robust feature data is obtained; where f r+ The third non-robust feature data, mask is the mask data, f nr This is the first non-robust feature data.

[0173] Optionally, the first robust feature data is aligned to obtain the second robust feature data, including:

[0174] The first robust feature data is subjected to global average pooling to obtain a representation vector;

[0175] The channel dimension of the representation vector is normalized to obtain the feature similarity matrix;

[0176] Based on the feature similarity matrix, determine the difference data of the similarity matrix;

[0177] Based on the difference data, the second robust feature data is obtained.

[0178] Optionally, the feature dataset is input into a target object recognition model for processing to obtain the recognition result of the target object in the image data, including:

[0179] The feature dataset is input into the target object recognition model for processing to obtain feature vectors;

[0180] The feature vectors are compressed, transformed, and aligned to obtain the recognition result of the target object in the image data.

[0181] It should be noted that this device is a device corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0182] like Figure 9 As shown, this embodiment of the invention also provides a computing device 90, including a processor 91, a memory 92, and a program or instructions stored in the memory 92 and executable on the processor 91. When the program or instructions are executed by the processor 91, they implement the various processes of the above-described virtual object synchronization information processing method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here. It should be noted that the computing device in this embodiment of the invention includes the above-described mobile electronic devices and non-mobile electronic devices.

[0183] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0184] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0185] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0186] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0188] The data is stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0189] Furthermore, it should be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent solutions of the present invention. Moreover, the steps performing the above-described series of processes can naturally be executed in the order described, but are not necessarily required to be executed in chronological order; some steps can be executed in parallel or independently of each other. Those skilled in the art will understand that all or any step or component of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or network of computing devices, in hardware, firmware, software, or a combination thereof. This is something that those skilled in the art can achieve by using their basic programming skills after reading the description of the present invention.

[0190] Therefore, the object of the present invention can also be achieved by running a program or a set of programs on any computing device. The computing device can be a known general-purpose device. Therefore, the object of the present invention can also be achieved simply by providing a program product containing program code for implementing the method or apparatus. That is, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any known storage medium or any storage medium developed in the future. It should also be noted that in the apparatus and method of the present invention, it is obvious that the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered equivalent to the present invention. Furthermore, the steps for performing the above series of processes can naturally be performed in the order described, but are not necessarily required to be performed in chronological order. Some steps can be performed in parallel or independently of each other.

[0191] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for identifying target objects in an autonomous driving scenario, characterized in that, include: Acquire image data in autonomous driving scenarios; The image data is subjected to feature processing to obtain a feature dataset; The feature dataset is input into the target object recognition model for processing to obtain the recognition result of the target object in the image data; The target object recognition model inputs the training dataset into the feature separation layer of a preset network model for feature separation processing to obtain first robust feature data and first non-robust feature data. Based on the first robust feature data and the first non-robust feature data, training feature data is obtained. The training feature data is then input into the target object recognition layer of the preset network model for training. Specifically, the training dataset is input into the feature separation layer of a pre-defined network model for feature separation processing, resulting in first robust feature data and first non-robust feature data, including: The training dataset is input into the feature separation layer of the preset network model for feature separation processing to obtain intermediate layer feature data. The intermediate layer feature data is decomposed element-wise based on the robustness score to obtain the first robust feature data and the first non-robust feature data. The training feature data is obtained based on the first robust feature data and the first non-robust feature data, including: The first non-robust feature data is compressed to obtain the second non-robust feature data; The first non-robust feature data is transformed to obtain the third non-robust feature data; The first robust feature data is aligned to obtain the second robust feature data; The second non-robust feature data, the third non-robust feature data, and the second robust feature data are processed to obtain training feature data; The first non-robust feature data is compressed to obtain the second non-robust feature data, including: Obtain the weight data of the auxiliary layer of the target object recognition model; The weighted data is multiplied by the first non-robust feature data to obtain the second non-robust feature data. The first non-robust feature data is transformed to obtain the third non-robust feature data, including: Based on the training feature data, the mask data is obtained; Through formula f r+ =(1- mask )× f nr The third non-robust feature data is obtained; among them, f r+ This is the third non-robust feature data. mask For mask data, f nr This is the first non-robust feature data; The first robust feature data is aligned to obtain the second robust feature data, which includes: The first robust feature data is subjected to global average pooling to obtain a representation vector; The channel dimension of the representation vector is normalized to obtain the feature similarity matrix; Based on the feature similarity matrix, determine the difference data of the similarity matrix; Based on the difference data, the second robust feature data is obtained.

2. The method for identifying target objects in an autonomous driving scenario according to claim 1, characterized in that, The image data is processed to obtain a feature dataset, including: The image data is processed to obtain a feature dataset containing natural samples; The dataset containing natural samples is perturbed or noise is added to obtain a feature dataset of adversarial samples.

3. The method for identifying target objects in an autonomous driving scenario according to claim 1 or 2, characterized in that, The feature dataset is input into a target object recognition model for processing to obtain the recognition result of the target object in the image data, including: The feature dataset is input into the target object recognition model for processing to obtain feature vectors; The feature vectors are compressed, transformed, and aligned to obtain the recognition result of the target object in the image data.

4. A target object recognition device for autonomous driving scenarios, characterized in that, The apparatus is used to implement the method as described in any one of claims 1 to 3, the apparatus comprising: The acquisition module is used to acquire image data in autonomous driving scenarios; The processing module is used to perform feature processing on the image data to obtain a feature dataset; The generation module is used to input the feature dataset into the target object recognition model for processing, and obtain the recognition result of the target object in the image data; The target object recognition model inputs the training dataset into the feature separation layer of the preset network model for feature separation processing to obtain first robust feature data and first non-robust feature data, and obtains training feature data based on the first robust feature data and the first non-robust feature data; the training feature data is then input into the target object recognition layer of the preset network model for training.

5. A computing device, characterized in that, include: A processor, a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method as described in any one of claims 1 to 3.

Images