3D object unsupervised anomaly detection method and system, storage medium and terminal

Abstract

The invention discloses a 3D object unsupervised anomaly detection method and system, a storage medium and a terminal, and belongs to the field of 3D object detection.The method mainly comprises the steps that an auto-encoder is constructed, a memory enhancement module is introduced into the auto-encoder, and the memory enhancement module is used for storing normal sample features; training an auto-encoder, extracting normal sample features through the auto-encoder, and storing the normal sample features in a memory enhancement module; and performing anomaly detection by using the trained auto-encoder, querying normal sample features most similar to the features of the scene sample to be detected in the memory enhancement module, connecting the normal sample features with the features of the scene sample to be detected, and reconstructing an original image through a decoder. According to the method, the defects of an existing 3D object anomaly detection method are overcome, an existing auto-encoder is improved, the reconstruction error is small, the calculation complexity is low, and meanwhile the detection model has high adaptability.

Description

Technical Field

[0001] The present invention relates to the field of 3D object detection, and in particular to unsupervised anomaly detection of 3D objects. Background Art

[0002] As 3D object detection becomes increasingly important in areas such as autonomous driving and robotics, accurate 3D object detection is crucial for safe driving of vehicles. However, these models are vulnerable to adversarial attacks, where specific modifications to the input data cause the model to output incorrect results. For example, despite the improvement in accuracy and efficiency achieved by AdvMono3D, monocular 3D object detection models often fail when faced with attacks, making them ineffective. Therefore, enhancing the adversarial robustness of 3D detection models becomes a key issue.

[0003] Adversarial attacks can cause anomalies in the input data of 3D object detection models, which may manifest as changes in data distribution, distortion of features, etc. Anomaly detection methods can identify these anomalies to determine whether an adversarial attack has occurred. For example, when an adversarial attack interferes with the depth information of a 3D object, anomaly detection can detect this depth anomaly that does not conform to the laws of normal data. By detecting and preventing adversarial attacks, the safety of 3D object detection models in practical applications can be improved. In autonomous driving scenarios, safety is the primary consideration, and accurate detection and prevention of adversarial attacks are of great significance to ensuring driving safety.

[0004] In recent years, there have been many research results on adversarial defense strategies for 3D point cloud recognition and 3D object detection models, but due to the large scale of scenes and sparse points, the improvement of adversarial robustness in the field of 3D object detection is still limited. Traditional anomaly detection methods are developed from rules, state estimation and statistical models, which are sensitive to noise in data. When the input space changes, it may take a long time to update the model.

[0005] With the popularization of deep learning technology, anomaly detection and autoencoders based on deep learning have been introduced into anomaly detection tasks. The autoencoder-based method learns the representation of normal samples by training the autoencoder model, and then inputs the adversarial samples into the autoencoder for reconstruction. Normal samples can be well restored after reconstruction by the autoencoder, while adversarial samples will have large deviations during reconstruction due to their differences from normal samples, so as to detect adversarial attacks. Although this method has the powerful function of reconstructing samples in an unsupervised manner, it faces certain challenges in the process of reconstructing the original scene. For example, when the attack method changes, new feature patterns may be generated. When new adversarial samples are input, the original autoencoder model may not be able to effectively identify the new adversarial samples, and the autoencoder model needs to be retrained. The computational complexity is high, and the reconstructed abnormal samples often have large reconstruction errors and cannot restore the original scene well. In large-scale and complex driving environments, its adaptability is not high. Summary of the invention

[0006] The purpose of the present invention is to overcome the defects of the existing 3D object anomaly detection method, improve the existing autoencoder, and solve the problems of large reconstruction error, high computational complexity and insufficient adaptability.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] In a first aspect, a 3D object unsupervised anomaly detection method is provided, comprising:

[0009] Construct an autoencoder, and introduce a memory enhancement module into the autoencoder, wherein the memory enhancement module is used to store normal sample features;

[0010] Train the autoencoder, extract the normal sample features through the autoencoder and store them in the memory enhancement module;

[0011] The trained autoencoder is used for anomaly detection. The normal sample features that are most similar to the features of the scene samples to be detected are queried in the memory enhancement module. The normal sample features are connected with the features of the scene samples to be detected and the original image is reconstructed through the decoder.

[0012] In some embodiments, extracting normal sample features through an autoencoder and storing them in a memory enhancement module includes:

[0013] If the space of the memory enhancement module is full, the similarity between the newly extracted normal sample features and all the normal sample features in the memory enhancement module is calculated; if the similarity loss of the newly extracted normal sample features is less than the maximum similarity loss in the memory enhancement module, the stored normal sample features corresponding to the maximum similarity loss are replaced.

[0014] In some embodiments, querying the memory enhancement module for the normal sample features that are most similar to the features of the scene sample to be detected includes:

[0015] Calculate the similarity between each feature vector in the feature graph of the scene sample to be detected and each feature vector in the memory enhancement module;

[0016] A probability weight is calculated according to the similarity calculation result, and the most similar normal sample feature is retrieved according to the probability weight.

[0017] Preferably, the similarity between each feature vector in the feature map of the scene sample to be detected and each feature vector in the memory enhancement module is calculated by cosine similarity and Hausdorff distance.

[0018] Preferably, the calculating the probability weight according to the similarity calculation result includes:

[0019] The softmax function is used to convert the results of cosine similarity and Hausdorff distance into corresponding probability weights.

[0020] Preferably, the comprehensive anomaly detection loss function of the autoencoder combines reconstruction loss, feature similarity loss and sparsity loss, and balances the losses through hyperparameters.

[0021] In some embodiments, it also includes:

[0022] Ablation experiments are performed on different components in the memory enhancement module.

[0023] In some embodiments, the method preprocesses the data before using the autoencoder for detection, and the preprocessing includes:

[0024] Convert the 3D point cloud scene generated by LiDAR into a bird's eye view (BEV) format.

[0025] In a second aspect, a 3D object unsupervised anomaly detection system is provided, comprising:

[0026] An autoencoder construction module is configured to construct an autoencoder, and introduce a memory enhancement module into the autoencoder, wherein the memory enhancement module is used to store normal sample features;

[0027] An autoencoder training module, configured to train an autoencoder, extract normal sample features through the autoencoder and store them in the memory enhancement module;

[0028] The anomaly detection module is configured to perform anomaly detection using the trained autoencoder, query the normal sample features that are most similar to the features of the scene sample to be detected in the memory enhancement module, connect the normal sample features with the features of the scene sample to be detected, and reconstruct the original image through the decoder.

[0029] In a third aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for unsupervised anomaly detection of 3D objects described in any embodiment is implemented.

[0030] In a fourth aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and wherein the processor executes the 3D object unsupervised anomaly detection method described in any embodiment when running the computer instructions.

[0031] It should be further explained that the technical features corresponding to the above embodiments can be combined or replaced with each other to form a new technical solution without conflict.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] (1) The present invention uses a deep encoder combined with a memory enhancement module to detect anomalies, wherein a large number of normal sample features are stored in the memory enhancement module. These normal sample features represent the normal distribution of the data, so that the model can effectively capture and store the feature distribution of normal samples and use them to detect abnormal samples that deviate from this distribution, that is, the memory enhancement module enhances the feature representation ability of the autoencoder, so that it can capture richer normal sample patterns. This enhanced feature representation helps to more accurately distinguish normal samples from abnormal samples, thereby reducing the reconstruction error of abnormal samples. During the detection of the present invention, by retrieving the normal sample features that are most similar to the input features in the memory enhancement module, the autoencoder can use this information to guide the reconstruction process, thereby reducing the impact of the abnormal part in the adversarial sample (the scene sample to be detected) on the reconstruction result, improving the accuracy of the reconstruction, and helping the reconstruction process to more accurately restore the original scene. At the same time, the memory enhancement module can adapt to new normal sample features and update its content during the training process. This adaptability enables the model to generalize to unseen data (such as data containing slight anomalies or disturbances), improves the adaptability of the model, and reduces the computational complexity when facing different adversarial samples.

[0034] (2) In one example, the present invention converts LiDAR point cloud data into a BEV map and then performs anomaly detection. There is no need to use labeled abnormal samples for training, which reduces the dependence on labeled data and avoids the time consumption and cost overhead caused by labeling abnormal samples.

[0035] (3) In one example, when the space of the memory enhancement module of the present invention is full, if the similarity loss of the newly extracted normal sample feature is less than the maximum similarity loss in the memory enhancement module, the stored normal sample feature corresponding to the maximum similarity loss is replaced, so that the memory enhancement module can adapt to the new normal sample feature and distinguish normal samples from abnormal samples more accurately.

[0036] (4) In one example, ablation experiments are performed on different components in the memory enhancement module to evaluate the impact of the components on the model performance. By changing the combination of these components and observing the changes in model performance, it is possible to determine which components are most critical to improving anomaly detection performance, providing a reference for subsequent model adjustment and optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A flow chart of a 3D object unsupervised anomaly detection method according to an embodiment of the present invention.

[0038] Figure 2 The figure is a schematic diagram of the decoder structure according to an embodiment of the present invention.

[0039] Figure 3 This is a visualization scene of the reconstruction errors of normal samples and adversarial samples shown in an embodiment of the present invention.

[0040] Figure 4 The effect of the memory enhancement module size on the model performance AUC shown in the embodiment of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention is clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various configurations. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0042] It should be noted that the defects existing in the solutions in the above-mentioned prior art are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above-mentioned problems and the solutions proposed in the embodiments of the present application for the above-mentioned problems below should all be the contributions made by the inventor to the present application in the process of invention and creation, and should not be understood as technical contents known to technical personnel in this field.

[0043] In view of the technical problems pointed out in the background technology, the embodiments provided by the present invention are as follows:

[0044] Example 1

[0045] Reference Figure 1 In an exemplary embodiment, taking radar scene anomaly detection based on deep learning as an example, a 3D object unsupervised anomaly detection method is provided, including:

[0046] Construct an autoencoder, and introduce a memory enhancement module into the autoencoder, wherein the memory enhancement module is used to store normal sample features;

[0047] Train the autoencoder, extract the normal sample features through the autoencoder and store them in the memory enhancement module;

[0048] The trained autoencoder is used for anomaly detection. The normal sample features that are most similar to the features of the scene samples to be detected are queried in the memory enhancement module. The normal sample features are connected with the features of the scene samples to be detected and the original image is reconstructed through the decoder.

[0049] The present invention first defines an anomaly detection formula, and uses an autoencoder to reveal abnormal samples through unsupervised learning. P represents a benign radar scene, and P' represents an adversarial sample generated from P. AE(P) and AE(P') represent the reconstructed outputs of P and P' generated by the autoencoder AE, respectively, and Dis(,) represents the distance measure between the original scene and the scene reconstructed by the autoencoder. The target of the autoencoder is defined as follows:

[0050] Dis(AE(P′),P′)≥ξs.t.Dis(AE(P),(P)

[0051] where ξs.t. is the threshold of reconstruction error used to distinguish normal samples from adversarial samples.

[0052] The 3D point cloud scene generated by the laser radar (LiDAR) is converted into a bird's-eye view (BEV) form. Secondly, an autoencoder is constructed to reconstruct the input LiDAR scene. The design of the autoencoder consists of two main parts: the encoder maps the input scene to a low-dimensional feature space, while the decoder restores the information in the feature space to the original scene. By comparing the reconstruction result with the original input, anomalies can be detected. The specific implementation is as follows Figure 2 As shown in Figure 2. The encoder maps the input image to a latent space, represented by a feature map X. Each feature f of the feature map X (in this case, the feature map of the normal sample) will become a query item of the memory enhancement module to facilitate the retrieval of the relevant normal sample features in the memory enhancement module. In addition, the autoencoder updates the normal sample features as memory items of the memory enhancement module during training. is the reconstructed feature vector of f, which is composed of all the feature items of the memory enhancement module. The feature map X (in this case, the feature map corresponding to the scene sample to be detected) is obtained by The addressing process retrieves its similar feature map The decoder compares the feature map X with the retrieved similar feature maps Concatenate to reconstruct the image.

[0053] Furthermore, the memory enhancement module is designed to store the feature representation of normal samples, which are extracted from the benign dataset by the encoder. N×C Represented as, where N is the size of the memory enhancement module, that is, the number of features stored, and C is the number of channels for each memory feature.

[0054] Secondly, feature storage and updating are performed. During the training phase of the autoencoder, normal samples are extracted as features by the encoder. After the feature vector f of the BEV image is extracted from the encoder, if there is free space in the memory enhancement module, the feature vector f is directly stored. When the memory enhancement module is full, the feature with the largest similarity loss with the feature vector f is found in the memory enhancement module. At the same time, the maximum similarity loss in the memory enhancement module is calculated. If the similarity loss of the new feature is less than the maximum similarity loss in the memory module, the stored normal sample feature vector corresponding to the maximum similarity loss is found and replaced with the new feature vector f.

[0055] Then feature retrieval is performed. In the detection phase, the features of the input sample (the scene sample to be detected) will query the memory enhancement module to find the most similar normal features. Specifically, the reconstructed feature vector It is based on the query feature vector f and each memory item (feature vector) m in the memory enhancement module i The probability weight ω is calculated based on the similarity between them to retrieve. The similarity calculation uses cosine similarity and Hausdorff distance to calculate the similarity between feature vectors. At the same time, the softmax function is used to convert the results of cosine similarity and Hausdorff distance into corresponding probability weights ω i , used to retrieve the most similar features from the memory module. Probability weight ω i The calculation formula is as follows:

[0056]

[0057] The Hausdorff distance is expressed as:

[0058]

[0059] Among them, m k represents the kth feature vector stored in the memory enhancement module; m ij Represents the i-th feature vector m in the memory enhancement module i The jth element of j Represents the jth element in the query feature vector f.

[0060] Furthermore, the decoder compares the feature map X with the retrieved similar feature maps Connect to reconstruct the image, including:

[0061] 1) Feature retrieval: Each feature vector f in the feature map X output by the encoder is used to retrieve the most similar normal feature in the memory module

[0062] 2) Feature splicing: retrieved feature map Concatenate with the original feature map X along the feature dimension to form a new input

[0063] 3) Image reconstruction: The decoder receives the concatenated feature map and reconstruct the image.

[0064] Furthermore, the comprehensive anomaly detection loss function of the autoencoder in this method combines reconstruction loss, feature similarity loss and sparsity loss, and balances the loss through hyperparameters α and β. Among them, the reconstruction loss uses the L2 norm to define the reconstruction loss to minimize the difference between the input and reconstructed images; the feature similarity loss introduces feature similarity loss to promote the distinguishability of features in the memory module; the sparsity loss ensures the sparsity of feature weights by minimizing entropy, thereby utilizing more memory items.

[0065] Furthermore, the anomaly detection method of the present invention is experimentally verified, which specifically includes the following parts:

[0066] Dataset selection: Three datasets, KITTI, nuScenes, and Waymo, were used for experimental verification. The datasets were divided into training sets and validation sets, and half of the validation set was used to generate adversarial samples. In the process of generating adversarial samples and anomaly detection, due to the differences between detectors and datasets, the experiment only considered the vehicle category and did not include other categories.

[0067] Experimental setup: The proposed anomaly detection algorithm is applied to advanced adversarial attack methods. The adversarial attack methods generate different adversarial samples based on five different detectors, including PIXOR, PointPillars, PointRCNN, PV-RCNN, and CenterPoint. The detector code implementation uses the MMDetection3d platform that integrates advanced 3D object detection models.

[0068] Performance evaluation: Using AUC as the performance evaluation indicator, the AUC performance of the proposed anomaly detection method on the test set containing adversarial samples is reported and compared with existing point cloud-based and LiDAR data-based anomaly detection methods.

[0069] Quantitative Evaluation: AUC is used to quantitatively evaluate the performance of the model and compare it with existing methods.

[0070] Qualitative evaluation: Qualitatively evaluate the effect of the model by visualizing the reconstruction error, showing the difference in reconstruction effect between normal samples and adversarial samples.

[0071] Error visualization: Compare the original BEV image, the reconstructed BEV image, and the reconstruction error between the two.

[0072] Ablation experiment:

[0073] Different components in the memory module (cosine similarity, Hausdorff distance, feature similarity loss L sep , sparsity loss L spa We conduct ablation studies on the components (size N, memory module size N) to evaluate their impact on the model performance. By changing the combination of these components, we observe the change in AUC to determine which components are most critical to improving the anomaly detection performance.

[0074] In summary, the present invention uses a deep encoder combined with a memory enhancement module to detect anomalies, wherein a large number of normal sample features are stored in the memory enhancement module, and these normal sample features represent the normal distribution of the data, so that the model can effectively capture and store the feature distribution of normal samples, and is used to detect abnormal samples that deviate from this distribution, that is, the memory enhancement module enhances the feature representation ability of the autoencoder, so that it can capture a richer normal sample pattern. This enhanced feature representation helps to more accurately distinguish normal samples from abnormal samples, thereby reducing the reconstruction error of abnormal samples. During the detection of the present invention, by retrieving the normal sample features that are most similar to the input features in the memory enhancement module, the autoencoder can use this information to guide the reconstruction process, thereby reducing the impact of the abnormal part in the adversarial sample (scene sample to be detected) on the reconstruction result, improving the accuracy of reconstruction, and helping the reconstruction process to more accurately restore the original scene. At the same time, the memory enhancement module can adapt to new normal sample features and update its content during the training process. This adaptability enables the model to generalize to unseen data (such as data containing slight anomalies or disturbances), improves the adaptability of the model, and reduces the computational complexity when facing different adversarial samples.

[0075] Furthermore, the existing detection methods are mainly aimed at two-dimensional images and videos. This method performs anomaly detection after converting LiDAR point cloud data into BEV images in the early stage. It does not need to use labeled abnormal samples for training, reduces the dependence on labeled data, and avoids the time consumption and cost overhead caused by labeling abnormal samples. At the same time, it expands the anomaly detection task in the field of three-dimensional object detection. This method can be flexibly applied to various 3D detection models, effectively improving the robustness of 3D object detection models in the face of adversarial attacks.

[0076] Example 2

[0077] Based on the inventive concept of Example 1, this example introduces in detail the practical application of the anomaly detection method, including:

[0078] Data preprocessing:

[0079] LiDAR scene conversion: Convert LiDAR point cloud data into BEV images, making it more suitable for 2D image processing techniques by simplifying the processing of point cloud data.

[0080] Region selection: For the KITTI dataset, a range of [-40,40]×[-1,3]×[0,70] meters was selected for BEV image conversion; for the nuScenes and Waymo datasets, a range of [-51.2, 51.2]×[-4,4]×[-51.2, 51.2] was selected. The number of epochs in the training phase was set to 80, and the initial learning rate was set to 2e-5. The hyperparameters α and β were selected by grid search and set to 0.1 during the training phase.

[0081] Model architecture settings:

[0082] Autoencoder structure: A deep autoencoder is constructed, which consists of two parts: encoder and decoder. The encoder maps the input BEV image to the latent feature space, and the decoder reconstructs the image from the feature space.

[0083] Memory enhancement module: A memory enhancement module is introduced in the autoencoder to store and update the features of normal samples. The design of the memory enhancement module allows the model to retrieve the normal features that are most similar to the input sample during the detection phase.

[0084] Model training:

[0085] Normal sample training: Use normal samples in the dataset to train the autoencoder and update the memory enhancement module. During the training process, the features of the normal samples are encoded and stored in the memory enhancement module.

[0086] Feature update: When the memory module is not full, the newly extracted features are directly stored; when the memory module is full, the features in the memory module are updated based on similarity calculation to maintain diversity and discrimination.

[0087] Detection phase:

[0088] Feature retrieval: For a new input sample, the encoder is used to extract features, and then the memory-augmented module is queried to retrieve the most similar normal sample features.

[0089] Feature reconstruction: The retrieved normal sample features are combined with the input features and sent to the decoder to reconstruct the BE V image.

[0090] Reconstruction loss calculation: Calculate the difference between the input BEV image and the reconstructed image as the reconstruction loss. Adversarial examples usually have higher reconstruction losses.

[0091] Loss function and optimization:

[0092] The overall anomaly detection loss function L anomaly Combined with the reconstruction loss L rec , feature similarity loss L sim and the sparsity loss L spa , and balance the loss through parameters α and β.

[0093] Where S and They represent the input BEV scenario and output scenario of the proposed unsupervised anomaly detection, respectively. where ω t Represents f in feature map X t The probability weight of . Then the overall anomaly loss detection function is expressed as:

[0094] L anomaly =L rec +αL sim +βL spa

[0095] Post-processing:

[0096] Anomaly detection: Anomaly detection is performed on samples based on reconstruction loss. Adversarial samples usually have higher reconstruction loss due to their large differences from normal samples, so they can be identified as anomalies.

[0097] Experimental verification:

[0098] Quantitative evaluation: Comparison with several existing anomaly detection methods, including CARLO, M3DM, and CPMF. For each detector in the KITTI, nuScenes, and Waymo datasets, the AUC value of the proposed anomaly detection method is calculated. The AUC values ​​of other baseline methods on the same dataset and victim detector are collected and recorded. Tables 1, 2, and 3 show that the proposed method outperforms existing anomaly detection techniques on all datasets, especially performing well under adversarial attacks.

[0099] Table 1 Comparison results of several anomaly detection methods under KITTI dataset

[0100]

[0101] Table 2 Comparison results of several anomaly detection methods on the nuScenes dataset

[0102]

[0103] Table 3 Comparison results of several anomaly detection methods under Waymo dataset

[0104]

[0105] Qualitative assessment:

[0106] Normal samples and adversarial samples are selected from the KITTI dataset, and PointRCNN is used as the victim detector to generate adversarial samples. The selected normal samples and adversarial samples are converted into BEV images and then reconstructed using the proposed anomaly detection method.

[0107] Error Visualization:

[0108] Figure 3 A visualization of the reconstruction error on normal and adversarial samples is shown.

[0109] Input scene (left): shows the original BEV image, which is what the input scene looks like.

[0110] Reconstructed scene (center): shows the BEV image reconstructed by the autoencoder, that is, the result of the model trying to restore the input scene.

[0111] Reconstruction Error (right): A graph showing the distribution of reconstruction error, where high error areas (red areas) indicate where the difference between the input scene and the reconstructed scene is the largest.

[0112] The input in the first row is a normal scene, and the input in the second and third rows is an adversarial scene.

[0113] Result analysis:

[0114] For normal samples, the reconstruction error is generally small, indicating that the model is able to accurately reconstruct the input scene. For adversarial samples, the reconstruction error is larger, especially in the areas introduced by the adversarial perturbation. This shows that the model is able to detect anomalies in the input scene and try to correct them during the reconstruction process.

[0115] Ablation experiment:

[0116] Experimental design:

[0117] Baseline model: The autoencoder without a memory module is used as the baseline model.

[0118] Component changes: Change the components in the memory module one by one or in combination, for example, not using cosine similarity or Hausdorff distance, not using feature similarity loss or sparsity loss, or changing the size of the memory module.

[0119] Experimental process:

[0120] For each variation, the model was retrained using the same dataset and conditions, and the performance was evaluated on the same test set. The model performance under each variation was recorded, using AUC as the performance metric.

[0121] Experimental results:

[0122] Similarity metrics: Compare the performance of models using only cosine similarity, only Hausdorff distance, and a combination of both.

[0123] Loss function: Analyze the performance changes after using only reconstruction loss, adding feature similarity loss and sparsity loss.

[0124] Memory Module Size: Explore the impact of different memory module sizes on performance, including smaller and larger memory modules.

[0125] Result analysis:

[0126] Table 4 shows the performance of different module combinations in the anomaly detection model. Among all the modules, the memory enhancement module has the greatest impact on the performance. The combination of cosine similarity and Hausdorff distance is more effective in measuring feature similarity. sep and L spa It plays an important role in the construction and selection of memory modules. In addition, the experiment also studies the impact of the memory module size N on the performance. Experiments are conducted on the PointRCNN detector using the KITTI dataset. Figure 4 It can be seen that increasing the size of the memory enhancement module can effectively improve the AUC. The memory enhancement module ultimately represents the probability distribution of a typical normal sample, rather than the situation under the constraint of memory capacity N. When the memory enhancement module is large, increasing N has little effect on performance. However, expanding the memory module will also significantly increase the training time.

[0127] Table 4 Performance comparison of different module combinations in anomaly detection model

[0128]

[0129] Experiments show that this method has significant improvements in detecting adversarial samples, which is better than the most advanced LiDAR data anomaly detection method. At the same time, adversarial attack methods based on deep learning can be detected by anomaly detection methods based on deep learning, which improves the robustness of 3D object detection models against adversarial attacks.

[0130] Example 3

[0131] Based on the same inventive concept as that of Example 1, a 3D object unsupervised anomaly detection system is provided, comprising:

[0132] An autoencoder construction module is configured to construct an autoencoder, and introduce a memory enhancement module into the autoencoder, wherein the memory enhancement module is used to store normal sample features;

[0133] An autoencoder training module, configured to train an autoencoder, extract normal sample features through the autoencoder and store them in the memory enhancement module;

[0134] The anomaly detection module is configured to perform anomaly detection using the trained autoencoder, query the normal sample features that are most similar to the features of the scene sample to be detected in the memory enhancement module, connect the normal sample features with the features of the scene sample to be detected, and reconstruct the original image through the decoder.

[0135] It should be noted that the functions corresponding to each step in the implementation method of each module are not repeated here.

[0136] Example 4

[0137] Based on the same inventive concept as Example 1, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the 3D object unsupervised anomaly detection method provided by the embodiment of the present invention is implemented. Based on such an understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0138] Example 5

[0139] Based on the same inventive concept as Example 1, an electronic device is provided, including a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, and when the processor executes the computer instructions, the 3D object unsupervised anomaly detection method provided in the embodiment of the present invention is executed.

[0140] The processor may be a single-core or multi-core central processing unit or a specific integrated circuit, or one or more integrated circuits configured to implement the present invention.

[0141] Embodiments of the subject matter and functional operations described in this specification may be implemented in: tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or a combination of one or more of them. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier to be executed by a data processing device or to control the operation of the data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode and transmit information to a suitable receiver device for execution by the data processing device.

[0142] The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by special purpose logic circuits, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits), and the apparatus can also be implemented as special purpose logic circuits.

[0143] Processors suitable for executing computer programs include, for example, general and / or special microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from a read-only memory and / or a random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, the computer will also include one or more large-capacity storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or the computer will be operably coupled to this large-capacity storage device to receive data from it or to transmit data to it, or both. However, the computer does not necessarily have such a device. In addition, the computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, just to name a few.

[0144] It should be understood that each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] The above specific implementation methods are detailed descriptions of the present invention. It cannot be determined that the specific implementation methods of the present invention are limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions and substitutions can be made without departing from the concept of the present invention, which should be regarded as belonging to the protection scope of the present invention.

Claims

1. A 3D object unsupervised anomaly detection method, characterized in that: include: Construct an autoencoder, and introduce a memory enhancement module into the autoencoder, wherein the memory enhancement module is used to store normal sample features; Train the autoencoder, extract the normal sample features through the autoencoder and store them in the memory enhancement module; The trained autoencoder is used for anomaly detection. The normal sample features that are most similar to the features of the scene samples to be detected are queried in the memory enhancement module. The normal sample features are connected with the features of the scene samples to be detected and the original image is reconstructed through the decoder.

2. The unsupervised anomaly detection method for 3D objects according to claim 1, characterized in that: The extracting of normal sample features through the autoencoder and storing them in the memory enhancement module includes: If the space of the memory enhancement module is full, the similarity between the newly extracted normal sample features and all the normal sample features in the memory enhancement module is calculated; if the similarity loss of the newly extracted normal sample features is less than the maximum similarity loss in the memory enhancement module, the stored normal sample features corresponding to the maximum similarity loss are replaced.

3. The unsupervised anomaly detection method for 3D objects according to claim 1, characterized in that: The step of searching the memory enhancement module for the normal sample features that are most similar to the features of the scene sample to be detected includes: Calculate the similarity between each feature vector in the feature graph of the scene sample to be detected and each feature vector in the memory enhancement module; A probability weight is calculated according to the similarity calculation result, and the most similar normal sample feature is retrieved according to the probability weight.

4. The 3D object unsupervised anomaly detection method according to claim 3, characterized in that: The similarity between each feature vector in the feature map of the scene sample to be detected and each feature vector in the memory enhancement module is calculated through cosine similarity and Hausdorff distance.

5. The 3D object unsupervised anomaly detection method according to claim 4, characterized in that: The calculating the probability weight according to the similarity calculation result includes: The softmax function is used to convert the results of cosine similarity and Hausdorff distance into corresponding probability weights.

6. The unsupervised anomaly detection method for 3D objects according to claim 5, characterized in that: The comprehensive anomaly detection loss function of the autoencoder combines reconstruction loss, feature similarity loss and sparsity loss, and balances the losses through hyperparameters.

7. The unsupervised anomaly detection method for 3D objects according to claim 1, characterized in that: Also includes: Ablation experiments are performed on different components in the memory enhancement module.

8. A 3D object unsupervised anomaly detection system, characterized in that: include: An autoencoder construction module is configured to construct an autoencoder, and introduce a memory enhancement module into the autoencoder, wherein the memory enhancement module is used to store normal sample features; An autoencoder training module, configured to train an autoencoder, extract normal sample features through the autoencoder and store them in the memory enhancement module; The anomaly detection module is configured to perform anomaly detection using the trained autoencoder, query the normal sample features that are most similar to the features of the scene sample to be detected in the memory enhancement module, connect the normal sample features with the features of the scene sample to be detected, and reconstruct the original image through the decoder.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the 3D object unsupervised anomaly detection method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, wherein: When the processor runs the computer instructions, it executes the 3D object unsupervised anomaly detection method described in any one of claims 1-7.

Images