Incremental learning-oriented out-of-distribution image detection method and system

Abstract

The invention relates to an incremental learning-oriented out-of-distribution image detection method and system, the system comprises an image recognition model part and a model weight updating part, the image recognition model part converts a new image sample received by an image sample buffer into a first feature vector of a test sample, and converts the first feature vector into prediction category information; converting the first feature vector into a second feature vector; judging whether the test sample is an out-of-distribution sample or not; labeling new category data for the out-of-distribution samples, forming known category samples, and predicting the known category samples to obtain a prediction result; a model weight updating part extracts an image sample corresponding to a known category sample based on a prediction result, and generates a virtual distribution outer sample based on a Hamilton Monte Carlo algorithm and a first feature vector of the image sample; and updating the weight of the image recognition model based on the first feature vector of the virtual distribution outer sample, the image sample and the image sample original data corresponding to the known category sample.

Description

Technical Field

[0001] This invention relates to the field of intelligent image detection technology, and in particular to an out-of-distribution image detection method and system oriented towards incremental learning. Background Technology

[0002] With the rapid development of artificial intelligence, deep neural networks have been widely applied in various real-world image recognition scenarios. Traditional deep learning models are typically based on the independent and identically distributed (ICD) assumption, meaning that the training and test sets have the same data distribution. However, in real-world applications, the distribution of image data often changes over time, prompting the development of incremental learning image classification models—which continuously update model weights to adapt to new data distribution changes. Efficient incremental learning techniques are crucial for ensuring the model's performance in practical image recognition applications.

[0003] In the context of incremental learning, a key technical challenge lies in detecting and monitoring changes in the distribution of image data. Image samples from distributions different from the training set or from unknown categories are called out-of-distribution samples. When the distribution of image data changes over time, existing models may make biased or distorted predictions about these out-of-distribution samples. Therefore, image recognition models need to be able to dynamically detect out-of-distribution samples and update in real time to adapt to these new samples; this is known as out-of-distribution sample detection. The performance of out-of-distribution sample detection is extremely important for ensuring the safety and controllability of incremental learning models, especially in safety-critical fields such as medical diagnosis, autonomous driving, and weather forecasting.

[0004] For example, in the context of in-vehicle intelligent driving, the challenge faced by road object classification systems is that the characteristics of objects on the road (such as vehicles, pedestrians, obstacles, road signs, etc.) may change over time or geographically. When a road object recognition model trained on old road image data is deployed in a real-world environment, it may misclassify objects whose characteristics have changed, or misclassify newly appearing objects into known categories, leading to errors in the intelligent driving system's judgment and severely impacting its reliability and safety. Therefore, incremental learning techniques must be used to periodically update the road object recognition model. On the one hand, newly collected image data needs to be preliminarily evaluated to identify out-of-distribution samples; on the other hand, labeled out-of-distribution samples are used to train and update the model, enabling it to adapt to new data distributions without affecting its performance on the original data distribution.

[0005] For example, CN113259331A discloses an online detection method and system for unknown abnormal traffic based on incremental learning. This includes constructing an initial classification model based on deep learning and initially training the model using a Softmax classifier; determining whether the input data sample belongs to a known or unknown category through an unknown detection module, then automatically labeling the features of the unknown category samples and adding these features to a network update module and a buffer module; updating the initial classification model through the network update module to convert unknown categories to known categories; and storing the sample features and labels of unknown categories through a buffer module to adapt to online modes and help distinguish the types of unknown anomalies. However, the unknown detection module in this technical solution heavily relies on the performance of the initial classification model. If the performance of the initially trained model is poor, the detection rate of subsequent unknown distribution samples will drop significantly, leading to insufficient model adaptability. Furthermore, this method mixes new and original categories for training, which may cause a decrease in the model's classification performance on the original categories. Moreover, the solution provided by this patent can only handle relatively low-dimensional online traffic feature data and is difficult to adapt to high-dimensional data such as images.

[0006] Therefore, how to provide an efficient out-of-distribution image detection method for incremental learning image recognition models, so as to dynamically adapt to changing distributions, improve the detection rate of unknown category samples of image recognition models, and maintain the classification performance of the model in the original image categories, is an urgent problem to be solved.

[0007] Furthermore, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making this invention, but due to space limitations, not all details and contents were listed in detail. However, this does not mean that the present invention does not possess the features of these prior art. On the contrary, the present invention already possesses all the features of the prior art, and the applicant reserves the right to add relevant prior art to the background art. Summary of the Invention

[0008] In training incremental learning image recognition models, newly arrived image data typically requires manual screening and annotation, consuming significant manpower and limiting the model's application scale and iteration speed. Conversely, for unsupervised incremental learning that doesn't require manual annotation, the performance of image recognition models largely depends on their basic classification capabilities. Insufficient basic classification ability can lead to a sharp decline in model performance during incremental learning. Therefore, in incremental learning scenarios, using out-of-distribution image detection to assist in screening samples of unknown categories is a more efficient and reliable technical approach.

[0009] To enhance the out-of-distribution image detection performance of models, existing methods typically introduce additional regularization loss terms during training or use additional labeled out-of-distribution datasets for fine-tuning. Specifically, introducing additional large-scale out-of-distribution data refers to collecting class data that does not overlap with the original training distribution, thereby directly training the model's out-of-distribution image detection capabilities.

[0010] For example, US10147049A discloses a classifier for out-of-distribution data scraping and monitoring based on multiple user data sets. This includes a storage device that needs access to all user data, and an out-of-distribution data generator that scrapes data from multiple other users for a single user. The generated out-of-distribution data is used to train and update the model. However, this method relies on a large user base; if the number of users is insufficient, it cannot obtain enough high-quality out-of-distribution data. Furthermore, access restrictions on user data can introduce security risks of data leakage and increase data sampling costs. Additionally, this technical solution is specifically designed for user feature data and cannot handle image data.

[0011] Therefore, generating virtual out-of-distribution image samples is a more feasible approach. Virtual out-of-distribution image samples eliminate the need for costly manual annotation, accelerating model iteration and reducing training costs. Furthermore, they avoid security issues such as data leakage, making them a more acceptable solution in security-critical domains. However, generating virtual out-of-distribution image samples may introduce additional computational costs, and the diversity, representativeness, and accuracy of the generated out-of-distribution data are difficult to guarantee.

[0012] For example, CN113392890A proposes an out-of-distribution anomaly detection method based on outlier data augmentation. In the feature extraction stage, the encoder part of an autoencoder algorithm is used to extract feature vectors from the input in-distribution samples. In the data augmentation and reconstruction stage, feature-based data augmentation methods are used to transform the features extracted in the feature extraction stage, and the decoder part of the autoencoder is used to generate a sufficient number of auxiliary out-of-distribution anomaly sample datasets containing semantic information from the augmented feature vectors. In the sample labeling stage, the prediction accuracy of the original classifier for the reconstructed samples is used as the soft label value of the out-of-distribution anomaly sample. In the classifier retraining stage, the classifier is retrained by combining the in-distribution training dataset containing hard-labeled supervision signals and the auxiliary out-of-distribution anomaly sample dataset containing soft-labeled supervision signals. However, the outlier feature vectors synthesized in this technical solution during the feature extraction stage are entirely based on Gaussian sampling, without incorporating prior knowledge of the existing dataset as guidance. This can easily lead to the generated outlier feature vectors being too homogeneous, lacking diversity and representativeness, thus affecting the model's out-of-distribution detection performance. At the same time, the generated outlier feature vectors do not undergo the necessary correctness verification stage, which may cause them to be confused with the original in-distribution vectors, thus destroying the model's representation space.

[0013] To address the shortcomings of existing technologies, this invention provides, from a first aspect, an out-of-distribution image detection system oriented towards incremental learning, the system comprising an image recognition model unit and a model weight update unit.

[0014] The image recognition model unit converts new image samples received from the image sample buffer into a first feature vector of the test sample, and then converts the first feature vector into predicted category information; it also converts the first feature vector into a second feature vector; it calculates the scalar score and scalar score threshold of the second feature vector to determine whether the test sample is an out-of-distribution sample; it labels the out-of-distribution samples with new category data to form known category samples, and then predicts the known category samples to obtain the prediction result. The model weight update unit extracts image samples corresponding to known category samples based on the prediction results, generates virtual out-of-distribution samples based on the Hamilton Monte Carlo algorithm and the first feature vector of the image samples, and updates the weights of the image recognition model based on the feature vector of the virtual out-of-distribution samples, the image samples, and the original data of the image samples corresponding to known category samples.

[0015] According to a preferred embodiment, the image recognition model unit includes: The feature extractor extracts the first feature vector from the image sample; The classifier transforms the first feature vector into predicted category information; The mapper performs regularization calculations based on the first feature vector to obtain a low-dimensional hyperspherical feature vector with a modulus of 1, which is then used as the second feature vector. The out-of-distribution sample detector calculates the k-nearest neighbor distance scalar based on the second feature vector and performs out-of-distribution sample discrimination based on the scalar score.

[0016] According to a preferred embodiment, the model weight update unit includes: The image acquisition unit acquires new image samples and sends them to the weight training unit; The feature vector acquisition unit extracts image samples corresponding to known category samples based on the prediction results; The out-of-distribution sample generation unit is a computational component used to generate virtual out-of-distribution sample feature vectors. It receives the second feature vector of known category data and generates virtual out-of-distribution samples. The weight training unit obtains the old model weights from the image recognition model unit, updates the latest weights of the image recognition model based on the virtual distribution out-of-sample feature vectors, image samples, and the original data of image samples corresponding to known category samples, and sends the latest weights to the image recognition model unit.

[0017] According to a preferred embodiment, the system further includes: Image sample buffer, which stores received image samples; A data feature vector buffer stores the first and second feature vectors after processing by the image recognition model unit. Known category data cache stores image data of known categories; A new category data cache stores image data of unknown categories.

[0018] According to a preferred embodiment, the step of generating out-of-distribution samples based on the Hamiltonian Monte Carlo algorithm in the model weight update section includes: Select two clusters of class feature vectors that are close to each other, calculate the midpoint between the two clusters and use it as the sampling starting point; The hypersphere is modeled using the von Mises-Fischer kernel function, and the probability likelihood of the sampling starting point is calculated. The out-of-distribution probability likelihood is modeled using a class-conditional probability density function and kernel density estimation. The hard-interval threshold is calculated based on the in-distribution probability likelihood at the sampling start point to filter out the correct out-of-distribution sample points during the sampling process. Within the iteration period of Hamiltonian Monte Carlo sampling, the out-of-distribution probability likelihood is calculated based on the current sample points of the in-distribution classes that are close to each other; Calculate the potential energy function in the Hamiltonian energy system based on the out-of-distribution probability likelihood estimation; Solve the Hamiltonian equation using the derivative, sample the solution using the Metropolis probability, and obtain the current feature vector sample points based on the sampling results; The current samples are filtered based on the hard margin threshold and the probability likelihood within the sample point distribution. Sample points whose probability likelihood within the sample point distribution is lower than the hard margin threshold are retained as samples outside the virtual distribution.

[0019] According to a preferred embodiment, the model weight update unit further performs robust fine-tuning on the model based on virtual distribution out-of-samples, wherein, The model is trained using out-of-distribution sample discrimination loss to separate in-distribution and out-of-distribution samples; Using in-distribution class contrast loss to train the model makes the in-distribution classes more compact; Use cross-entropy loss to train the model and maintain the performance of the original classification task.

[0020] According to a preferred embodiment, the step of updating the model weights by the model weight updating unit includes: The generated out-of-distribution samples are mixed with the actual in-distribution samples to update the model weights; The model parameters are optimized by combining outlier discrimination loss, contrastive learning techniques, and cross-entropy loss.

[0021] This invention provides, from a second aspect, an out-of-distribution image detection method oriented towards incremental learning, the method comprising: The new image sample received by the image sample buffer is converted into the first feature vector of the test sample, and the first feature vector is converted into predicted category information; and the first feature vector is converted into the second feature vector; the scalar score and scalar score threshold of the second feature vector are calculated to determine whether the test sample is an out-of-distribution sample; new category data are labeled for the out-of-distribution samples to form known category samples, and prediction is performed on the known category samples to obtain the prediction result; Based on the prediction results, image samples corresponding to known category samples are extracted. Virtual out-of-distribution samples are generated based on the Hamiltonian Monte Carlo algorithm and the first feature vector of the image samples. The weights of the image recognition model are updated based on the first feature vector of the virtual out-of-distribution samples, the original data of the image samples and the image samples corresponding to known category samples.

[0022] According to a preferred embodiment, the step of generating virtual out-of-distribution samples based on the Hamiltonian Monte Carlo algorithm and the first feature vector of image samples includes: Select two clusters of class feature vectors that are close to each other, calculate the midpoint between the two clusters and use it as the sampling starting point; The hypersphere is modeled using the von Mises-Fischer kernel function, and the probability likelihood of the sampling starting point is calculated. The out-of-distribution probability likelihood is modeled using a class-conditional probability density function and kernel density estimation. The hard-interval threshold is calculated based on the in-distribution probability likelihood at the sampling start point to filter out the correct out-of-distribution sample points during the sampling process. Within the iteration period of Hamiltonian Monte Carlo sampling, the out-of-distribution probability likelihood is calculated based on the current sample points of the in-distribution classes that are close to each other; Calculate the potential energy function in the Hamiltonian energy system based on the out-of-distribution probability likelihood estimation; Solve the Hamiltonian equation using the derivative, sample the solution using the Metropolis probability, and obtain the current feature vector sample points based on the sampling results; The current samples are filtered based on the hard margin threshold and the probability likelihood within the sample point distribution. Sample points whose probability likelihood within the sample point distribution is lower than the hard margin threshold are retained as samples outside the virtual distribution.

[0023] According to a preferred embodiment, the method further includes: robustly fine-tuning the model based on virtual out-of-distribution samples, wherein the model is trained using out-of-distribution sample discriminative loss to separate in-distribution and out-of-distribution samples; the model is trained using in-distribution class contrastive loss to make in-distribution classes more compact; and the model is trained using cross-entropy loss to maintain the performance of the original classification task. Attached Figure Description

[0024] Figure 1 This is an overall structural diagram of the out-of-distribution image generation and detection method oriented towards incremental learning provided by the present invention. Figure 2 This is a structural diagram of the image recognition model in the incremental learning scenario provided by the present invention; Figure 3 This is a structural diagram of the model weight update unit of the image recognition model for incremental learning provided by the present invention; Figure 4 This is a flowchart of the image distribution out-of-sample generation and detection method for incremental learning provided by the present invention; Figure 5 This is a flowchart of the operation of the efficient out-of-distribution sample generation method based on the Hamilton Monte Carlo algorithm provided by the present invention; Figure 6 This is a flowchart of the model robustness fine-tuning method and the operation of updating model weights based on virtual distribution out-of-samples provided by the present invention.

[0025] List of reference numerals 10: Image sample buffer; 20: Image recognition model unit; 30: Data feature vector buffer; 40: Known category data buffer; 50: New category data buffer; 60: Model weight update unit; 100: Feature extractor; 110: Classifier; 120: Mapper; 130: Out-of-distribution sample detector; 200: Image acquisition unit; 210: Feature vector acquisition unit; 220: Out-of-distribution sample generation unit; 230: Weight training unit. Detailed Implementation

[0026] The following is a detailed explanation with reference to the accompanying drawings.

[0027] This invention provides explanations for some terms and nouns.

[0028] Incremental learning is a machine learning paradigm that continuously updates model weights, allowing the model to learn and update itself from new data after initial training without retraining the entire model. This approach is particularly suitable for scenarios where data changes over time or where all data cannot be obtained at once. Through incremental learning, the system can effectively adapt to new knowledge while retaining already learned information, avoiding the problem of storing large amounts of historical data and performing repeated training in traditional batch learning methods. Considering resource constraints, incremental learning algorithms need to be designed to be highly efficient to process newly arriving data in real time and make rapid decisions.

[0029] Out-of-distribution detection refers to the ability to identify data that has not been seen during training or whose distribution differs significantly from the training data. This capability is crucial for improving the robustness and safety of models, especially in safety-critical applications such as autonomous driving, medical diagnosis, and financial risk assessment. In incremental learning scenarios, out-of-distribution detection is often used to identify unknown categories and assist in dynamically expanding the model training set. In this invention, based on a multi-classification task scenario, where... Represents the input space of the model. This represents the current label space of the model. Represents the original classification task dataset, with independent and identically distributed sampling at... On ,and In order to be in The edge distribution on the distribution. For out-of-distribution detection tasks, it is modeled as a binary classification task, that is, for The model needs to determine whether it belongs to or One of them Indicates not to There are overlapping unknown distributions. For out-of-distribution sample detectors, then use... It means that among them For scoring functions, The threshold for discrimination is usually set to the scoring function value when the true positive rate (TPR) is 95%.

[0030] Hamiltonian Monte Carlo algorithm: refers to a Markov chain Monte Carlo (MCMC) method for sampling from complex probability distributions. It is particularly suitable for sampling problems in high-dimensional spaces and, in many cases, is more efficient than traditional MCMC methods such as Metropolis-Hastings. The Hamiltonian Monte Carlo algorithm combines Hamiltonian dynamics from physical systems with the idea of ​​Monte Carlo sampling to explore complex target distributions more efficiently. Given a target distribution... The algorithm will It is considered as a high-dimensional space coordinate, and an auxiliary momentum is introduced. And depict the current phase space coordinates of the system as And based on Hamilton's equations, new phase space coordinates are proposed. Hamiltonian energy is defined as follows: .

[0031] Hamilton's equation is then defined as follows: .

[0032] In the above formula, t represents the time step. During each round of sample generation, the Hamiltonian Monte Carlo algorithm calculates the time step based on the current phase space coordinates. ,conduct Step, step length is Solving Hamilton's equations using the frog-jump discretization method: ; ;

[0033] .

[0034] Ultimately, the new coordinates As samples obtained from sampling, the probability is tested based on Metropolis. The algorithm receives data. Because Hamiltonian algorithms are modeled based on Hamiltonian dynamics, their sampling rate is close to 1, resulting in good convergence and improving the sampling efficiency of out-of-distribution samples.

[0035] The present invention provides an out-of-distribution image detection method and system for incremental learning, comprising an image recognition model unit 20 in an in-vehicle intelligent driving system and a method and system for dynamically detecting out-of-distribution images and updating model weights. The present invention enables the image recognition model unit 20 to detect input out-of-distribution images with higher accuracy and dynamically update model weights to adapt to new data distributions.

[0036] Example 1 The distributed out-of-distribution image generation and detection system for incremental learning of the present invention includes an image recognition model unit 20 and an in-vehicle intelligent driving system with a data buffer connected thereto.

[0037] like Figure 1 As shown, the out-of-distribution image generation and detection system for incremental learning of the present invention includes an image sample buffer 10, an image recognition model unit 20, a data feature vector buffer 30, a known category data buffer 40, a new category data buffer 50, and a model weight update unit 60.

[0038] like Figure 1 As shown, the image sample buffer 10 is connected to the image recognition model unit 20 and transmits data. The image recognition model unit 20 is connected to the data feature vector buffer 30 and transmits data. The data feature vector buffer 30 is connected to the known category data buffer 40 and the new category data buffer 50, respectively, and transmits data. The known category data buffer 40 and the new category data buffer 50 are respectively connected to the model weight update unit 60 and transmit data. Preferably, the image sample buffer 10 is connected to the known category data buffer 40 and the new category data buffer 50, respectively, and transmits data. The image recognition model unit 20 is connected to the model weight update unit 60 and transmits data.

[0039] Preferably, the image sample cache 10 is a first memory, and its physical hardware is preferably a high-speed solid-state drive (SSD) or dynamic random access memory (DRAM) to realize the data caching function. The image recognition model unit 20 is a first processor, and its physical hardware is a processor or a dedicated integrated chip, such as an embedded GPU (e.g., NVIDIA Jetson series) or a dedicated AI acceleration chip (e.g., TPU), responsible for performing image feature extraction and classification tasks. The data feature vector cache 30 is a second memory, and its physical hardware is preferably high-bandwidth memory (HBM) or a low-latency storage array. The known category data cache 40 is a third memory, and the new category data cache 50 is a fourth memory. The physical hardware of the known category data cache 40 and the new category data cache 50 is preferably non-volatile memory (e.g., NVMe SSD). The model weight update unit 60 is a second processor, and its physical hardware is preferably a programmable gate array (FPGA) or an AI chip that supports online learning, dynamically adjusting the model weight parameters through the backpropagation algorithm. Each hardware unit realizes data interaction through a PCIe or high-speed Ethernet interface to meet the real-time and reliability requirements of the vehicle system.

[0040] Preferably, the image sample buffer 10 is a first memory used to temporarily store newly received image data.

[0041] The image recognition model unit 20 contains an image recognition model. This image recognition model is a multi-class deep learning neural network model used to predict image categories. The image recognition model unit 20 is communicatively connected to the image sample buffer 10 to obtain updated image data from it; simultaneously, it is communicatively connected to the model weight update unit 60 to update the model weights, enabling iterative updates of the model weight parameters.

[0042] The data feature vector buffer 30 is a second memory used to temporarily store image feature vectors generated by the image recognition model unit 20. The data feature vector buffer 30 is communicatively connected to the image recognition model unit 20 to obtain image feature vector data. The data feature vector buffer 30 is also communicatively connected to the model weight update unit 60 to transmit image feature vector data thereto.

[0043] The known category data buffer 40 is a third memory used to temporarily store known category image data. The known category data buffer 40 is communicatively connected to the image sample buffer 10 and the data feature vector buffer 30, and is used to obtain known category image data according to the image detection logic.

[0044] The new category data buffer 50 is a fourth memory used to temporarily store image data of unknown categories. The new category data buffer 50 is communicatively connected to the image sample buffer 10 and the data feature vector buffer 30, and is used to obtain image data of unknown categories according to image detection logic.

[0045] The model weight update unit 60 is a computation module used to update the weights of an image recognition model based on image data. The model weight update unit 60 is communicatively connected to the image recognition model unit 20, the data feature vector cache 30, the known category data cache 40, and the new category data cache 50. It is used to obtain feature vector data from the data feature vector cache 30, training sample data from the known category data cache 40 and the new category data cache 50, and historical weight parameters from the image recognition model unit 20 to perform the weight update operation.

[0046] Figure 2 This is a structural diagram representing the image recognition model unit 20. For example... Figure 2 As shown, the image recognition model unit 20 includes a feature extractor 100, a classifier 110, a mapper 120, and an out-of-distribution sample detector 130.

[0047] Feature extractor 100 is a multi-layered deep learning neural network model used to extract feature vectors from image samples. When feature extractor 100 is physical hardware, it is a GPU or TPU with an internally configured multi-layered deep learning neural network model.

[0048] Classifier 110 is a single-layer fully connected linear network classifier that communicates with feature extractor 100 to acquire feature vectors of the image and convert these feature vectors into predicted category information. In the case where classifier 110 is physical hardware, it is a CPU with an internal classification encoding program.

[0049] Mapper 120 is a neural network with a multilayer perceptron structure. It communicates with feature extractor 100 to acquire feature vectors of the image and transforms the feature vectors into low-dimensional hyperspherical feature vectors. When mapper 120 is physical hardware, it is a GPU or TPU with an internally configured multilayer perceptron neural network.

[0050] The out-of-distribution sample detector 130 is an outlier discriminator based on the k-nearest neighbor algorithm. It communicates with the mapper 120, calculates the k-nearest neighbor distance scalar based on the low-dimensional hyperspherical feature vector, and discriminates out-of-distribution samples according to a threshold. When the out-of-distribution sample detector 130 is physical hardware, it can be a dedicated AI accelerator with an internal k-nearest neighbor algorithm or integrated into an existing GPU / TPU.

[0051] Preferably, the feature extractor 100, classifier 110, mapper 120 and out-of-distribution sample detector 130 can be connected via an on-chip bus to form an image recognition model unit 20.

[0052] Figure 3 This is a structural diagram of the model weight update unit 60. (See diagram for example.) Figure 3 As shown, the model weight update unit 60 includes an image acquisition unit 200, a feature vector acquisition unit 210, an out-of-distribution sample generation unit 220, and a weight training unit 230.

[0053] The image acquisition unit 200 is a device that communicates with the known category data buffer 40 and the new category data buffer 50 to acquire new image samples. The image acquisition unit 200 is connected and communicates with the weight training unit 230 to provide image training data.

[0054] The feature vector acquisition unit 210 is a device that communicates with the data feature vector buffer 30 and is used to acquire feature vector data of image data.

[0055] The out-of-distribution sample generation unit 220 is a computational component for generating virtual out-of-distribution sample feature vectors. The out-of-distribution sample generation unit 220 communicates with the feature vector acquisition unit 210 to acquire feature vectors of known category data. The out-of-distribution sample generation unit 220 is connected and communicates with the weight training unit 230 to provide virtual out-of-distribution samples.

[0056] The weight training unit 230 is a computational component used to update model weights. It communicates with the image recognition model unit 20 to obtain the old model weights and send the updated model weights. The weight training unit 230 is connected to the image acquisition unit 200 to acquire image samples. The weight training unit 230 is connected to the out-of-distribution sample generation unit 220 to obtain virtual out-of-distribution sample feature vectors.

[0057] The steps of the out-of-distribution image detection method based on incremental learning of the present invention are as follows: Figure 4 As shown.

[0058] S000: Start.

[0059] S101: Get the data for round T.

[0060] Specifically, the incremental learning iteration process begins. In the Tth iteration, the image sample buffer 10 receives and stores new image samples from the external environment.

[0061] Preferably, the external environment here refers to the physical input source for acquiring image samples. This physical input source is, for example, an image acquisition device or an image database. In the incremental learning iterative process, the image sample buffer 10 acquires image samples from the image acquisition device or the image database.

[0062] S102: Obtain the data feature vector (second feature vector).

[0063] The feature extractor 100 of the image recognition model unit 20 converts newly received image samples in the image sample buffer 10 into the first feature vector of the test sample: In the first eigenvector, This represents the value of the Nth dimension of the first feature vector.

[0064] The first feature vector is obtained by the mapper 120 in the image recognition model unit 20 based on regularization, resulting in a second feature vector with a modulus of 1 and located on a hypersphere in a high-dimensional space: .

[0065] In the above formula, This represents the L2 normalization operation, which divides the first eigenvector by its Euclidean norm (L2 norm) so that the normalized vector has a magnitude of 1. This represents a Multi-Layer Perceptron (MLP), which is the structure of mapper 120. A MLP is a feedforward neural network composed of multiple fully connected layers used to perform nonlinear transformations on input features. θ represents the parameters (weights and biases) of the MLP, which are learnable variables in the neural network. This indicates that the input will be... The transformation is performed using the parameter θ through an MLP network. This indicates that the input feature vector, i.e. the first feature vector, is the feature vector extracted from the image sample by the feature extractor 100.

[0066] The principle behind the above formula is as follows: First, the first feature vector x output by the feature extractor 100 is input into the MLP. The MLP uses the parameter θ to process the first feature vector... A nonlinear transformation is performed. Then, the output of the MLP is L2 normalized to obtain a second eigenvector z with a magnitude of 1. The second eigenvector z lies on the unit hypersphere in the high-dimensional space and is the output of mapper 120.

[0067] The image recognition model unit 20 stores the second feature vector into the data feature vector cache 30.

[0068] S103: Perform out-of-distribution sample detection.

[0069] The out-of-distribution sample detector 130 uses a scoring function based on a distance metric derived from the k-nearest neighbor algorithm to calculate a scalar score on the second feature vector of all image samples.

[0070] The formula for calculating this scoring function is: .

[0071] In the above formula, Represents a scalar fraction. This represents the second eigenvector. Represents the second eigenvector The average of the k nearest neighbor feature vectors is calculated. This average is obtained by calculating the distance between the second feature vector z and all sample feature vectors in the training set, selecting the k nearest sample feature vectors, and then calculating their average. This indicates the number of nearest neighbors considered. This represents the threshold.

[0072] S104: Determine whether it is an out-of-distribution sample.

[0073] The out-of-distribution sample detector 130 uses the scalar score of the second feature vector and a threshold. Determine whether the image sample is an out-of-distribution sample. If the image sample is an out-of-distribution sample, proceed to step S110; otherwise, that is, the image sample is not an out-of-distribution sample, proceed to step S105.

[0074] S105: Category prediction.

[0075] The classifier 110 in the image recognition model unit 20 receives known category samples from the feature extractor 100. The classifier 110 predicts the category of the detected known category samples and proceeds to step S111. The classifier 110 in the image recognition model unit 20 receives the first feature vector x of the known category samples from the feature extractor 100. The classifier 110 projects the first feature vector onto a space of dimension C using a single fully connected layer to obtain a sample category vector. Then, it uses the Softmax function to convert the sample category vector into a category probability distribution, taking the index with the highest probability as the predicted category of the sample. Here, C is the total number of known categories.

[0076] Then proceed to step S106.

[0077] S106: Update the feature vector of the data within the distribution.

[0078] Based on the predicted category sent by classifier 110, feature extractor 100 retrieves image samples corresponding to known category samples from image sample buffer 10 and stores them in known category data buffer 40 as in-distribution prior knowledge for subsequent out-of-distribution sample generation. This known category data buffer 40 is dynamically updated, ensuring both low storage overhead and time-adaptive characteristics. The known category data buffer 40 allocates a fixed-size data buffer (e.g., 1000 image samples) for each known category's image samples and maintains a counter for each location. Each time the data buffer is updated, all counters are incremented by 1. When a new image sample is received for each category, the corresponding category's data buffer erases the same number of the oldest image samples (those with the largest counter value), stores the new image sample for that category, and sets the corresponding counter to 0.

[0079] S107: Generation of out-of-distribution samples.

[0080] Using image samples from the known category data buffer 40, the out-of-distribution sample detector 130 generates a virtual out-of-distribution sample feature vector, i.e., the third feature vector, based on the Hamilton Monte Carlo algorithm. See subsequent sections for detailed steps.

[0081] S108: Update model weights.

[0082] The model weight update unit 60 stores the generated virtual out-of-distribution sample feature vectors in the known category data cache 40, and updates the weights of the image recognition model based on the unknown category image samples stored in the unknown category data cache 50 labeled in step S110.

[0083] S109: Determine whether to stop incremental learning iteration? The model weight update unit 60 determines whether the incremental learning iterations have reached the upper limit. If the incremental learning iterations have been reached, the model is stopped from iteratively updating; if the incremental learning iterations have not been reached, the process returns to step S101.

[0084] Preferably, the model weight update unit 60 includes a counter, which increments by 1 each time the model weights are updated. When the value of the counter reaches a set increment cycle, the iterative update of the model stops.

[0085] S110: Label new category data.

[0086] The out-of-distribution sample detector 130 labels the detected unknown category samples to obtain category information, and retrieves the image samples corresponding to the category from the image sample cache 10 and stores them in the new category data cache 50 for subsequent updates to the model weights to adapt to the new data distribution.

[0087] S111: Output predicted category.

[0088] Classifier 110 outputs prediction results for samples of known categories, which are used for decision-making information in the in-vehicle intelligent driving system.

[0089] for Figure 4 Step S107, in particular, describes an efficient out-of-distribution sample generation method based on the Hamiltonian Monte Carlo algorithm. (See [link to relevant documentation]). Figure 5 . Figure 5 The operation flow of an efficient out-of-distribution sample generation method based on the Hamilton Monte Carlo algorithm is shown.

[0090] S10701: Obtain the data feature vector.

[0091] The feature vector acquisition unit 210 in the model weight update unit 60 acquires the second feature vector from the data feature vector buffer 30. The feature vector acquisition unit 210 transmits the second feature vector to the out-of-distribution sample generation unit 220 and the weight training unit 230, respectively.

[0092] S10702: Calculate the sampling start point vector.

[0093] In the out-of-distribution sample generation unit 220, pairwise intra-distribution category feature vector clusters that are close in distance are selected, and the midpoint between the two clusters is calculated. As the sampling start point vector.

[0094] S10703: Calculate the probability likelihood within the distribution of the sampling starting point.

[0095] In the out-of-distribution sample generation unit 220, the in-distribution probability likelihood of the hypersphere is modeled according to the von Mises-Fischer (vMF) kernel function, and the in-distribution probability likelihood of the sampling starting point is calculated: .

[0096] In the above formula, The mean of the vMF kernel function. To concentrate on super parameters, Category 1 The modified Bessel function (v=d / 2-1, used to normalize the vMF distribution) is a second-order function. Here, represents the regularization coefficient. Subsequently, the probability likelihood within the distribution is modeled using the class-conditional probability density function and kernel density estimation (KDE): .

[0097] .

[0098] In the above formula, This is the result of kernel density estimation (KDE) of the probability likelihood within the distribution. Let N be the training sample set of class c (the known set of feature vectors of samples within the distribution). N is the number of training samples in class c (i.e., ...). (Number of samples) The feature vector of the i-th sample in the c-th training sample set ( ∈ C represents the total number of categories (the total number of all categories in the dataset). The posterior probability of a feature vector sample point z belonging to class c is given by normalization of the class conditional density. Then proceed to step S10708.

[0099] S10704: Calculate the out-of-distribution probability likelihood.

[0100] During the iteration cycle of Hamiltonian Monte Carlo sampling in the out-of-distribution sample generation unit 220, the out-of-distribution probability likelihood is calculated based on the current sample points of each pair of closely spaced in-distribution classes. The out-of-distribution probability likelihood of a sample for a pair of in-distribution classes is expressed as the average of the k-nearest neighbor scores of the sample to the two class clusters, calculated as follows: .

[0101] in and These represent the cached feature vectors of the two in-distribution categories, For the sample of the first The k-nearest neighbor distance metric for a class, i.e., the distance to the k-nearest neighbor of the The Euclidean distance of distant samples.

[0102] S10705: Calculate the Hamiltonian potential function and its derivative.

[0103] In the out-of-distribution sample generation unit 220, based on the out-of-distribution probability likelihood estimate obtained in step S10704, the potential energy function in the Hamiltonian energy system, i.e., the negative logarithm of the out-of-distribution probability likelihood, is calculated: .

[0104] First, use the sampling start point The Hamiltonian energy of a single Markov chain is calculated: .

[0105] Then, the partial derivative of the energy function with respect to the sample points is taken. The partial derivative of the energy function is shown below: .

[0106] S10706: Solve Hamilton's equations.

[0107] In the out-of-distribution sample generation unit 220, the Hamiltonian equation is solved using the derivative calculated in step S10705. Since the Hamiltonian equation cannot be solved directly, an approximate solution is obtained using a leapfrog discretization method: ; ; ; .

[0108] In the above formula, The timestamp for the frog-jump discretization. Let be the step size for the frog-jump discretization. Meanwhile, because all sample feature vectors are regularized to samples on a unit hypersphere in a high-dimensional space, the above formula also ensures that the newly sampled sample points lie on the hypersphere.

[0109] S10707: Sampling based on Metropolis probability.

[0110] In the out-of-distribution sample generation unit 220, the solution results from step S10706 are sampled using Metropolis probability. Since the potential function is defined as the negative logarithm of the out-of-distribution probability likelihood, the sampled points will converge towards the unknown distribution. Therefore, the Metropolis probability obtained by sampling is close to 1, maintaining high sampling efficiency.

[0111] S10708: Calculate the hard-interval threshold.

[0112] In the out-of-distribution sample generation unit 220, the in-distribution probability likelihood of the sampling starting point is calculated according to step S10703, and a hard-margin threshold is calculated to filter out correct out-of-distribution sample points during the sampling process: .

[0113] In the above formula, For hyperparameter hard spacing, The sampling starting point for each category The probability likelihood of the hard-margin threshold Its function is to reject samples that are too close to any class cluster within any distribution.

[0114] S10709: Calculate the probability likelihood within the distribution.

[0115] In the out-of-distribution sample generation unit 220, the current feature vector sample point is obtained based on the sampling results in step S10707. The in-distribution probability likelihood of the current sample point is calculated. The in-distribution probability likelihood calculation process is as follows.

[0116] Calculate the in-distribution probability likelihood of the current feature vector sample points: .

[0117] In the above formula, The mean of the vMF kernel function. To concentrate on super parameters, Category 1 The modified Bessel function (v=d / 2-1, used to normalize the vMF distribution) is a second-order function. Here, represents the regularization coefficient. Subsequently, the probability likelihood within the distribution is modeled using the class-conditional probability density function and kernel density estimation (KDE): .

[0118] .

[0119] In the above formula, This is the result of kernel density estimation (KDE) of the probability likelihood within the distribution. Let N be the training sample set of class c (the known set of feature vectors of samples within the distribution). N is the number of training samples in class c (i.e., ...). (Number of samples) The feature vector of the i-th sample in the c-th training sample set ( ∈ C represents the total number of categories (the total number of all categories in the dataset). Let z be the in-distribution posterior probability of a feature vector sample point z belonging to class c (obtained by normalizing the class conditional density).

[0120] S10710: Determine whether the probability likelihood within the distribution is less than the hard-margin threshold.

[0121] In the out-of-distribution sample generation unit 220, the current sample is filtered according to the hard interval threshold calculated in step S10708 and the probability likelihood of the sample point distribution calculated in step S10709. If the probability likelihood of the current sample distribution is higher than the threshold, the process proceeds to step S10711. If the probability likelihood of the current sample distribution is lower than the threshold, the sample is retained and the process proceeds to step S10712.

[0122] S10711: Discard the current sample point and return to step S10704.

[0123] S10712: Determine whether the current sampling round has reached the upper limit.

[0124] If the upper limit is reached, proceed to step S10713; if the upper limit is not reached, retain the current sample point, use the current sample point as the sampling starting point, and return to step S10704.

[0125] S10713: Output the feature vector of the out-of-sample virtual distribution.

[0126] The out-of-distribution sample generation unit 220 outputs the feature vectors of the virtual out-of-distribution samples generated in all rounds and transmits them to the weight training unit 230.

[0127] for Figure 4 The method for updating model weights in step S108 is described in detail below. Figure 6 . Figure 6 The method for robust fine-tuning of the model based on out-of-sample data from a virtual distribution and the procedure for updating model weights are demonstrated.

[0128] S10801: Obtain the feature vector of the out-of-sample virtual distribution.

[0129] Weighted Training Department 230 from Figure 4 In step S107, the out-of-distribution sample generation unit 220 obtains the generated virtual outlier feature vector.

[0130] S10802: Obtain the feature vector of samples within the distribution.

[0131] The weight training unit 230 obtains the feature vectors of the samples in the distribution from the feature vector acquisition unit 210.

[0132] S10803: Calculate the out-of-distribution discriminant loss.

[0133] The weight training unit 230 calculates the out-of-distribution discrimination loss based on the feature vectors of the out-of-distribution samples and the feature vectors of the in-distribution samples. The purpose of this loss function is to gradually separate the in-distribution and out-of-distribution samples on the hypersphere. .

[0134] In the above formula, Indicates the category within the distribution The cluster centroid, This is a temperature hyperparameter. The loss function calculates the out-of-distribution discrimination loss as the logarithm of the distance from each virtual out-of-distribution sample to the center of each in-distribution class cluster. Optimizing this loss function can gradually distance all out-of-distribution samples from the in-distribution samples, thereby improving the model's out-of-distribution detection performance.

[0135] S10804: Calculate the in-distribution contrast loss.

[0136] The weight training unit 230 uses the feature vectors of in-distribution samples to calculate the in-distribution class contrast loss. To train the model, we can widen the in-distribution classes from each other and make the in-distribution class clusters more compact: ; .

[0137] These two losses represent increasing the distance between classes and decreasing the distance between intra-class samples, respectively. This is the loss term balancing parameter. This loss function makes the in-distribution classes on the hypersphere more compact, thereby increasing the space for potential out-of-distribution samples and improving the model's robustness to out-of-distribution samples.

[0138] S10805: Calculate the cross-entropy loss.

[0139] The weight training unit 230 uses in-distribution feature vectors, calculates class probabilities using classifier 110, and then calculates cross-entropy loss. To maintain the performance of classifier 110 on the original image sample recognition task.

[0140] S10806: Update model weights.

[0141] The weight training unit 230 adds the out-of-distribution discrimination loss, in-distribution contrast loss and crossover loss calculated in steps S10803, S10804 and S10805, and updates the model weights through backpropagation.

[0142] S10807: Obtain image samples of the new category.

[0143] The weight training unit 230 obtains new category image data from the new category data cache 50, and uses the feature extractor 100 and classifier 110 in sequence to calculate the softmax probability of the new category data.

[0144] S10808: Calculate the cross-entropy loss.

[0145] The weight training unit 230 calculates the cross-entropy loss on the new class data based on the Softmax probability obtained in step S10807.

[0146] S10809: Update the model.

[0147] The weight training unit 230 updates the weights of the image recognition model based on the cross-entropy loss calculated in step S10808. The image recognition model unit 20 updates the image recognition model using the new model weights.

[0148] Example 2 This embodiment is a further example of Embodiment 1, and repeated content will not be repeated.

[0149] In this embodiment of the out-of-distribution image detection system for incremental learning, the image sample buffer 10 is preferably an image storage chip. The data feature vector buffer 30 is preferably a data feature vector buffer chip, used to store the first feature vector and the second feature vector processed by the image recognition model unit 20. The known category data buffer 40 is preferably a known category data buffer chip, used to store known category image data. The new category data buffer 50 is preferably a new category data buffer chip, used to store unknown category image data.

[0150] Preferably, the image storage chip, the data feature vector cache chip, the known category data cache chip, and the new category data cache chip are all dedicated storage chips with SRAM as the core and integrated with dedicated control logic.

[0151] The image recognition model unit 20 is preferably an image recognition processor.

[0152] In the image recognition processor, the feature extractor 100 is preferably a dedicated feature extraction circuit, used to extract a first feature vector from image samples. The classifier 110 is preferably a category classification logic circuit, used to convert the first feature vector into predicted category information. The mapper 120 is preferably a feature mapping transformation circuit, used to perform regularization calculation based on the first feature vector to obtain a low-dimensional hyperspherical feature vector with a modulus of 1, and use the low-dimensional hyperspherical feature vector as a second feature vector. The out-of-distribution sample detector 130 is preferably an out-of-distribution sample detection circuit, used to calculate a k-nearest neighbor distance scalar based on the second feature vector, and to discriminate out-of-distribution samples based on the scalar score.

[0153] The dedicated feature extraction circuit, category classification logic circuit, feature mapping conversion circuit, and out-of-distribution sample detection circuit are connected via a shared bus. These circuits are clocked by a unified clock generator within the image recognition processor (e.g., a 1GHz master clock), and setup / hold time conflicts in data transmission are eliminated through register timing. These four circuits share the image recognition processor's internal power distribution network and ground plane, reducing noise interference (especially for high-precision feature calculation circuits).

[0154] The output of the feature extraction circuit is simultaneously connected to the input buffer register of the category classification logic circuit (used to transmit the first feature vector) and the input buffer register of the feature mapping conversion circuit (used to transmit the first feature vector) via an on-chip synchronous data bus (such as a high-speed link of the AXI-Stream protocol).

[0155] The dedicated feature extraction circuit sends a "Data Valid" signal to the category classification logic circuit and feature mapping conversion circuit via a clock synchronization signal line (from the same source as the processor's main clock) to ensure that downstream modules start processing when the data is stable.

[0156] The first feature vector, sent by the dedicated feature extraction circuit, is received via the aforementioned data bus. The category classification logic circuit temporarily stores it in its own input buffer register (the capacity of which is adapted to the feature vector dimension, such as a 256-bit register group). The category classification logic circuit then transmits the predicted category information to the main control unit of the image recognition processor (for subsequent decision-making) via the result output bus, while simultaneously feeding back a "classification complete" signal (Done) via a status signal line.

[0157] The feature mapping conversion circuit receives the first feature vector sent by the dedicated feature extraction circuit via the data bus, temporarily stores it in the input buffer register, and then generates a second feature vector (low-dimensional hyperspherical feature) through an internal arithmetic unit (such as an L2 normalization circuit). The second feature vector is directly written into the input FIFO buffer (first-in-first-out queue, adapting to data rate differences) of the out-of-distribution sample detection circuit via a dedicated feature link (a bus independent of the classification circuit, reducing latency). Preferably, the feature mapping conversion circuit sends a "Feature Ready" signal to the out-of-distribution sample detection circuit, triggering the out-of-distribution sample detection circuit to start processing.

[0158] The out-of-distribution sample detection circuit receives the second feature vector from the dedicated link of the feature mapping conversion circuit, temporarily stores it in the FIFO buffer, and then completes the out-of-distribution sample discrimination through the internal k-nearest neighbor calculation unit (parallel distance comparison circuit) and threshold judgment circuit.

[0159] The out-of-distribution sample detection circuit transmits the detection results ("in-distribution / out-of-distribution" identifier + scalar score) to the main control unit of the image recognition processor via the detection result bus, and at the same time notifies the main control unit whether the model update process needs to be started via the interrupt signal line (IRQ).

[0160] The model weight update unit 60 is preferably a weight update control chip, used to update the weights of the image recognition model.

[0161] The model weight update unit 60 includes an image acquisition unit 200, a feature vector acquisition unit 210, an out-of-distribution sample generation unit 220, and a weight training unit 230.

[0162] The image acquisition unit 200 is preferably an image data reading circuit. The image data reading circuit includes circuitry for a data interface and a buffer unit, used to read new image samples from external buffers. Preferably, the image data reading circuit is connected to a known category data buffer 40 and a new category data buffer 50 respectively via a bidirectional data bus (such as an SPI protocol bus), and reads new image samples from the known category data buffer 40 and the new category data buffer 50. Specifically, the image data reading circuit sends a "data read request" signal (including a sample index) to the known category data buffer 40 and the new category data buffer 50, and performs subsequent processing after receiving new image samples from the known category data buffer 40 and the new category data buffer 50 respectively.

[0163] The feature vector acquisition unit 210 is preferably a feature vector reading circuit. The feature vector reading circuit integrates interface logic and data filtering unit circuitry, and is connected to the data feature vector buffer 30 through a high-speed feature interface (such as the AXI-Stream protocol interface) to acquire feature vectors from the data feature vector buffer 30.

[0164] The out-of-distribution sample generation unit 220 is preferably a virtual out-of-distribution sample generation circuit. The virtual out-of-distribution sample generation circuit includes a dedicated circuit of the Hamiltonian Monte Carlo sampling operation unit for generating virtual out-of-distribution sample feature vectors.

[0165] The weight training unit 230 is preferably a weight training processing circuit. The weight training processing circuit is integrated from multiple sub-circuits.

[0166] The image data reading circuit sends temporarily stored image training data (original data of known category samples and new image samples) to the weight training processing circuit via an on-chip synchronization link. After receiving the image training data from the image data reading circuit, the weight training processing circuit writes the data into its own "image sample buffer" for subsequent training.

[0167] The feature vector reading circuit sends the filtered feature vectors of the known category data to the virtual out-of-distribution sample generation circuit via a dedicated feature transmission link. After receiving the feature vectors of the known category data from the feature vector reading circuit, the virtual out-of-distribution sample generation circuit generates virtual out-of-distribution sample feature vectors through its internally integrated Hamiltonian Monte Carlo sampling operation unit.

[0168] Once the virtual out-of-distribution sample feature vector is generated, the virtual out-of-distribution sample generation circuit sends the virtual out-of-distribution sample feature vector to the weight training processing circuit via a high-bandwidth data link. After receiving the virtual out-of-distribution sample feature vector from the virtual out-of-distribution sample generation circuit, the weight training processing circuit writes the virtual out-of-distribution sample feature vector into the "virtual sample buffer," which then participates in weight updates in conjunction with the image samples.

[0169] The weight training processing circuit is connected to the image recognition processor (image recognition model unit 20) via a weight interface bus. The weight training processing circuit sends an "old weight acquisition request" to the image recognition processor. After receiving the old model weights (such as convolution kernel parameters and classifier weights) from the image recognition processor, the weight training processing circuit loads the old model weights into the weight update subcircuit as initial parameters.

[0170] After obtaining image samples from the image data reading circuit and virtual out-of-distribution sample feature vectors from the virtual out-of-distribution sample generation circuit, the weight training processing circuit generates the latest model weights through internally integrated sub-circuits.

[0171] More preferably, the weight training unit 230 integrates a data buffer sub-circuit, a matrix multiplication and addition sub-circuit, a loss calculation sub-circuit, a weight update sub-circuit, and an interface sub-circuit.

[0172] The data buffer sub-circuit includes a FIFO register, which receives image samples and virtual samples from other circuits, operating on the same principle as the buffer unit in the image data reading circuit. The matrix multiply-add sub-circuit performs multiplication and addition operations on weights and features, belonging to the arithmetic operation circuit category, similar to the virtual distribution out-of-distribution sample generation circuit. The loss calculation sub-circuit is used to implement cross-entropy and contrastive loss calculations in hardware, belonging to the logic operation circuit category, similar to the feature vector reading circuit. The weight update sub-circuit is used to complete gradient descent and weight iteration. The interface sub-circuit integrates an AXI bus and a dedicated link interface, ensuring full compatibility with the interface protocols of all external circuits (such as SPI and high-speed feature interfaces).

[0173] In the weight training processing circuit, the image data reading circuit sends image samples to the data buffer sub-circuit via an on-chip synchronization bus (such as AXI-Stream). After receiving data from the image data reading circuit, the buffer sub-circuit temporarily stores the data and sends a "data ready" signal to the weight update sub-circuit.

[0174] The feature vector reading circuit sends known category feature vectors to the virtual out-of-distribution sample generation circuit via a dedicated feature link. After receiving the known category feature vectors from the feature vector reading circuit, the virtual out-of-distribution sample generation circuit initiates virtual sample generation.

[0175] The virtual out-of-bandwidth sample generation circuit sends virtual sample feature vectors to the data buffer sub-circuit via a high-bandwidth data link. After receiving the virtual samples from the virtual out-of-bandwidth sample generation circuit, the data buffer sub-circuit integrates them with the image samples and synchronously transmits them to the matrix multiply-add sub-circuit.

[0176] The weight training processing circuit sends new weights to the image recognition processor (image recognition model unit 20) through the weight interface bus. After receiving the new weights from the weight training processing circuit, the image recognition processor replaces the old model weights.

[0177] It should be noted that the specific embodiments described above are exemplary. Those skilled in the art can devise various solutions inspired by the disclosure of this invention, and these solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents. This specification contains multiple inventive concepts; phrases such as "preferredly" or "according to a preferred embodiment" indicate that the corresponding paragraph discloses an independent concept. The applicant reserves the right to file divisional applications based on each inventive concept.

Claims

1. An out-of-distribution image detection system oriented towards incremental learning, characterized in that, The system includes: The image recognition model unit (20) converts the new image sample received by the image sample buffer (10) into the first feature vector of the test sample, converts the first feature vector into predicted category information, and converts the first feature vector into the second feature vector; calculates the scalar score and scalar score threshold of the second feature vector to determine whether the test sample is an out-of-distribution sample; labels the out-of-distribution sample with new category data and forms a known category sample, and predicts the known category sample and obtains the prediction result; The model weight update unit (60) extracts image samples corresponding to the known category samples based on the prediction results, generates virtual out-of-distribution samples based on the Hamilton Monte Carlo algorithm and the first feature vector of the image samples, and updates the weights of the image recognition model based on the feature vector of the virtual out-of-distribution samples, the image samples and the original data of the image samples corresponding to the known category samples.

2. The system according to claim 1, characterized in that, The image recognition model unit (20) includes: Feature extractor (100) extracts a first feature vector from the image sample; The classifier (110) transforms the first feature vector into predicted category information; Mapper (120) performs regularization calculation based on the first feature vector to obtain a low-dimensional hypersphere feature vector with a modulus of 1, which is used as the second feature vector; The out-of-distribution sample detector (130) calculates the k-nearest neighbor distance scalar based on the second feature vector and performs out-of-distribution sample discrimination based on the scalar score.

3. The system according to claim 1 or 2, characterized in that, The model weight update unit (60) includes: The image acquisition unit (200) acquires new image samples and sends them to the weight training unit (230). The feature vector acquisition unit (210) extracts image samples corresponding to the known category samples based on the prediction results; The out-of-distribution sample generation unit (220) is a computational component for generating virtual out-of-distribution sample feature vectors. It receives the second feature vector of known category data and generates virtual out-of-distribution samples. The weight training unit (230) obtains the old model weights from the image recognition model unit (20), and updates the latest weights of the image recognition model based on the virtual distribution out-of-sample feature vector, image samples and original data of image samples corresponding to known category samples, and sends the latest weights to the image recognition model unit (20).

4. The system according to any one of claims 1 to 3, characterized in that, The system also includes: Image sample buffer (10) stores the received image samples; The data feature vector buffer (30) stores the first feature vector and the second feature vector after being processed by the image recognition model unit (20); Known category data cache (40) stores known category image data; A new category data cache (50) stores image data of unknown categories.

5. The system according to any one of claims 1 to 4, characterized in that, The step of the model weight update unit (60) generating out-of-distribution samples based on the Hamilton Monte Carlo algorithm includes: Select two clusters of class feature vectors that are close to each other, calculate the midpoint between the two clusters and use it as the sampling starting point; The hypersphere is modeled using the von Mises-Fischer kernel function, and the probability likelihood of the sampling starting point is calculated. The out-of-distribution probability likelihood is modeled using a class-conditional probability density function and kernel density estimation. The hard-interval threshold is calculated based on the in-distribution probability likelihood at the sampling start point to filter out the correct out-of-distribution sample points during the sampling process. Within the iteration period of Hamiltonian Monte Carlo sampling, the out-of-distribution probability likelihood is calculated based on the current sample points of the in-distribution classes that are close to each other; Calculate the potential energy function in the Hamiltonian energy system based on the out-of-distribution probability likelihood estimation; Solve the Hamiltonian equation using the derivative, sample the solution using the Metropolis probability, and obtain the current feature vector sample points based on the sampling results; The current samples are filtered based on the hard margin threshold and the probability likelihood within the sample point distribution. Sample points whose probability likelihood within the sample point distribution is lower than the hard margin threshold are retained as samples outside the virtual distribution.

6. The system according to any one of claims 1 to 5, characterized in that, The model weight update unit (60) also performs robust fine-tuning on the model based on virtual distribution out-of-samples, wherein, The model is trained using out-of-distribution sample discrimination loss to separate in-distribution and out-of-distribution samples; Using in-distribution class contrast loss to train the model makes the in-distribution classes more compact; Use cross-entropy loss to train the model and maintain the performance of the original classification task.

7. The system according to any one of claims 1 to 6, characterized in that, The steps of updating the model weights by the model weight updating unit (60) include: The generated out-of-distribution samples are mixed with the actual in-distribution samples to update the model weights; The model parameters are optimized by combining outlier discrimination loss, contrastive learning techniques, and cross-entropy loss.

8. An out-of-distribution image detection method oriented towards incremental learning, characterized in that, The method includes: The new image sample received by the image sample buffer (10) is converted into the first feature vector of the test sample, and the first feature vector is converted into predicted category information; and the first feature vector is converted into the second feature vector; the scalar score and scalar score threshold of the second feature vector are calculated to determine whether the test sample is an out-of-distribution sample; new category data are labeled for the out-of-distribution sample and known category samples are formed, and the known category samples are predicted and the prediction result is obtained; Based on the prediction results, image samples corresponding to the known category samples are extracted. Virtual out-of-distribution samples are generated based on the Hamiltonian Monte Carlo algorithm and the first feature vector of the image samples. The weights of the image recognition model are updated based on the feature vector of the virtual out-of-distribution samples, the image samples, and the original data of the image samples corresponding to the known category samples.

9. The method according to claim 8, characterized in that, The steps for generating virtual out-of-distribution samples based on the Hamiltonian Monte Carlo algorithm and the first feature vector of image samples include: Select two clusters of class feature vectors that are close to each other, calculate the midpoint between the two clusters and use it as the sampling starting point; The hypersphere is modeled using the von Mises-Fischer kernel function, and the probability likelihood of the sampling starting point is calculated. The out-of-distribution probability likelihood is modeled using a class-conditional probability density function and kernel density estimation. The hard-interval threshold is calculated based on the in-distribution probability likelihood at the sampling start point to filter out the correct out-of-distribution sample points during the sampling process. Within the iteration period of Hamiltonian Monte Carlo sampling, the out-of-distribution probability likelihood is calculated based on the current sample points of the in-distribution classes that are close to each other; Calculate the potential energy function in the Hamiltonian energy system based on the out-of-distribution probability likelihood estimation; Solve the Hamiltonian equation using the derivative, sample the solution using the Metropolis probability, and obtain the current feature vector sample points based on the sampling results; The current samples are filtered based on the hard margin threshold and the probability likelihood within the sample point distribution. Sample points whose probability likelihood within the sample point distribution is lower than the hard margin threshold are retained as samples outside the virtual distribution.

10. The method according to claim 8 or 9, characterized in that, The method further includes: robustly fine-tuning the model based on virtual distribution out-of-samples, wherein, The model is trained using out-of-distribution sample discrimination loss to separate in-distribution and out-of-distribution samples; Using in-distribution class contrast loss to train the model makes the in-distribution classes more compact; Use cross-entropy loss to train the model and maintain the performance of the original classification task.

Images