A small sample multi-modal response mode space coordination anomaly detection method
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SANJIANG UNIVERSITY
- Filing Date
- 2026-04-24
- Publication Date
- 2026-07-14
Smart Images
Figure CN122391738A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and industrial anomaly detection, and in particular to an anomaly detection method based on spatial coordination of small-sample multimodal response patterns. Background Technology
[0002] With the continuous improvement of industrial automation, visual information-based anomaly detection technology is playing an increasingly important role in product quality inspection, defect identification, and equipment condition monitoring. In actual industrial inspection scenarios, due to the small number and diverse types of abnormal samples, it is difficult to cover all possible anomalies. Therefore, anomaly detection methods based on normal samples are more in line with practical application needs.
[0003] Currently, anomaly detection methods primarily rely on single-modal information for feature modeling, such as methods based on 2D image data or 3D point cloud data. While 2D image-based methods effectively extract texture and appearance information from target surfaces, they struggle to accurately depict the spatial structural features of the target. 3D point cloud-based methods can reflect the geometric structure of objects, but their ability to represent appearance anomalies such as changes in surface texture is limited. In real-world industrial inspection scenarios, anomalous regions often involve both structural and appearance changes. Relying solely on single-modal information is insufficient to comprehensively characterize anomalous features. Therefore, the gradual introduction of multimodal information for collaborative modeling has become an important development direction for anomaly detection technology.
[0004] To improve anomaly detection performance, researchers have recently begun exploring anomaly detection methods based on multimodal data. By fusing two-dimensional image information with three-dimensional structural information, these methods achieve a more comprehensive characterization of anomalous regions. However, most existing multimodal anomaly detection methods focus on independent modeling or simple fusion of features from different modalities, typically using a single anomaly score to determine the detection results. This makes it difficult to characterize the changing features of anomalies across different mechanistic dimensions. Furthermore, existing methods usually only model the feature space of normal samples, lacking the ability to continuously model the evolution of anomalous patterns. In industrial scenarios with limited initial sample sizes or continuously changing anomalous patterns, this can easily lead to a decline in detection performance.
[0005] Furthermore, in actual industrial inspection processes, the number of normal samples available in the initial stage of a system is usually limited, while abnormal patterns may continue to evolve with changes in the production environment. If the feature space is incrementally updated directly based on a single detection result, misjudged samples are easily introduced into the model space, leading to spatial structure shifts or even model degradation. Therefore, how to construct an anomaly detection modeling method that can stably evolve under small sample conditions has become one of the key problems that urgently need to be solved in the current field of multimodal anomaly detection.
[0006] Therefore, it is necessary to propose a multimodal anomaly detection method that can achieve decoupled modeling of anomaly mechanisms and support co-evolutionary updates of response mode space, so as to improve the anomaly detection capability and adaptability of the system in complex industrial scenarios. Summary of the Invention
[0007] This invention provides an anomaly detection method based on small-sample multimodal response pattern spatial collaboration. In order to solve the problem that the existing technology uses a single anomaly score to judge the detection results, it is difficult to characterize the changing characteristics of anomalies under different mechanism dimensions. Moreover, existing methods usually only model the feature space of normal samples and lack the ability to continuously model the evolution process of anomaly patterns. In industrial scenarios where the initial number of samples is limited or the anomaly patterns are constantly changing, the detection performance is prone to decline. Incremental updates of samples based on detection results can easily introduce misjudged samples into the model space, resulting in spatial structure shift or even model degradation.
[0008] To solve the above technical problems, the technical solution of the present invention is as follows:
[0009] A method for anomaly detection based on spatial coordination of small-sample multimodal response modes, characterized by: including...
[0010] Step 1. Obtain normal sample data, analyze and process the data, generate abnormal response pattern vectors containing three dimensions: local structural relationship disruption, cross-modal cooperative change mismatch, and overall joint distribution deviation, and construct a set of normal response pattern vectors;
[0011] Step 2. Construct a normal response region model based on the normal response pattern vector set;
[0012] Step 3. Obtain the data of the sample to be tested and generate the abnormal response pattern vector of the sample to be tested;
[0013] Step 4. Calculate the degree of deviation of the abnormal response pattern vector of the test sample relative to the normal response region model space. Determine whether the test sample is abnormal based on the degree of deviation. If abnormal, construct an abnormal response pattern vector set based on the abnormal response pattern vector of the test sample, construct an abnormal response region model based on the abnormal response pattern vector set, and gradually update the abnormal response region model as the detection process progresses. If normal, update the abnormal response pattern vector of the test sample to the normal response pattern vector set, and update the normal response region model.
[0014] Furthermore, the process of generating anomaly response pattern vectors specifically involves: extracting apparent and geometric features from the data, including two-dimensional image data and corresponding three-dimensional structural data; and calculating local structural relationship disruption response values, cross-modal cooperative variation mismatch response values, and overall joint distribution deviation response values based on the apparent and geometric features, thereby generating anomaly response pattern vectors containing three dimensions: local structural relationship disruption, cross-modal cooperative variation mismatch, and overall joint distribution deviation. .
[0015] Furthermore, the calculation of the local structural relationship failure response value is as follows:
[0016]
[0017] in, is the response value for local structural relationship failure; N is the total number of local locations; Let i be the feature vector at the i-th position; For feature vectors Local neighborhood aggregation features;
[0018] The cross-modal cooperative variation mismatch response value is calculated as follows:
[0019]
[0020] in, This represents the mismatch response value for cross-modal cooperative variation; The degree of deviation in cross-modal consistency at the pixel level; The weighting coefficient represents the pixel-level deviation. The degree of deviation from regional cross-modal consistency; Weighting coefficients for regional deviations;
[0021] The overall joint distribution deviation response value is calculated as follows:
[0022]
[0023] in, This represents the overall joint distribution offset response value; This represents the global feature vector of the sample to be detected. The center of the normal sample distribution; The normal distribution covariance matrix; Let be the Mahalanobis distance norm.
[0024] Furthermore, the degree of deviation is used to determine whether the sample under test is abnormal, specifically:
[0025] Spatial deviation is calculated using a distance metric function:
[0026]
[0027] Among them, the abnormal response pattern vector of the sample to be detected Indicates the degree of spatial deviation. Represents the distance metric function. This is a normal response region model;
[0028] The distance metric function can be calculated using one or a combination of Euclidean distance, Mahalanobis distance, or other spatial distribution deviation metric methods.
[0029] When the spatial deviation meets the requirements ,in The threshold indicates the anomaly detection threshold. If the sample to be detected is an anomaly sample, it is considered a normal sample otherwise.
[0030] Furthermore, anomaly detection methods also include:
[0031] When an abnormal response region model exists, and the sample to be tested is determined to be abnormal, the offset vector of the sample to be tested relative to the center of the normal response region model and the center of the abnormal response region model is calculated. Based on the contribution ratio of the offset vector of the center of the normal response region model and the offset vector of the center of the abnormal response region model in each dimension, dual attribution information is output.
[0032] When there is no abnormal response region model, and the test sample is judged to be abnormal, the offset vector of the test sample relative to the center of the normal response region model is calculated. Based on the contribution ratio of the offset vector of the center of the normal response region model in each dimension, the single attribution information is output.
[0033] Furthermore, the attribution information is output, specifically: the offset vector of the test sample relative to the model center of the normal response region. ;
[0034] in, This represents the vector of abnormal response patterns of the sample to be detected. The center of the normal sample distribution;
[0035] Offset vector of the test sample relative to the center of the abnormal response region: ;
[0036] in, The distribution center of abnormal samples is defined; the offset degree in each dimension is normalized to obtain the contribution ratio of the offset vector of the normal response region model center and the offset vector of the normal response region model center in each dimension:
[0037]
[0038] in, The offset vector of the test sample relative to the center of the normal response region The component value in the k-th dimension;
[0039] The offset vector of the test sample relative to the center of the normal response region The contribution ratio in the k-th dimension;
[0040] The offset vector of the test sample relative to the center of the normal response region The sum of the absolute values of all K dimensional components;
[0041] k is the k-th dimension; K is the total number of dimensions;
[0042]
[0043] in, The offset vector of the sample to be tested relative to the center of the abnormal response region. The component value in the k-th dimension;
[0044] The offset vector of the sample to be tested relative to the center of the abnormal response region. The contribution ratio in the k-th dimension;
[0045] The offset vector of the sample to be tested relative to the center of the abnormal response region. The sum of the absolute values of all K dimensional components;
[0046] The dominant offset intensity index for each dimension is calculated as follows:
[0047]
[0048] in, The larger the value, the more dominant that dimension is, indicating that the sample deviates from the normal range. The larger the value, the more closely the sample under test reflects an abnormal pattern;
[0049] Dominant deviation strength index for all dimensions Sort the data and construct the dominant dimension set. The abnormal mechanism type and its dominant deviation strength corresponding to each dimension in the dominant dimension set are output as the attribution information of the abnormal mechanism in descending order of dominant deviation strength.
[0050] Furthermore, construct the dominant dimension set, including at least one of the following methods:
[0051] (1) Select The largest single dimension is designated as the sole dominant dimension.
[0052] (2) Select The top two or three largest dimensions are used as the joint dominant dimensions;
[0053] (3) Select the one that satisfies All dimensions exceeding the preset intensity threshold are considered as joint dominant dimensions.
[0054] Furthermore, the construction methods for the normal response region model include at least one of the following:
[0055] (1) Based on the statistical distribution structure, the region modeling method constructs a probability distribution model to describe the center location and distribution range of the normal response region by estimating the statistical distribution parameters of the normal response mode vector in the abnormal response mode space;
[0056] (2) Based on the prototype library structure, the typical response structure in the normal response region is discretized by the normal response pattern vector set, thereby forming a normal response region model;
[0057] (3) The response region modeling method based on regional boundary constraints determines the spatial coverage of the normal response region by constructing the boundary constraint relationship of the normal response mode vector set in the abnormal response mode space;
[0058] The construction method of the abnormal response region model is the same as that of the normal response region model.
[0059] Furthermore, when a sample to be tested is determined to be a normal sample or an abnormal sample, its position is used to determine the abnormal response pattern vector of the sample to be tested, and to determine whether it can be used to update the corresponding set.
[0060] The confidence level is calculated as follows:
[0061]
[0062] in, This indicates an update to the confidence level. Indicators representing the degree of spatial deviation This indicates the stability index of the dominant deviation direction. Indicates the consistency index of historical testing. , Indicates the weighting coefficient;
[0063] when If the result is greater than or equal to the update confidence threshold, the abnormal response pattern vector corresponding to the test sample is added to the normal response region model or the abnormal response region model for spatial update; otherwise, the test sample is marked as a sample to be confirmed; when the number of times the sample to be confirmed appears repeatedly in subsequent detections reaches the preset number threshold, or is confirmed as a real abnormal sample by manual annotation, the abnormal response pattern vector of the sample to be confirmed is then used for the spatial update of the corresponding normal response region model or the abnormal response region model.
[0064] The present invention has the following beneficial effects:
[0065] This invention generates anomaly response pattern vectors by mapping the sample data to be detected into an anomaly response pattern space composed of three anomaly mechanism response dimensions: local structural relationship disruption, cross-modal cooperative change mismatch, and overall joint distribution deviation. This achieves a physical decoupling representation of different anomaly formation mechanisms and overcomes the problem that a single anomaly score is difficult to characterize the changing features of anomalies under multiple mechanism dimensions.
[0066] A normal response region model is constructed based on the abnormal response pattern vectors corresponding to normal samples to determine anomalies in the samples to be detected. Simultaneously, an abnormal response region model is gradually constructed based on the abnormal response pattern vectors corresponding to the confirmed abnormal samples during the detection process, forming a dual-space collaborative response structure. The abnormal response pattern vectors of the determined samples to be detected are then updated to the sets corresponding to the normal response region model or the abnormal response region model, respectively. This allows the normal response region model or the abnormal response region model to continuously evolve as the samples to be detected after anomaly detection become new samples, transforming from static modeling to dynamic evolutionary modeling. This improves the anomaly modeling capability under small sample conditions and enhances the model's adaptability to scenarios with continuously changing anomaly patterns. Compared to traditional methods that rely solely on normal sample modeling, this approach can more accurately characterize the spatial boundaries of anomaly patterns, improving the accuracy and stability of anomaly detection.
[0067] For samples identified as abnormal, their offset vectors relative to the model center of the normal response region and the model center of the abnormal response region are calculated respectively. The contribution ratio of the offset vectors in each response dimension of the abnormal mechanism is jointly analyzed to determine the dominant mechanism causing the abnormality. This expands the abnormality detection results from "whether it is abnormal" to "explanation of the abnormal mechanism", improving the interpretability of the detection results and the ability to assist decision-making.
[0068] Furthermore, by introducing a confidence assessment mechanism based on the degree of spatial deviation, the stability of the dominant deviation direction, and the consistency of historical detection, the samples intended for model updates are screened. Only high-confidence samples are used to update the corresponding normal response region model or abnormal response region model. This effectively avoids the spatial structure shift and model degradation caused by misjudged samples introduced by direct incremental updates under small sample conditions, and improves the stability and reliability of the evolution modeling process of the normal response region model and the abnormal response region model. Attached Figure Description
[0069] Figure 1 This is a schematic flowchart of the overall detection method according to an embodiment of the present invention;
[0070] Figure 2 This is a schematic diagram of the abnormal response pattern vector construction process according to an embodiment of the present invention;
[0071] Figure 3 This is a schematic diagram of the reasoning process for judging anomalies in the test samples and analyzing spatial deviations according to an embodiment of the present invention.
[0072] Figure 4 This is a schematic diagram illustrating the dual-space collaborative incremental update process for the normal response region model and the abnormal response region model in an embodiment of the present invention. Detailed Implementation
[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1-4 The present invention will be further described in detail with reference to specific embodiments.
[0074] like Figure 1 As shown, this embodiment provides an anomaly detection method based on spatial coordination of small-sample multimodal response modes, including:
[0075] Step 1. Construct an anomaly response pattern space containing multiple anomaly mechanisms to decouple response dimensions, obtain normal sample data, generate anomaly response pattern vectors for normal samples, and construct a set of normal response pattern vectors;
[0076] Among them, the anomaly mechanism decoupling response dimension refers to the response dimension used to characterize different types of abnormal changes or abnormal action mechanisms; each response dimension is relatively independent or focuses on different abnormal features, and is used to describe the abnormal response pattern of the sample from different perspectives.
[0077] Anomaly response pattern space refers to a multidimensional response representation space composed of multiple anomaly mechanisms decoupled from response dimensions. It is used to uniformly represent the response state of a sample under the action of different anomaly mechanisms and to characterize the deviation features of the sample relative to normal and anomaly modes.
[0078] An anomaly response pattern vector is a vector composed of the response values of each anomaly mechanism decoupling response dimension in the anomaly response pattern space, used to characterize the anomaly response pattern of the sample.
[0079] Step 2. Construct a normal response region model based on the normal response pattern vector set, which is used to divide the normal response region in the abnormal response pattern space;
[0080] The normal response region model is a region representation model constructed based on the set of response pattern vectors of normal samples, used to describe the distribution characteristics of normal response regions in the abnormal response pattern space.
[0081] Step 3. Obtain the data of the sample to be tested and generate the abnormal response pattern vector of the sample to be tested.
[0082] Step 4. Calculate the degree of deviation of the abnormal response pattern vector of the test sample relative to the normal response region model space. Determine whether the test sample is abnormal based on the degree of deviation. If abnormal, construct an abnormal response pattern vector set based on the abnormal response pattern vector of the test sample, and construct an abnormal response region model based on the abnormal response pattern vector set to divide the abnormal response region in the abnormal response pattern space. The abnormal response region model is updated gradually as the detection process progresses. If normal, update the abnormal response pattern vector of the test sample to the normal response pattern vector set and update the normal response region model.
[0083] Among them, the abnormal response region model refers to a region representation model constructed based on the set of response pattern vectors of abnormal samples, which is used to describe the distribution characteristics of abnormal response regions in the abnormal response pattern space.
[0084] The anomaly detection method for the test sample is as follows: the test sample is mapped to the anomaly response pattern space to generate the anomaly response pattern vector of the test sample, and the spatial deviation of the test sample is analyzed and reasoned by combining the normal response region and the anomaly response region to realize anomaly detection and auxiliary discrimination of anomaly type. At the same time, after anomaly detection of the test sample, the stability and reliability of the dynamic evolution modeling process of the response pattern space are improved by combining the dual-space collaborative incremental update mechanism of the normal response region and the anomaly response region and introducing the update confidence constraint strategy.
[0085] A schematic diagram of the abnormal response pattern vector construction process is shown below. Figure 2 As shown in the diagram, the reasoning process for anomaly detection and spatial deviation analysis of the test sample is as follows: Figure 3 As shown in the diagram, the process of performing dual-space collaborative incremental updates on the normal response region model and the abnormal response region model is illustrated below. Figure 4 As shown.
[0086] Anomaly response pattern vector construction:
[0087] The process involves acquiring multimodal input data of samples, including two-dimensional image data and corresponding three-dimensional structural data. Feature information that can characterize different anomaly mechanisms is extracted from the multimodal input data and converted into multiple independent anomaly response dimensions. Finally, a unified anomaly response pattern vector is formed, providing a basic representation for subsequent response pattern spatial modeling and spatial deviation analysis.
[0088] The decoupling response dimension construction phase of the anomaly mechanism specifically includes the following steps:
[0089] Multimodal features are extracted to obtain multimodal input data of the samples, including two-dimensional image data and corresponding three-dimensional structural data. The apparent features and geometric features are obtained through corresponding feature extraction methods, so that the abnormal information under different modalities can be effectively represented at the feature level.
[0090] Specifically, let the modal features of a two-dimensional image be represented as:
[0091]
[0092] The three-dimensional structural modal characteristics are represented as follows:
[0093]
[0094] Where N is the total number of local locations.
[0095] One specific embodiment: First, acquire the two-dimensional RGB image data of the sample to be detected and its corresponding three-dimensional point cloud data. Based on the camera intrinsic and extrinsic parameter matrices of the image acquisition device, establish a spatial mapping relationship between the two-dimensional image pixel coordinate system and the three-dimensional point cloud spatial coordinate system, and accordingly complete the local positional correspondence between the two-dimensional image plane and the three-dimensional space. Subsequently, feature extraction is performed on the two-dimensional RGB image data and the three-dimensional point cloud data respectively to obtain the two-dimensional image modal features and three-dimensional structural modal features at each corresponding local position. Represents the space-aligned first... Local location unit, This represents the total number of local location units participating in anomaly response modeling after spatial alignment is completed. Indicates the first Two-dimensional image modal features corresponding to local locations Indicates the first Three-dimensional structural modal features corresponding to local locations. A two-dimensional image modal feature set can be constructed using the above method. With three-dimensional structural modal feature set This provides a unified multimodal feature foundation for subsequent decoupling response modeling of anomaly mechanisms.
[0096] Multiple anomaly mechanism decoupled response dimensions are constructed. Based on the extracted multimodal features, multiple independent anomaly response dimensions are built from different anomaly formation mechanisms, including a local structural relationship disruption response dimension, a cross-modal cooperative change mismatch response dimension, and an overall joint distribution deviation response dimension. Each anomaly response dimension corresponds to a different type of anomaly mode. For example, the presence of local structural relationship disruption is determined by calculating the Euclidean distance between the test area features and its spatial neighborhood background features; the presence of cross-modal cooperative change mismatch is determined by calculating the mutual information of the two-dimensional image features and the three-dimensional point cloud features after spatial reprojection; and the presence of overall joint distribution deviation is determined by calculating the Mahalanobis distance of the sample features under the global Gaussian distribution. This transforms the original single anomaly scoring problem into a multi-mechanism decoupled anomaly response representation problem.
[0097] The local structural relationship disruption response dimension is used to characterize the degree of change in the consistency of the local neighborhood structure of a sample.
[0098]
[0099] in, This is the response value for local structural relationship disruption, used to measure whether local spatial structural dependencies have been broken; for example, abnormal texture arrangement, surface structure perturbation, and abnormal neighborhood geometry. RGB neighborhood can be represented by gradient changes, local texture differences, feature mean or variance, etc., while 3D neighborhood can be represented by curvature changes, normal changes, depth changes, etc. N is the total number of local locations; Let i be the feature vector at the i-th position; For feature vectors The local neighborhood aggregation feature.
[0100] For describing local neighborhood features, RGB neighborhoods can be characterized by combining information such as gradient changes, local texture differences, feature mean or variance, while 3D neighborhoods can be characterized by combining information such as curvature changes, normal changes, and depth changes.
[0101] In one specific embodiment, for the construction of local neighborhood aggregation features, for the first... For each local location, first determine its local neighborhood set. The local neighborhood can be determined based on spatial proximity relationships in a two-dimensional image plane or geometric proximity relationships in a three-dimensional point cloud. Then, the features at each location within the neighborhood are aggregated to obtain the first... Neighborhood aggregation features of local locations One feasible implementation method is:
[0102]
[0103] in, Indicates the first The set of neighborhood locations corresponding to each local location This represents the number of locations within the neighborhood. If the... If a local location maintains normal structural consistency with its neighborhood, then the location feature... Features of neighborhood aggregation The differences between the two are relatively small; however, if the local texture arrangement, surface morphology, or neighborhood geometry is disrupted, the differences increase. By statistically analyzing the above differences at all local locations, the response value of local structural relationship disruption can be obtained. .
[0104] The cross-modal cooperative variation mismatch response dimension is used to characterize the degree of change in semantic correspondence between different modalities:
[0105]
[0106]
[0107]
[0108] in, This represents the mismatch response value for cross-modal cooperative variation; The degree of deviation in cross-modal consistency at the pixel level; The weighting coefficient represents the pixel-level deviation. The degree of deviation from regional cross-modal consistency; Weighting coefficients for regional deviations; It is a pixel-level cross-modal mapping function; This is a regional-level cross-modal mapping function.
[0109] The pixel-level cross-modal mapping function With regional cross-modal mapping function It can be trained based on normal samples. Specifically, during the training phase, two-dimensional image features and three-dimensional structural features with established spatial correspondences are extracted from normal samples. Using the two-dimensional image modal features as input and the three-dimensional structural modal features of the corresponding position or region as the supervision target, the cross-modal mapping function is optimized and trained to learn the stable correspondence between different modalities under normal conditions. After training, during the detection phase, the mapping function is used to perform cross-modal feature mapping on the samples to be detected, and the degree of mismatch in cross-modal cooperative change is characterized by the difference between the mapping result and the actual corresponding modal features.
[0110] By using multi-granular cross-modal consistency, the degree of change in local semantic correspondence and regional structural correspondence between different modalities is jointly characterized. This is used to measure whether image information and structural information still maintain a cooperative and consistent relationship, such as abnormal patterns like appearance changes but structure does not change or structure changes but texture does not change.
[0111] The overall joint distribution deviates from the response dimension, used to characterize the degree of deviation of the overall feature distribution of the sample from the normal pattern space:
[0112]
[0113] in, The overall joint distribution shift response value is used to measure whether the overall statistical characteristics of the sample deviate from the normal pattern space; This is the multimodal joint global feature vector of the sample to be detected; The center of the normal sample distribution; The normal distribution covariance matrix; Let be the Mahalanobis distance norm.
[0114] The multimodal joint global feature vector F is composed of two-dimensional image modal features at each local location. With three-dimensional structural modal features It is obtained by fusion.
[0115] A feasible integration method:
[0116]
[0117] here Indicates feature splicing, This represents the joint representation of two-dimensional image information and three-dimensional structural information. All joint features collectively describe the joint feature distribution across the two-dimensional image modality and the three-dimensional structural modality, thus characterizing the overall multimodal statistical distribution properties of the samples. During the training phase, a normal multimodal joint global feature distribution is constructed based on the multimodal joint global feature vector of normal samples. The center of the normal joint distribution can be obtained by calculating the mean. And obtain the corresponding covariance matrix. For the sample to be detected, construct its multimodal joint global feature vector. Calculate the degree of deviation of the sample to be tested from the normal joint distribution. .
[0118] Based on multiple anomaly mechanism decoupling response dimensions, this paper unifies and represents these dimensions, combining the response results of the same sample across different anomaly mechanism decoupling response dimensions into a single anomaly response pattern vector. This vector characterizes the sample's position in the anomaly response pattern space. Through this vector, a unified expression of sample anomaly patterns can be achieved, providing a fundamental spatial representation for subsequent normal response region modeling, spatial deviation analysis, and anomaly type auxiliary discrimination.
[0119] Anomaly response pattern vector: This enables the transformation from a single anomaly score to a multi-mechanism anomaly response representation, improving the expressive power and interpretability of anomaly patterns.
[0120] Construct normal response region model and abnormal response region model
[0121] The normal response region model is used to characterize the distribution structure of normal samples in the abnormal response pattern space, providing a spatial reference for subsequent anomaly deviation analysis. The specific construction is as follows:
[0122] Collect data from multiple normal samples and construct their corresponding abnormal response pattern vectors to form a set of normal response pattern vectors. , is used to describe the distribution structure of normal samples in the abnormal response pattern space; where M is the number of normal samples.
[0123] Based on the set of normal response pattern vectors, the distribution structure of normal samples in the abnormal response pattern space is modeled, and a normal response region model is constructed. It is used to characterize the range of the normal response area.
[0124] The construction methods for a normal response region model include at least one of the following:
[0125] (1) Based on the statistical distribution structure, the region modeling method is to construct a probability distribution model to describe the center location and distribution range of the normal response region by estimating the statistical distribution parameters of the normal response mode vector in the abnormal response mode space; in a specific implementation, the mean and covariance statistics of the normal response mode vector set can be performed to obtain the distribution center and distribution range parameters of the normal response region, thereby constructing a statistical distribution model of the normal response region.
[0126] (2) Based on the prototype library structure, the typical response structure in the normal response region is discretely represented by the normal response pattern vector set, thereby forming a normal response region model; where the typical response structure refers to the representative response structure in the normal response pattern vector set that can reflect the main distribution characteristics of the normal response region.
[0127] In one specific implementation, a representative subset that covers the main distribution characteristics of the original response patterns can be selected from the set of normal response pattern vectors. The response pattern vectors in this representative subset are then used as prototype vectors to construct a normal response prototype library. These prototype vectors can be determined through cluster center selection, similarity filtering, or coverage constraints, thereby representing the typical response structure in the normal response region as a discrete prototype set. The prototype vectors exhibit good dispersion and coverage in the abnormal response pattern space, thus enabling the representation of the main distribution characteristics in the normal response region with a relatively small number of vectors.
[0128] (3) The response region modeling method based on regional boundary constraints determines the spatial coverage of the normal response region by constructing the boundary constraint relationship of the normal response mode vector set in the abnormal response mode space. In one specific implementation, an envelope boundary, hypersphere boundary, hyperplane boundary, or confidence boundary can be constructed based on the distribution range of the normal response mode vector set in the abnormal response mode space to limit the spatial coverage of the normal response region.
[0129] Output a normal response region model to describe the structure of the normal response region and serve as a reference benchmark for judging the degree of sample abnormality in the subsequent spatial deviation analysis inference stage.
[0130] The anomaly response region model is constructed as follows: when an anomaly sample is detected, the anomaly response pattern vector corresponding to the anomaly sample is recorded, and a set of anomaly response pattern vectors is constructed. An anomaly response region model is constructed based on the set of anomaly response pattern vectors. .
[0131] The construction method of the abnormal response region model is the same as that of the normal response region model.
[0132] Anomaly detection, spatial deviation analysis, and reasoning are performed on the test samples.
[0133] Acquire multimodal input data of the sample to be detected and generate abnormal response pattern vectors of the sample to be detected. The spatial deviation of the abnormal response pattern vector from the normal response region model is calculated to determine whether the sample is abnormal.
[0134] Spatial deviation is calculated using a distance metric function:
[0135]
[0136] in, Indicates the degree of spatial deviation. This represents the distance metric function.
[0137] The distance metric function can be calculated using Euclidean distance, Mahalanobis distance, or one or a combination of other spatial distribution deviation metrics.
[0138] When the spatial deviation meets the requirements ,in The threshold value represents the anomaly detection threshold. If a sample is deemed an anomaly, it is considered an anomaly; otherwise, it is considered a normal sample. Through the above calculation of spatial deviation, anomaly detection and discrimination based on the spatial structure of anomaly response patterns is achieved.
[0139] After calculating the degree of spatial deviation, the offset vector of the abnormal response pattern vector of the test sample relative to the normal response region and the abnormal response region is further calculated. This is used to analyze the contribution ratio of the sample in each abnormal mechanism response dimension, thereby realizing the auxiliary discrimination of abnormal mechanisms.
[0140] Construct the offset vector of the test sample relative to the center of the normal response region. Normal offset vector: ;
[0141] in, This represents the vector of abnormal response patterns of the sample to be detected. The center of the normal sample distribution;
[0142] Construct the offset vector of the test sample relative to the center of the abnormal response region. Abnormal offset vector: ;
[0143] in, As the center of the abnormal sample distribution;
[0144] After normalizing the degree of offset in each dimension, we obtain the contribution ratios of the abnormal mechanism response dimension to the normal offset vector and the abnormal mechanism response dimension to the abnormal offset vector, respectively:
[0145]
[0146] in, The offset vector of the test sample relative to the center of the normal response region The component value in the k-th dimension;
[0147] The offset vector of the test sample relative to the center of the normal response region The contribution ratio in the k-th dimension;
[0148] The offset vector of the test sample relative to the center of the normal response region The sum of the absolute values of all K dimensional components;
[0149] k is the k-th dimension; K is the total number of dimensions;
[0150]
[0151] in, The offset vector of the sample to be tested relative to the center of the abnormal response region. The component value in the k-th dimension;
[0152] The offset vector of the sample to be tested relative to the center of the abnormal response region. The contribution ratio in the k-th dimension;
[0153] The offset vector of the sample to be tested relative to the center of the abnormal response region. The sum of the absolute values of all K dimensional components.
[0154] By jointly analyzing the contribution ratio of the normal response region offset vector and the abnormal response region offset vector in each abnormal mechanism response dimension, the dominant deviation direction of the sample is determined.
[0155] To more clearly differentiate the degree of support from different dimensions for the dominant deviation direction, a "dominant deviation strength" index can be constructed for each dimension. One directly implementable approach is:
[0156] Calculate the dominant offset intensity index for each dimension:
[0157]
[0158] in, The larger the value, the more dominant that dimension is, indicating that the sample deviates from the normal range. The larger the value, the more closely the sample in question reflects an abnormal pattern; after multiplying the two, The larger the value, the more likely that the dimension is to be the key dimension that determines the dominant deviation direction. This is because if a dimension deviates greatly from the normal but is still far from the abnormal, it can only indicate that it is "abnormal" but not necessarily that it has "devied in which abnormal direction". If a dimension is both significantly far from the normal and significantly closer to the abnormal, then this dimension is a very strong dominant deviation dimension.
[0159] Dominant deviation strength index for all dimensions Sort the data and construct the dominant dimension set. The abnormal mechanism type and its dominant deviation strength corresponding to each dimension in the dominant dimension set are output as the attribution information of the abnormal mechanism in descending order of dominant deviation strength.
[0160] Constructing the dominant dimension set can be done in at least one of the following ways:
[0161] (1) Select The largest single dimension is designated as the sole dominant dimension.
[0162] (2) Select The top two or three largest dimensions are used as the joint dominant dimensions;
[0163] (3) Select the one that satisfies All dimensions exceeding the preset intensity threshold are considered as joint dominant dimensions.
[0164] After this step, it can be assumed that the dominant deviation direction of the sample is mainly determined by the set. These abnormal mechanisms and response dimensions are jointly determined.
[0165] The determination of the dominant deviation direction can be divided into two cases:
[0166] If only one dominant dimension is ultimately selected If a dimension exhibits the most significant deviation in both "far from normal" and "close to abnormal," then the dominant deviation direction of the test sample is determined as the abnormal mechanism response direction corresponding to that dimension.
[0167] If multiple dominant dimensions are ultimately selected, the direction of the dominant deviation is not determined by a single dimension, but rather by the combined responses of multiple anomaly mechanisms. In this case, instead of outputting only one dimension, all dimensions are retained and processed according to... Sort by size. Which one? The larger the dimension, the stronger its influence on the dominant deviation direction. This results in a "joint dominant deviation direction," rather than a single direction.
[0168] A dual-space collaborative incremental update is performed on the normal response region model and the abnormal response region model. This is used to dynamically update the response pattern space structure based on the test samples after confidence comparison during the detection process. By constructing the abnormal response region model and collaboratively adjusting the normal response region model, the response pattern space can continuously approximate the real pattern structure, thereby realizing the dynamic evolution modeling capability of the response pattern space; specifically:
[0169] When a sample is determined to be a normal sample, its abnormal response pattern vector is added to the normal response pattern vector set, and the normal response region model is updated. When a sample is determined to be an abnormal sample, its abnormal response pattern vector is added to the abnormal response pattern vector set, and the abnormal response region model is updated. Through this update mechanism, the normal response region model and the abnormal response region model are updated collaboratively.
[0170] The test samples that have undergone anomaly detection are used as new samples. By continuously introducing the anomaly response pattern vectors of the new samples, the normal response region model and the anomaly response region model are iteratively updated. This makes the spatial distribution structure of the anomaly response pattern gradually approximate the distribution structure of the real sample pattern. As the number of samples increases, the spatial structure of the anomaly response pattern can gradually evolve from the initial small sample modeling state to a stable statistical structure, thereby improving the adaptability to complex anomaly patterns and environmental changes, and realizing dynamic evolution modeling of the anomaly response pattern space.
[0171] Before performing an anomaly response pattern space update, the anomaly response pattern vector of the newly added sample is updated with confidence to avoid misjudged samples directly participating in the space update and causing the response pattern space structure to shift.
[0172] Anomaly response pattern vector for new samples Calculate its update confidence:
[0173]
[0174] in, This indicates an update to the confidence level. Indicators representing the degree of spatial deviation This indicates the stability index of the dominant deviation direction. Indicates the consistency index of historical testing. , This represents the weighting coefficient. When C is greater than or equal to the update confidence threshold, the abnormal response pattern vector corresponding to the newly added sample is added to the normal response region model or the abnormal response region model for spatial update. Otherwise, the newly added sample is marked as a sample to be confirmed. When the number of times the sample to be confirmed appears in subsequent detections reaches a preset threshold, or is confirmed as a true abnormal sample by manual annotation, the abnormal response pattern vector of the sample to be confirmed is then used for spatial update of the corresponding normal response region model or the abnormal response region model. By evaluating the confidence of the abnormal response pattern vector of the newly added sample, the stability and reliability of the abnormal response pattern spatial dynamic evolution modeling process can be improved, and the adaptive modeling capability under small sample conditions can be enhanced.
[0175] In the initial stage, since it is often difficult to obtain abnormal samples for constructing the abnormal response region model, a normal response region model can be constructed based solely on normal samples, and this normal response region model serves as the initial reference structure for the response pattern space. During operation, when an anomaly is detected, and the update confidence of the abnormal response pattern vector of the sample to be tested is greater than or equal to the update confidence threshold, the corresponding abnormal response pattern vector is recorded, and the abnormal response region model is gradually constructed, so that the abnormal response pattern space gradually evolves from an initial single-space structure to a dual-space collaborative structure composed of normal and abnormal response regions. Furthermore, when the number of abnormal samples is small, the abnormal response region model can be initialized using a region modeling method based on a prototype library structure, that is, using a single or a small number of abnormal response pattern vectors as abnormal response prototypes to form the initial representation of the abnormal response region. As the number of abnormal samples gradually increases, the abnormal response region model can be further updated and improved using a statistical distribution structure or region boundary constraint method according to the size and distribution characteristics of the abnormal response pattern vector set.
[0176] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.
Claims
1. A method for anomaly detection based on spatial coordination of small-sample multimodal response modes, characterized in that: include: Step 1. Obtain normal sample data, analyze and process the data, generate abnormal response pattern vectors containing three dimensions: local structural relationship disruption, cross-modal cooperative change mismatch, and overall joint distribution deviation, and construct a set of normal response pattern vectors; Step 2. Construct a normal response region model based on the normal response pattern vector set; Step 3. Obtain the data of the sample to be tested and generate the abnormal response pattern vector of the sample to be tested; Step 4. Calculate the degree of deviation of the abnormal response pattern vector of the test sample relative to the normal response region model space. Determine whether the test sample is abnormal based on the degree of deviation. If abnormal, construct an abnormal response pattern vector set based on the abnormal response pattern vector of the test sample, construct an abnormal response region model based on the abnormal response pattern vector set, and gradually update the abnormal response region model as the detection process progresses. If normal, update the abnormal response pattern vector of the test sample to the normal response pattern vector set, and update the normal response region model.
2. The anomaly detection method based on spatial coordination of small-sample multimodal response modes according to claim 1, characterized in that: The process of generating anomaly response pattern vectors is as follows: Data includes two-dimensional image data and corresponding three-dimensional structural data; appearance and geometric features are extracted. Based on these features, response values for local structural relationship disruption, cross-modal cooperative variation mismatch, and overall joint distribution deviation are calculated, generating anomaly response pattern vectors containing these three dimensions. .
3. The anomaly detection method based on spatial coordination of small-sample multimodal response modes according to claim 2, characterized in that: The calculation of the local structural relationship failure response value is as follows: in, is the response value for local structural relationship failure; N is the total number of local locations; Let i be the feature vector at the i-th position; For feature vectors Local neighborhood aggregation features; The cross-modal cooperative variation mismatch response value is calculated as follows: in, This represents the mismatch response value for cross-modal cooperative variation; The degree of deviation in cross-modal consistency at the pixel level; The weighting coefficient represents the pixel-level deviation. The degree of deviation from regional cross-modal consistency; Weighting coefficients for regional deviations; The overall joint distribution deviation response value is calculated as follows: in, This represents the overall joint distribution offset response value; This represents the global feature vector of the sample to be detected. The center of the normal sample distribution; The normal distribution covariance matrix; Let be the Mahalanobis distance norm.
4. The anomaly detection method based on spatial coordination of small-sample multimodal response modes according to claim 1, characterized in that: The degree of deviation is used to determine whether the sample under test is abnormal. Specifically: Spatial deviation is calculated using a distance metric function: Among them, the abnormal response pattern vector of the sample to be detected Indicates the degree of spatial deviation. Represents the distance metric function. This is a normal response region model; The distance metric function can be calculated using one or a combination of Euclidean distance, Mahalanobis distance, or other spatial distribution deviation metric methods. When the spatial deviation meets the requirements ,in The threshold indicates the anomaly detection threshold. If the sample to be detected is an anomaly sample, it is considered a normal sample otherwise.
5. The anomaly detection method based on spatial coordination of small-sample multimodal response modes according to claim 1, characterized in that: Anomaly detection methods also include: When an abnormal response region model exists, and the sample to be tested is determined to be abnormal, the offset vector of the sample to be tested relative to the center of the normal response region model and the center of the abnormal response region model is calculated. Based on the contribution ratio of the offset vector of the center of the normal response region model and the offset vector of the center of the abnormal response region model in each dimension, dual attribution information is output. When there is no abnormal response region model, and the test sample is judged to be abnormal, the offset vector of the test sample relative to the center of the normal response region model is calculated. Based on the contribution ratio of the offset vector of the center of the normal response region model in each dimension, the single attribution information is output.
6. The anomaly detection method based on spatial coordination of small-sample multimodal response modes according to claim 5, characterized in that: The output attribution information is specifically: the offset vector of the test sample relative to the model center of the normal response region. ; in, This represents the vector of abnormal response patterns of the sample to be detected. The center of the normal sample distribution; Offset vector of the test sample relative to the center of the abnormal response region: ; in, The distribution center of abnormal samples is defined; the offset degree in each dimension is normalized to obtain the contribution ratio of the offset vector of the normal response region model center and the offset vector of the normal response region model center in each dimension: in, The offset vector of the test sample relative to the center of the normal response region The component value in the k-th dimension; The offset vector of the test sample relative to the center of the normal response region The contribution ratio in the k-th dimension; The offset vector of the test sample relative to the center of the normal response region The sum of the absolute values of all K dimensional components; k is the k-th dimension; K is the total number of dimensions; in, The offset vector of the sample to be tested relative to the center of the abnormal response region. The component value in the k-th dimension; The offset vector of the sample to be tested relative to the center of the abnormal response region. The contribution ratio in the k-th dimension; The offset vector of the sample to be tested relative to the center of the abnormal response region. The sum of the absolute values of all K dimensional components; The dominant offset intensity index for each dimension is calculated as follows: in, The larger the value, the more dominant that dimension is, indicating that the sample deviates from the normal range. The larger the value, the more closely the sample under test reflects an abnormal pattern; Dominant deviation strength index for all dimensions Sort the data and construct the dominant dimension set. The abnormal mechanism type and its dominant deviation strength corresponding to each dimension in the dominant dimension set are output as the attribution information of the abnormal mechanism in descending order of dominant deviation strength.
7. The anomaly detection method based on spatial coordination of small-sample multimodal response modes according to claim 6, characterized in that: Constructing the dominant dimension set can be done in at least one of the following ways: (1) Select The largest single dimension is designated as the sole dominant dimension. (2) Select The top two or three largest dimensions are used as the joint dominant dimensions; (3) Select the one that satisfies All dimensions exceeding the preset intensity threshold are considered as joint dominant dimensions.
8. The anomaly detection method based on spatial coordination of small-sample multimodal response modes according to claim 1, characterized in that: The construction methods for a normal response region model include at least one of the following: (1) Based on the statistical distribution structure, the region modeling method constructs a probability distribution model to describe the center location and distribution range of the normal response region by estimating the statistical distribution parameters of the normal response mode vector in the abnormal response mode space; (2) Based on the prototype library structure, the typical response structure in the normal response region is discretized by the normal response pattern vector set, thereby forming a normal response region model; (3) The response region modeling method based on regional boundary constraints determines the spatial coverage of the normal response region by constructing the boundary constraint relationship of the normal response mode vector set in the abnormal response mode space; The construction method of the abnormal response region model is the same as that of the normal response region model.
9. The anomaly detection method based on spatial coordination of small-sample multimodal response modes according to claim 1, characterized in that: When a sample to be tested is determined to be a normal sample or an abnormal sample, its confidence level is calculated to judge the abnormal response pattern vector of the sample to determine whether it can be used to update the corresponding set. The confidence level is calculated as follows: in, This indicates an update to the confidence level. Indicators representing the degree of spatial deviation This indicates the stability index of the dominant deviation direction. Indicates the consistency index of historical testing. , Indicates the weighting coefficient; when If the result is greater than or equal to the update confidence threshold, the abnormal response pattern vector corresponding to the test sample is added to the normal response region model or the abnormal response region model for spatial update; otherwise, the test sample is marked as a sample to be confirmed; when the number of times the sample to be confirmed appears repeatedly in subsequent detections reaches the preset number threshold, or is confirmed as a real abnormal sample by manual annotation, the abnormal response pattern vector of the sample to be confirmed is then used for the spatial update of the corresponding normal response region model or the abnormal response region model.