Abnormality detection method and related equipment

Abstract

The invention provides an anomaly detection method and related equipment. The method comprises the following steps: modeling a text prompt, and determining a text prompt corresponding to an obtained source domain data set; based on the text prompt, a first loss function is obtained through calculation according to the minimum distance between an anchor point of the text prompt and the most difficult negative sample; aligning the image features corresponding to the source domain data set with the text prompt, and calculating to obtain a second loss function; introducing an adaptive focusing loss mechanism to pay attention to samples difficult to classify, and calculating to obtain a third loss function; pixel-level optimization is carried out on the abnormal segmentation task, and a fourth loss function is obtained through calculation; training the model by using the loss function to obtain a trained anomaly detection model; and performing anomaly detection on the target domain data set by using the trained anomaly detection model to obtain an anomaly detection result. The embodiment of the invention realizes efficient cross-domain anomaly detection, adapts to multiple scenes, does not need target domain labeling, and enhances the model detection precision.

Description

Technical Field

[0001] This application relates to the field of anomaly detection technology, and in particular to an anomaly detection method and related equipment. Background Technology

[0002] With the development of industrial automation and intelligence, anomaly detection has become a crucial means of ensuring the safety of industrial equipment and improving production efficiency. In industrial scenarios, anomaly detection typically requires the analysis of massive amounts of industrial data to identify potential faults or abnormal states. This data may originate from different industrial equipment, environmental conditions, and application scenarios, exhibiting significant diversity and complexity. In recent years, deep learning-based anomaly detection methods have gradually become a research hotspot. These methods, through learning from massive amounts of data, can achieve automatic anomaly identification to a certain extent. However, these methods usually rely on large amounts of labeled data and require model training and optimization on static, fixed training sets. This assumption is often difficult to hold true in real industrial environments. Industrial data is usually dynamically changing, and due to data privacy or security requirements, some data may not be available or labeled in advance, further increasing the complexity of anomaly detection.

[0003] Existing anomaly detection methods suffer from several significant problems when applied to real-world industrial scenarios. First, most methods assume that the training and testing data distributions are identical. However, in practice, data often originates from different domains or devices, and this cross-domain distribution difference significantly reduces the model's detection performance. Second, existing methods typically rely on labeled data from the target domain. Labeling samples is not only costly but also faces data privacy and security constraints, making traditional methods unsuitable for anomaly detection on target domain samples. Furthermore, existing methods have limited generalization ability to unknown data. When applied to unseen data, they often exhibit poor robustness, failing to meet the demands of real-world industrial scenarios. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose an anomaly detection method and related equipment.

[0005] To achieve the above objectives, this application provides an anomaly detection method, comprising: Construct an anomaly detection model to be trained; Model the text prompts and determine the text prompts corresponding to the acquired source domain dataset; Based on the text prompt, the first loss function is calculated according to the minimum distance between the anchor point of the text prompt and the most difficult negative sample; Align the image features corresponding to the source domain dataset with the text prompts and calculate the second loss function; An adaptive focusing loss mechanism is introduced to focus on samples that are difficult to classify, and a third loss function is calculated. Pixel-level optimizations are performed on the anomaly segmentation task, and a fourth loss function is calculated. The anomaly detection model to be trained is trained based on the first loss function, the second loss function, the third loss function and the fourth loss function to obtain the trained anomaly detection model; The trained anomaly detection model is used to perform anomaly detection on the target domain dataset in the target domain, and the anomaly detection results are obtained.

[0006] In one possible implementation, the method further includes: Selective data augmentation is performed on the target domain dataset.

[0007] In one possible implementation, the text prompt is used to align the text prompt with the image features; the text prompt includes global text prompts and local text prompts; The step of modeling the text prompts and determining the text prompts corresponding to the acquired source domain dataset includes: Templates for creating global and local text hints for normal objects; Templates for creating global and local text hints for exception objects.

[0008] In one possible implementation, the step of calculating the first loss function based on the text prompt, according to the minimum distance between the anchor point of the text prompt and the most difficult negative sample, includes: In response to using the global text prompt of the normal object as an anchor point, the local text prompt of the normal object in the same instance is used as a positive sample, and the local text prompt of the most similar abnormal object in other instances is used as the most difficult negative sample. The first loss function is calculated based on the minimum distance between the global text prompt of the normal object and the local text prompt of the most similar abnormal object in other instances. In response to using the global text prompt of the anomalous object as an anchor point, the local text prompt of the anomalous object in the same instance is used as a positive sample, and the local text prompt of the most similar normal object in other instances is used as a negative sample. The first loss function is calculated based on the minimum distance between the global text prompt of the anomalous object and the local text prompt of the most similar normal object in other instances.

[0009] In one possible implementation, the image features include global image features and local image features; The step of aligning the image features corresponding to the source domain dataset with the text prompt and calculating the second loss function includes: The global image features and the global text prompt are subjected to feature linear modulation to obtain modulated global image features and modulated global text prompt; Based on the temperature coefficient, the global image features, and the global text prompts, the original cross-modal similarity score is calculated. Based on the temperature coefficient, the modulated global image features, and the modulated global text prompt, the modulated cross-modal similarity score is calculated. The original cross-modal loss is calculated based on the original cross-modal similarity score; The modulation cross-modal loss is calculated based on the modulation cross-modal similarity score; The global cross-modal loss is calculated based on the original cross-modal loss and the modulation cross-modal loss; Based on the focus alignment similarity algorithm, the local cross-modal loss of the local image features and the local text prompt is calculated; The second loss function is calculated based on the global cross-modal loss and the local cross-modal loss.

[0010] In one possible implementation, the introduction of an adaptive focusing loss mechanism to focus on samples that are difficult to classify, and the calculation of a third loss function, includes: The adaptive focusing loss mechanism is introduced, and the third loss function is calculated based on the true label prediction probability of the sample, the class balance factor, and the upper and lower limits of the adaptive focusing index, in order to focus on the samples that are difficult to classify.

[0011] In one possible implementation, the pixel-level optimization of the anomaly segmentation task, and the calculation of the fourth loss function, include: By combining Dice loss and binary cross-entropy loss, pixel-level optimization is performed on the region overlap and pixel classification accuracy between the predicted and actual results of the anomaly segmentation task, and the fourth loss function is calculated.

[0012] Based on the same inventive concept, embodiments of this application also provide an anomaly detection device, including: The building block is configured to build the anomaly detection model to be trained; The text prompt modeling module is configured to model text prompts and determine the text prompts corresponding to the acquired source domain dataset; The first loss function calculation module is configured to calculate the first loss function based on the text prompt and the minimum distance between the anchor point of the text prompt and the most difficult negative sample. The second loss function calculation module is configured to align the image features corresponding to the source domain dataset with the text prompt and calculate the second loss function; The third loss function calculation module is configured to introduce an adaptive focusing loss mechanism to focus on samples that are difficult to classify, and calculate the third loss function. The fourth loss function calculation module is configured to perform pixel-level optimization on the anomaly segmentation task and calculate the fourth loss function. The training module is configured to train the anomaly detection model to be trained based on the first loss function, the second loss function, the third loss function and the fourth loss function, so as to obtain the trained anomaly detection model; The detection module is configured to use the trained anomaly detection model to perform anomaly detection on the target domain dataset in the target domain, and obtain anomaly detection results.

[0013] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the anomaly detection method as described in any of the above.

[0014] Based on the same inventive concept, embodiments of this application also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute any of the above-described anomaly detection methods.

[0015] As can be seen from the above, the anomaly detection method and related equipment provided in this application construct an anomaly detection model to be trained; model the text prompts and determine the text prompts corresponding to the acquired source domain dataset; calculate a first loss function based on the text prompts according to the minimum distance between the anchor point of the text prompts and the most difficult negative sample; align the image features corresponding to the source domain dataset and the text prompts and calculate a second loss function; introduce an adaptive focusing loss mechanism to focus on samples that are difficult to classify and calculate a third loss function; perform pixel-level optimization on the anomaly segmentation task and calculate a fourth loss function; train the anomaly detection model to be trained based on the first loss function, the second loss function, the third loss function, and the fourth loss function to obtain the trained anomaly detection model; and use the trained anomaly detection model in the target domain to perform anomaly detection on the target domain dataset in the target domain to obtain anomaly detection results. This application embodiment proposes an industrial anomaly detection method based on cross-modal contrastive learning, achieving the technical effect of efficiently detecting unknown target domain data without target domain data annotation. By combining global and local text prompts, multi-granularity semantic alignment mechanisms, similarity-aware triplet contrastive learning mechanism (STCL), featurewise linear modulation module (FiLM), and adaptive focal loss (AFL), this application effectively solves the problems of traditional methods, such as excessive dependence on target domain data, high annotation costs, and insufficient generalization ability in dynamic cross-domain scenarios. This method can better meet the needs of actual industrial scenarios, achieving high-precision anomaly detection under complex and variable industrial data distributions, while exhibiting strong robustness and generalization ability, providing an efficient and practical solution for intelligent industrial detection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic flowchart of the anomaly detection method according to an embodiment of this application; Figure 2 This is a schematic diagram of an object-independent text prompt module according to an embodiment of this application; Figure 3 This is a schematic diagram illustrating the framework of the anomaly detection model training process and the actual anomaly prediction process in an embodiment of this application. Figure 4 This is a schematic diagram of the anomaly detection device according to an embodiment of this application; Figure 5 This is a schematic diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0019] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0021] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0022] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0023] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0024] As described in the background section, with the rapid development of industrial automation and intelligence, anomaly detection, as an important means to ensure the safety of industrial equipment and improve production efficiency, faces severe challenges due to the diversity and complexity of industrial data. Existing deep learning-based anomaly detection methods rely on large amounts of labeled data and assume that the distribution of training data is consistent with that of test data, typically training models on static, fixed training sets. However, data in real-world industrial environments is often dynamically changing, and data privacy or security requirements prevent the acquisition or labeling of some target domain data in advance, significantly increasing the complexity of anomaly detection. Furthermore, traditional methods have poor adaptability to cross-domain distribution differences, and their detection performance significantly decreases on data from different domains or equipment. In addition, these methods have limited generalization ability and insufficient robustness when facing unknown data, making it difficult to meet the needs of efficient and flexible detection in real-world industrial scenarios.

[0025] Based on the above considerations, this application proposes an anomaly detection method, which involves constructing an anomaly detection model to be trained; modeling text prompts and determining the text prompts corresponding to the acquired source domain dataset; calculating a first loss function based on the text prompts and the minimum distance between the anchor point of the text prompt and the most difficult negative sample; aligning the image features corresponding to the source domain dataset with the text prompts and calculating a second loss function; introducing an adaptive focusing loss mechanism to focus on samples that are difficult to classify, and calculating a third loss function; performing pixel-level optimization on the anomaly segmentation task, and calculating a fourth loss function; training the anomaly detection model to be trained based on the first, second, third, and fourth loss functions to obtain a trained anomaly detection model; and using the trained anomaly detection model in the target domain to perform anomaly detection on the target domain dataset to obtain an anomaly detection result. This application effectively solves the limitations of traditional anomaly detection methods in practical applications through a multi-granularity cross-modal contrastive learning method, making the model more closely aligned with industrial scenario requirements. It can achieve efficient anomaly detection without target domain annotation, while exhibiting strong generalization ability and robustness. These technical features work together to provide an innovative and efficient solution for industrial anomaly detection.

[0026] The technical solutions of the embodiments of this application will be described in detail below through specific examples.

[0027] refer to Figure 1The anomaly detection method of this application includes the following steps: Step S101: Construct the anomaly detection model to be trained; Step S102: Model the text prompts and determine the text prompts corresponding to the obtained source domain dataset; Step S103: Based on the text prompt, calculate the first loss function according to the minimum distance between the anchor point of the text prompt and the most difficult negative sample; Step S104: Align the image features corresponding to the source domain dataset with the text prompts and calculate the second loss function; Step S105: An adaptive focusing loss mechanism is introduced to focus on samples that are difficult to classify, and the third loss function is calculated. Step S106: Perform pixel-level optimization on the anomaly segmentation task and calculate the fourth loss function; Step S107: Train the anomaly detection model to be trained based on the first loss function, the second loss function, the third loss function and the fourth loss function to obtain the trained anomaly detection model; Step S108: In the target domain, the trained anomaly detection model is used to perform anomaly detection on the target domain dataset to obtain anomaly detection results.

[0028] Regarding step S101, in this application, the anomaly detection method first constructs an anomaly detection model to be trained. This model aims to extract global and local image features through learning from source domain data and align them with designed global and local textual prompts, thereby enabling the model to detect anomalous data in the target domain. In industrial scenarios, the source domain dataset contains various specific types of industrial data. The data of each sub-source domain consists of samples and their corresponding labels. Multiple sub-source domains are merged to form a single source domain dataset as training data. The target domain dataset consists of test samples and corresponding labels, containing industrial data collected from specific scenarios. The core objective of this application is to achieve anomaly detection from the source domain to the target domain through cross-modal feature alignment and multi-granularity prompt modeling.

[0029] In this embodiment, a set of source domain data is given: Each sub-source domain represents a specific type of industrial data to be detected. Includes samples and their corresponding tags .in It contains K samples and their corresponding label sets Where 0 represents a normal sample and 1 represents an abnormal sample. Multiple sub-source domains are merged into a single source domain and processed as a unified source domain.

[0030] Regarding the target domain, this application treats each dataset used for detection as an independent target domain, with each target domain corresponding to industrial data collected in a specific scenario. If multiple target domain sets exist... ,in Let n represent the nth target domain. For the nth target domain... , This represents the j-th sub-target domain. Each sub-target domain represents a specific type of industrial data to be detected. Includes test samples With corresponding tags Specifically, Include K One normal sample and one abnormal sample, In this context, 0 represents a normal sample, and 1 represents an abnormal sample. This process can be expressed as: , This represents the anomaly detection function.

[0031] In some embodiments, the text prompt is used to align the text prompt with the image features; the text prompt includes global text prompts and local text prompts; the step of modeling the text prompts and determining the text prompts corresponding to the acquired source domain dataset includes: constructing templates for global text prompts and local text prompts for normal objects; constructing templates for global text prompts and local text prompts for abnormal objects.

[0032] refer to Figure 2 This is a schematic diagram of an object-independent text prompt module according to an embodiment of this application.

[0033] Figure 2 Table 1 provides a reference for the Chinese definitions of each English word.

[0034] Table 1 Figure 2 Chinese translations of each English word

[0035] like Figure 2 As shown, this application aims to model text prompts from the perspective of global and local comparison, thereby fully utilizing the cross-domain generalization capability of Contrastive Language-Image Pretraining (CLIP) to achieve effective alignment of image and text features across different domains. For text prompts, this application employs the following design:

[0036]

[0037]

[0038]

[0039] in, Indicates a global text suggestion. Superscript indicates a local text prompt. Indicates a normal object, superscript This represents the exception object. The global hint's [prefix] consists of multiple tags, such as Glocal or other terms with global semantic meaning. Similarly, the local hint's [prefix] consists of tags like Local, or other terms with localized semantic meaning. The components in the [suffix] are designed to distinguish the hint from the exception text and emphasize the semantics of the foreground object; in the formula... , representing a normal object, or This design allows the model to align textual cues with visual features at different granular levels, thereby enhancing its ability to classify and segment anomalies accurately.

[0040] For example, in industrial anomaly detection scenarios, this application designs global text prompts as "[prefix]normal equipment [scene without any damaged object]" and "[prefix]abnormal equipment [scene with damaged object]" for normal equipment in the source domain and abnormal equipment in the target domain, while local text prompts are "[prefix]normal local area [scene without any damaged object]" and "[prefix]abnormal local area [scene with damaged object]". In practical applications, the learnable context mechanism dynamically adjusts the semantic expression of "[prefix]" based on training data, for example, optimizing "[prefix]" to specific domain terms such as high-temperature equipment or precision parts, thereby enhancing the adaptability of prompts to normal and abnormal patterns in industrial scenarios and improving the accuracy of global and local feature alignment. This dynamic adjustment mechanism enables the model to detect equipment anomalies more effectively for different industrial environments.

[0041] refer to Figure 3 This is a schematic diagram illustrating the framework of the anomaly detection model training process and the actual anomaly prediction process in an embodiment of this application.

[0042] Figure 3 Table 1 provides a reference for the Chinese definitions of each English word.

[0043] Table 2 Figure 3 Chinese translations of each English word

[0044] like Figure 3 As shown, in some embodiments, the step of calculating a first loss function based on the text prompt and the minimum distance between the anchor point of the text prompt and the most difficult negative sample includes: in response to using the global text prompt of the normal object as the anchor point, using the local text prompt of the normal object in the same instance as a positive sample, using the local text prompt of the most similar abnormal object in other instances as the most difficult negative sample, and calculating the first loss function based on the minimum distance between the global text prompt of the normal object and the local text prompt of the most similar abnormal object in other instances; in response to using the global text prompt of the abnormal object as the anchor point, using the local text prompt of the abnormal object in the same instance as a positive sample, using the local text prompt of the most similar normal object in other instances as a negative sample, and calculating the first loss function based on the minimum distance between the global text prompt of the abnormal object and the local text prompt of the most similar normal object in other instances.

[0045] Based on object-independent global-local semantic cues, this application employs a similarity-aware triplet contrastive learning mechanism (STCL) to achieve multi-granular semantic alignment by constructing hierarchical triplet relations. In STCL, the global cue embedding serves as the anchor point, the local cue embedding of the same instance is used as a positive sample, and the local cue embeddings of other instances constitute a candidate set of negative samples, thus constructing a hierarchical contrastive structure. This design aligns with the hierarchical nature of visual cognition: global cues encode the semantics of the overall image, while local cues focus on detailed regional features. Given an anchor vector... (No. i Anchor vectors of source domain samples and a positive sample vector. (No. i (corresponding positive sample vectors of each source domain sample), and a set of k candidate negative vectors Batch size is B The similarity-perceived ternary contrast loss (first loss) is defined as follows:

[0046] in, Indicates the first loss. The minimum distance between the anchor point and the most difficult negative sample is defined. When the global text cue of a normal object is used as the anchor point, the corresponding local text cue of the normal object is brought closer as a positive sample, while the local text cue of the most similar anomalous object among other instances is pushed away as the most difficult negative sample. Similarly, when the global text cue of an anomalous object is used as the anchor point, the local text cue of its related anomalous object is brought closer as a positive sample, while the global text cue of the anomalous object is pushed away as a negative sample. This bidirectional alignment mechanism enables the model to construct a fine semantic contrast between global and local cues, thereby enhancing its sensitivity to local details. The total STCL loss is given by the following formula:

[0047] in, This represents the total first loss. The first loss represents the loss of a normal object. This represents the first loss of the abnormal object.

[0048] In a specific embodiment of industrial anomaly detection, it is assumed that the source domain contains image data of normal and abnormal products for detection. For an image of a normal product, the global text cue is "[the overall normal product]", and the local text cue is "[the local details of the normal product]". During training, the overall image features of the normal product and its corresponding local features are considered positive samples, and the model learns the consistency between global and local features by narrowing their semantic distance. Simultaneously, the most similar local features (e.g., "[the local damaged area of ​​the abnormal product]") are selected from other abnormal products as negative samples, and the model attempts to increase the semantic distance between the anchor (the global features of the normal product) and the negative samples. Similarly, for abnormal products, the model uses "[the overall abnormal product]" as the anchor, "[the local damaged area of ​​the abnormal product]" as the positive sample, and the local features of other normal products as negative samples, training the model to better distinguish between normal and abnormal products. This bidirectional comparison mechanism based on global and local features effectively captures the semantic differences between the overall and local features of a product, thereby more accurately detecting abnormal areas in the product.

[0049] In some embodiments, the image features include global image features and local image features; aligning the image features corresponding to the source domain dataset and the text prompt and calculating a second loss function includes: performing feature linear modulation on the global image features and the global text prompt to obtain modulated global image features and modulated global text prompt; calculating the original cross-modal similarity score based on the temperature coefficient, the global image features, and the global text prompt; calculating the modulated cross-modal similarity score based on the temperature coefficient, the modulated global image features, and the modulated global text prompt; calculating the original cross-modal loss based on the original cross-modal similarity score; calculating the modulated cross-modal loss based on the modulated cross-modal similarity score; calculating the global cross-modal loss based on the original cross-modal loss and the modulated cross-modal loss; calculating the local cross-modal loss of the local image features and the local text prompt based on a focus alignment similarity algorithm; and calculating the second loss function based on the global cross-modal loss and the local cross-modal loss.

[0050] In cross-modal contrastive learning, this application further introduces a lightweight Featurewise Linear Modulation Module (FiLM). This module adaptively adjusts the input global image features and global text features through learnable scale parameters (γ) and bias parameters (β) to enhance the alignment effect between different modalities. Given global image features... and global text features The formula for calculating the modulated features is as follows:

[0051]

[0052] in, This represents the global image features after modulation. This represents the modulated global text prompt. The symbol ⊙ indicates element-wise multiplication, i.e., independent operation on corresponding elements of two vectors or matrices of the same dimension. To align the original and modulated cross-modal features, this application computes a normalized cross-modal contrastive loss with label smoothing, as follows:

[0053]

[0054] in, Indicates the temperature coefficient. This represents the original cross-modal similarity score. This represents the modulation cross-mode loss.

[0055] This application utilizes temperature coefficient The negative Euclidean distance is calculated as the cross-modal similarity score. Furthermore, based on this score, the cross-modal loss is calculated as follows:

[0056]

[0057]

[0058] in, Represents the original cross-modal loss. Indicates the modulation cross-mode loss, C Represents the number of categories. For smoothing coefficients, This represents the global cross-modal loss.

[0059] In a specific embodiment, for the task of detecting product surface defects in manufacturing, the model needs to semantically align the global appearance features and local defect features of the product. For example, for a global image of a product, the global text cue is "[a flawless product appearance]", while the local text cue is "[minor scratches on the product surface]". Through a Feature Linear Modulation (FiLM) module, the model dynamically adjusts the global image features to better match the semantics of the global text cues, thereby establishing a clearer distinction between normal products and defective products. Simultaneously, by aligning local image features and local text cues, the model can identify fine defect features, such as "minor scratches" or "micro-dents". Through this global and local cross-modal alignment process, combined with the original and modulated cross-modal losses, the model can more accurately capture the semantic relationships between global and local features of the product surface, thus significantly improving the defect detection performance.

[0060] The aforementioned global cross-modal loss is calculated using cross-entropy, while the local cross-modal loss is calculated by measuring the difference between each predicted anomalous region and the actual anomalous region. This application employs Focal AlignedSimilarity (FAS) to calculate the similarity between local image-text cues. Given features at the local patch level... and local text prompt features Local similarity can be calculated. :

[0061] in, Represents the first extracted from the image i Local image features, Indicates categoryc Local text hints, In addition to categories c Local text hints for categories other than [those listed below]. Similarity score. Used to measure the i The degree of semantic matching between an image region and category c text. The temperature parameter τ adjusts the smoothness of the probability distribution.

[0062] Similarity score A multi-scale similarity map can be obtained and combined with the true label. An Adaptive Focal Loss (AFL) is introduced to focus on local regions where samples are difficult to classify. This includes: Calculating the third loss function based on the predicted probability of the true label, the class balance factor, and the upper and lower bounds of the adaptive focal index, to guide the model to focus more on samples that are difficult to classify, thereby improving overall generalization ability. According to the... i The predicted true label probability of a sample Category balance factor and the upper and lower limits of the adaptive focus index and :

[0063] in, This represents the third loss function. By adaptively focusing on difficult-to-classify samples while maintaining class balance, it improves the robustness and accuracy of the model under complex data distributions.

[0064] In one specific embodiment, for the detection of surface defects in automotive parts, the number of normal part samples far exceeds the number of abnormal samples in the data, leading to a class imbalance problem. To address this issue, this application introduces an Adaptive Focus Loss (AFL) mechanism. For example, in a detection task, for a part sample with a small crack, because the crack area is small and easily confused with the background, the model may tend to misclassify it as a normal sample. Through the AFL mechanism, the model dynamically adjusts its focus, allocating more weight to such difficult-to-classify abnormal samples, while reducing the weight of easily classified normal samples. In this way, the model can more accurately identify defects that are difficult to detect, such as small cracks, while avoiding excessive bias towards the large number of normal samples, thereby improving the overall robustness and accuracy of the detection.

[0065] In some embodiments, the pixel-level optimization of the anomaly segmentation task to calculate the fourth loss function includes: combining Dice loss and binary cross-entropy loss to perform pixel-level optimization on the region overlap and pixel classification accuracy between the predicted and true results of the anomaly segmentation task, and calculating the fourth loss function.

[0066] In this embodiment, for the anomaly segmentation task, this application combines Dice loss and classic binary cross-entropy loss to optimize the region overlap between the predicted and ground truth labels and the accuracy of pixel classification. Specifically, the Dice component focuses on the overall shape and size of the target region, imposing a stronger penalty on hard-to-classify samples (such as small targets or blurred boundaries), thereby improving the model's ability to detect these regions. The cross-entropy component fine-tunes the classification accuracy at the pixel level. These two components are linked by weight factors. Achieving a balance ensures the model achieves good segmentation quality and classification stability at the pixel level. Given pixel-level predicted values... With tag value The loss function can be expressed as:

[0067]

[0068] in, This represents a joint loss function that combines the Dice loss and the classic binary cross-entropy loss, used to measure pixel classification accuracy. This indicates Dice's loss. This represents the classic binary cross-entropy loss. x Indicates the first x 1 pixel, μ It is the Focal index. ε For smoothing terms. The first term is a focal variant of the Dice loss, which is obtained through exponential... μ The first term emphasizes pixels that are difficult to classify, while simultaneously promoting predictions that more closely approximate the true labels. The second term corresponds to the binary cross-entropy loss, which aims to enhance pixel-level classification accuracy. Weighting factors Used to adjust the balance between the two components. ε To ensure numerical stability, therefore, local loss... The expression is:

[0069] In this application, partial loss It is the sum of the third and fourth loss functions.

[0070] In this embodiment, the total contrastive learning loss in the source domain is:

[0071] The total contrastive learning loss is fed back to the constructed anomaly detection model to be trained, and the trained anomaly detection model is obtained.

[0072] In one specific embodiment, for the segmentation and detection of anomalous regions on the surface of a semiconductor chip, this application combines Dice loss and binary cross-entropy loss (BCE) to optimize the model's segmentation performance. Since anomalous regions on the chip surface are typically small and irregularly shaped, Dice loss focuses on the overall shape and size of the anomalous region, imposing a greater penalty on small, hard-to-identify target regions, thereby improving the detection capability for these regions. BCE, on the other hand, is used for pixel-wise optimization, enabling the model to more accurately distinguish anomalous regions from normal regions at the pixel level. For example, for an image of a chip surface with minute scratches, Dice loss ensures the model has a higher degree of overlap in the overall shape of the scratched region, while BCE further fine-tunes the model, making the prediction results more refined and accurate. By combining these two losses and dynamically adjusting their weights, the model can better balance the overall overlap of the region with the accuracy of pixel-level classification, thus achieving better detection results in the chip surface anomalous segmentation task.

[0073] In some embodiments, the method further includes: selectively augmenting the target domain dataset.

[0074] Within the target domain, this application employs Contrast Limited Adaptive Histogram Equalization (CLAHE) to selectively augment unlabeled samples. This is an optional augmentation operation designed based on data categories. Given unlabeled samples from the target domain... s The enhanced sample can be represented as:

[0075] in, Let G represent the cumulative distribution function of the histogram, and G represent the gray levels of sample s. For samples in the target domain, corresponding representations are obtained through feature extraction. Given the global image features of the samples to be detected... and global text features The global similarity is calculated as follows:

[0076] in, This represents the global image feature similarity of the samples to be detected. For local image feature similarity... :

[0077] The specific calculation process is the same as that in the training process, and will not be repeated here.

[0078] By using the trained anomaly detection model in the target domain, anomaly detection is performed on the target domain dataset to obtain anomaly detection results, thus enabling anomaly detection of samples in the target domain.

[0079] In one feasible embodiment, the anomaly detection method of this application has a wide range of applications, especially showing great potential in fields such as industrial inspection, intelligent manufacturing, medical image analysis, and autonomous driving. This method, by combining global and local text prompts, multi-granularity semantic alignment, adaptive focus loss, and pixel-level optimized segmentation, can not only generalize from the source domain to the target domain but also adapt to various data distributions and scenario requirements, thereby achieving efficient and accurate anomaly detection in complex environments.

[0080] In industrial inspection, this application can be used in various scenarios such as equipment fault diagnosis, product quality inspection, and production line monitoring. For example, in the quality inspection of parts on a production line, different parts may have different shapes and characteristics. The source domain data may include normal and abnormal samples of known parts, while the target domain may contain entirely new part categories. Through the global text prompts (such as "the part is generally flawless") and local text prompts (such as "the part has a local crack") of this application, combined with the alignment of image features, the model can quickly identify whether new parts have abnormal areas. At the same time, the adaptive focus loss (AFL) can dynamically adjust the attention to difficult-to-classify samples such as small cracks, ensuring that even in cases of imbalanced sample categories, these minute anomalies can be accurately detected. In addition, the segmentation optimization combining Dice loss and binary cross-entropy loss enables the model to accurately segment the specific shape and location of cracks at the pixel level, thereby providing a reliable basis for subsequent automated maintenance.

[0081] In the field of intelligent manufacturing, this application can also be applied to the monitoring of the operational status of complex mechanical equipment. For the operation of large mechanical equipment, source domain data may contain the normal operating status of the equipment under different environments, while target domain data may involve the abnormal status of the equipment under unknown environments. This application uses text prompt modeling based on global and local comparisons to enable the model to quickly adapt to environmental changes. For example, global text prompts can describe the overall operational status of the equipment (such as "equipment is operating normally" or "equipment is operating abnormally"), while local text prompts can focus on specific abnormal components (such as "local wear of equipment bearings"). Through the Feature Linear Modulation (FiLM) module, the model can dynamically adjust the expression of global and local image features to make them more consistent with the text prompts, thereby achieving accurate monitoring of the equipment's operational status.

[0082] In medical image analysis, the technical solution of this application also has significant application value. For example, in the analysis of CT scans and MRI images, the source domain data can be labeled normal and abnormal medical images, while the target domain data may come from different patients or devices, exhibiting significant inter-domain differences. This application, through its cross-modal contrastive learning capability, enables the model to seamlessly transfer learning results from the source domain to the target domain. For example, global text prompts can describe the overall health status of the image (e.g., "no lesions in the lungs" or "a mass is present in the lungs"), while local text prompts can target specific lesion areas (e.g., "blurred tumor boundaries" or "abnormally dense lesion area"). By calculating the similarity between the local image and the text prompt through Focus Alignment Similarity (FAS), the model can more accurately identify and locate abnormal regions. Simultaneously, by combining the segmentation effects of Dice loss and binary cross-entropy loss, the model can segment lesion regions at the pixel level, providing doctors with more intuitive and accurate diagnostic information.

[0083] In autonomous driving scenarios, vehicles need to detect road conditions and potential hazards in real time, such as pedestrians suddenly crossing the road or vehicles ahead braking abruptly. This application combines global and local text prompt modeling, enabling the autonomous driving system to understand the road scene from both a holistic and detailed perspective. For example, global text prompts can describe the overall road condition (e.g., "Road clear" or "Danger ahead"), while local text prompts can focus on specific hazardous areas (e.g., "Pedestrian crossing the road on the left" or "Vehicle ahead braking suddenly"). Through a similarity-aware ternary contrastive learning mechanism (STCL), the model can accurately align global and local features, thereby quickly identifying potential hazards and taking appropriate action.

[0084] As can be seen from the above embodiments, the anomaly detection method described in this application constructs an anomaly detection model to be trained; models the text prompts and determines the text prompts corresponding to the acquired source domain dataset; calculates a first loss function based on the text prompts according to the minimum distance between the anchor point of the text prompts and the most difficult negative sample; aligns the image features corresponding to the source domain dataset with the text prompts and calculates a second loss function; introduces an adaptive focusing loss mechanism to focus on samples that are difficult to classify and calculates a third loss function; performs pixel-level optimization on the anomaly segmentation task and calculates a fourth loss function; trains the anomaly detection model to be trained based on the first loss function, the second loss function, the third loss function, and the fourth loss function to obtain the trained anomaly detection model; and uses the trained anomaly detection model in the target domain to perform anomaly detection on the target domain dataset in the target domain to obtain anomaly detection results. This application embodiment fully utilizes the cross-domain generalization capability of the Visual Language Contrast Model (CLIP) through the design of global and local text prompts. Global prompts describe the semantic information of the overall scene, while local prompts focus on the semantic expression of detailed regions. This multi-granularity text prompt design allows the model to align image and text features at both the global and local levels, thereby enhancing the model's anomaly detection capabilities. Especially in cross-domain scenarios, this text-based feature alignment method avoids dependence on prior knowledge of the target domain, significantly improving the model's adaptability and detection accuracy in unknown target domains.

[0085] Similarity-aware ternary contrastive learning (STCL) further achieves fine-grained semantic alignment between global and local cues by constructing a hierarchical ternary contrastive mechanism. By using global normal or abnormal cues as anchors, relevant local cues as positive samples, and local cues from other instances as a negative sample candidate set, STCL effectively brings semantically relevant positive samples closer together while pushing away semantically irrelevant negative samples. This bidirectional alignment mechanism significantly improves the model's sensitivity to anomalous detail regions, enabling it to more accurately detect anomalous regions in complex industrial data.

[0086] The Feature Linear Modulation (FiLM) mechanism adaptively adjusts the global image and text features of the input by learning adjustable scale and bias parameters, further enhancing the alignment effect of cross-modal features. The FiLM module optimizes the semantic consistency of image-text at the global feature level, enabling the model to more accurately understand and match semantic information in complex industrial scenarios. This not only improves the accuracy of cross-modal alignment but also provides the model with stronger robustness under dynamically changing data distributions.

[0087] AFL addresses the class imbalance problem commonly found in industrial anomaly detection. By dynamically adjusting the focus index, AFL guides the model to focus more on difficult-to-classify samples while balancing the influence of samples from different classes. This design significantly improves the model's classification performance under complex data distributions, especially when facing small samples or imbalanced data in the target domain, where AFL effectively reduces false positives and false negatives.

[0088] Furthermore, by combining global and local similarity calculations, this application achieves deep matching of image regions and text prompts in target domain anomaly detection. Global similarity is used to measure the semantic matching degree between the overall image and text, while local similarity focuses on fine-grained anomaly region detection. Through the synergistic effect of global and local similarity, the model can more comprehensively detect anomalies in the target domain data, thereby improving the overall detection performance.

[0089] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0090] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0091] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides an anomaly detection device.

[0092] refer to Figure 4 The anomaly detection device includes: Module 41 is configured to build an anomaly detection model to be trained; The text prompt modeling module 42 is configured to model text prompts and determine text prompts corresponding to the acquired source domain dataset; The first loss function calculation module 43 is configured to calculate the first loss function based on the text prompt and the minimum distance between the anchor point of the text prompt and the most difficult negative sample. The second loss function calculation module 44 is configured to align the image features corresponding to the source domain dataset with the text prompt and calculate the second loss function; The third loss function calculation module 45 is configured to introduce an adaptive focusing loss mechanism to focus on samples that are difficult to classify, and calculate the third loss function. The fourth loss function calculation module 46 is configured to perform pixel-level optimization on the anomaly segmentation task and calculate the fourth loss function. Training module 47 is configured to train the anomaly detection model to be trained based on the first loss function, the second loss function, the third loss function and the fourth loss function to obtain the trained anomaly detection model; The detection module 48 is configured to use the trained anomaly detection model to perform anomaly detection on the target domain dataset in the target domain, and obtain anomaly detection results.

[0093] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0094] The apparatus of the above embodiments is used to implement the corresponding anomaly detection method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0095] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the anomaly detection method described in any of the above embodiments.

[0096] Figure 5 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0097] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0098] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0099] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0100] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0101] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0102] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0103] The electronic devices described above are used to implement the corresponding anomaly detection methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0104] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the anomaly detection method as described in any of the above embodiments.

[0105] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0106] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the anomaly detection method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0107] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0108] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0109] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0110] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. An anomaly detection method, characterized in that, include: Construct an anomaly detection model to be trained; Model the text prompts and determine the text prompts corresponding to the acquired source domain dataset; Based on the text prompt, the first loss function is calculated according to the minimum distance between the anchor point of the text prompt and the most difficult negative sample; Align the image features corresponding to the source domain dataset with the text prompts and calculate the second loss function; An adaptive focusing loss mechanism is introduced to focus on samples that are difficult to classify, and a third loss function is calculated. Pixel-level optimizations are performed on the anomaly segmentation task, and a fourth loss function is calculated. The anomaly detection model to be trained is trained based on the first loss function, the second loss function, the third loss function and the fourth loss function to obtain the trained anomaly detection model; The trained anomaly detection model is used to perform anomaly detection on the target domain dataset in the target domain, and the anomaly detection results are obtained.

2. The method according to claim 1, characterized in that, The method further includes: Selective data augmentation is performed on the target domain dataset.

3. The method according to claim 1, characterized in that, The text prompt is used to align the text prompt with the image features; the text prompt includes global text prompts and local text prompts; The step of modeling the text prompts and determining the text prompts corresponding to the acquired source domain dataset includes: Templates for creating global and local text hints for normal objects; Templates for creating global and local text hints for exception objects.

4. The method according to claim 3, characterized in that, The step of calculating a first loss function based on the text prompt, according to the minimum distance between the anchor point of the text prompt and the most difficult negative sample, includes: In response to using the global text prompt of the normal object as an anchor point, the local text prompt of the normal object in the same instance is used as a positive sample, and the local text prompt of the most similar abnormal object in other instances is used as the most difficult negative sample. The first loss function is calculated based on the minimum distance between the global text prompt of the normal object and the local text prompt of the most similar abnormal object in other instances. In response to using the global text prompt of the anomalous object as an anchor point, the local text prompt of the anomalous object in the same instance is used as a positive sample, and the local text prompt of the most similar normal object in other instances is used as a negative sample. The first loss function is calculated based on the minimum distance between the global text prompt of the anomalous object and the local text prompt of the most similar normal object in other instances.

5. The method according to claim 3, characterized in that, The image features include global image features and local image features; The step of aligning the image features corresponding to the source domain dataset with the text prompt and calculating the second loss function includes: The global image features and the global text prompt are subjected to feature linear modulation to obtain modulated global image features and modulated global text prompt; Based on the temperature coefficient, the global image features, and the global text prompts, the original cross-modal similarity score is calculated. Based on the temperature coefficient, the modulated global image features, and the modulated global text prompt, the modulated cross-modal similarity score is calculated. The original cross-modal loss is calculated based on the original cross-modal similarity score; The modulation cross-modal loss is calculated based on the modulation cross-modal similarity score; The global cross-modal loss is calculated based on the original cross-modal loss and the modulation cross-modal loss; Based on the focus alignment similarity algorithm, the local cross-modal loss of the local image features and the local text prompt is calculated; The second loss function is calculated based on the global cross-modal loss and the local cross-modal loss.

6. The method according to claim 3, characterized in that, The introduction of an adaptive focusing loss mechanism to focus on samples that are difficult to classify yields a third loss function, which includes: The adaptive focusing loss mechanism is introduced, and the third loss function is calculated based on the true label prediction probability of the sample, the class balance factor, and the upper and lower limits of the adaptive focusing index, so as to focus on the samples that are difficult to classify.

7. The method according to claim 3, characterized in that, The pixel-level optimization of the anomaly segmentation task, calculated to obtain the fourth loss function, includes: By combining Dice loss and binary cross-entropy loss, pixel-level optimization is performed on the region overlap and pixel classification accuracy between the predicted and actual results of the anomaly segmentation task, and the fourth loss function is calculated.

8. An anomaly detection device, characterized in that, include: The building block is configured to build the anomaly detection model to be trained; The text prompt modeling module is configured to model text prompts and determine the text prompts corresponding to the acquired source domain dataset; The first loss function calculation module is configured to calculate the first loss function based on the text prompt and the minimum distance between the anchor point of the text prompt and the most difficult negative sample. The second loss function calculation module is configured to align the image features corresponding to the source domain dataset with the text prompt and calculate the second loss function; The third loss function calculation module is configured to introduce an adaptive focusing loss mechanism to focus on samples that are difficult to classify, and calculate the third loss function. The fourth loss function calculation module is configured to perform pixel-level optimization on the anomaly segmentation task and calculate the fourth loss function. The training module is configured to train the anomaly detection model to be trained based on the first loss function, the second loss function, the third loss function and the fourth loss function, so as to obtain the trained anomaly detection model; The detection module is configured to use the trained anomaly detection model to perform anomaly detection on the target domain dataset in the target domain, and obtain anomaly detection results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 7.

Images