A pixel-level anomaly detection method, device, equipment, medium and program

Abstract

The present disclosure relates to a pixel-level anomaly detection method, apparatus, device, storage medium and computer program. The method comprises: extracting global information of the input image by using a preset prompt encoding module, extracting multi-scale features according to the global information by using a residual full-dimensional dynamic convolution module of an omnidirectional state space model of pre-trained points, generating optimal prompts according to the multi-scale features by using the prompt encoding module, extracting features from a pre-acquired input image by using an encoding module of a pre-trained segmentation model to obtain visual features, and generating an anomaly detection mask according to the visual features and the optimal prompt by using a decoding module of the segmentation model.

Description

Technical Field

[0001] The present disclosure relates to the field of image recognition technology, and in particular to a pixel-level anomaly detection method, device, equipment, medium and program. Background Art

[0002] Pixel-level anomaly detection is a technology that can accurately identify abnormal pixels in an image. It is a technology that identifies data points or events that are significantly different from normal data patterns. In the field of image processing, pixel-level anomaly detection refers to the anomaly detection of each pixel of an image, thereby identifying abnormal areas in the image. This technology is widely used in road monitoring, industrial inspection, medical image analysis and other fields, and is of great significance for timely detection and handling of potential problems. However, this technology is highly dependent on the expertise and proficiency of analysts. Therefore, there is an urgent need to develop automated methods to replace the previous time-consuming and technically demanding manual detection process.

[0003] However, existing pixel-level anomaly detection methods are highly dependent on the analyst's expertise and proficiency, which increases the cost of anomaly detection and is inefficient. Summary of the invention

[0004] The present disclosure provides a pixel-level anomaly detection method, device, equipment, medium and program to solve the problem of high cost and low efficiency of existing pixel-level anomaly detection methods.

[0005] In a first aspect, the present disclosure provides a pixel-level anomaly detection method, comprising:

[0006] Extracting global information of the input image using a preset prompt encoding module;

[0007] Extracting multi-scale features based on the global information using a residual full-dimensional dynamic convolution module of an omnidirectional state space model of pre-trained points;

[0008] generating an optimal prompt based on the multi-scale features using the prompt encoding module;

[0009] Using the encoding module of the pre-trained segmentation model to extract features from the pre-acquired input image, obtaining visual features;

[0010] An anomaly detection mask is generated using a decoding module of the segmentation model based on the visual features and the optimal hint.

[0011] In some embodiments, the residual full-dimensional dynamic convolution module of the omnidirectional state space model using pre-trained points extracts multi-scale features according to the global information, including:

[0012] Discretizing the preset time scale parameter to obtain a discretized time scale parameter;

[0013] The global information is transformed using the discretized time scale parameter to obtain transformed information;

[0014] The residual full-dimensional dynamic convolution module is used to selectively scan the transformation information to obtain multi-scale features.

[0015] In some embodiments, the using the discretized time scale parameter to transform the global information to obtain the transformed information includes:

[0016] The global information is initially transformed using the following formula to obtain initial transformation information:

[0017]

[0018] Wherein, H(t) is the preliminary transformation information, is the first discretization parameter in the discretization time scale parameter, is the second discretization parameter in the discretization time scale parameter, h(t) is a preset hidden state function, and x(t) is the global information;

[0019] The following formula is used to perform final transformation on the preliminary transformation information to obtain transformation information:

[0020] Y(t)=C·H(t)

[0021] Among them, Y(t) is the transformation information, C is the preset projection matrix, and H(t) is the preliminary transformation information.

[0022] In some embodiments, before extracting multi-scale features according to the global information using the residual full-dimensional dynamic convolution module of the omnidirectional state space model of the pre-trained points, the method further includes:

[0023] Utilize a preset neural network model to output a predicted segmentation mask based on pre-acquired training data;

[0024] Calculating a final loss value according to the predicted segmentation mask and a standard segmentation mask preset in the training data using a preset loss function;

[0025] Judging whether the performance of the neural network model meets the requirements according to the final loss value;

[0026] If the performance of the neural network model does not meet the requirements, after optimizing the hyperparameters of the neural network model according to the final loss value, returning to the step of using the preset neural network model to output a predicted segmentation mask according to the pre-acquired training data;

[0027] If the performance of the neural network model meets the requirements, the model training is confirmed to be completed and an omnidirectional state space model is obtained.

[0028] In some embodiments, the calculating the final loss value according to the predicted segmentation mask and the standard segmentation mask preset by the training data by using a preset loss function includes:

[0029] The first loss value is calculated using the following formula:

[0030]

[0031] Among them, L BCE is the first loss value, N is the number of samples, is the i-th prediction result in the predicted segmentation mask, y i is the i-th standard mask in the standard segmentation mask.

[0032] The second loss value is calculated using the following formula:

[0033]

[0034] Among them, L Dice is the first loss value, N is the number of samples, is the i-th prediction result in the predicted segmentation mask, y i is the i-th standard mask in the standard segmentation mask, and ∈ is a preset constant.

[0035] The final loss value is calculated according to the first loss value and the second loss value using the following formula:

[0036] L=λ1L Dice +λ2L BCE

[0037] Wherein, L is the final loss value, L BCE is the first loss value, L Dice is the first loss value, and λ1 and λ2 are both preset balance factors.

[0038] In some embodiments, the using the hint encoding module to generate an optimal hint according to the multi-scale feature includes:

[0039] generating a preliminary prompt set based on the multi-scale features;

[0040] obtaining quality assessment data of the preliminary prompt set;

[0041] iteratively optimizing the preliminary prompt set based on the quality assessment data;

[0042] When the preset number of iterations is reached, the optimal prompt is obtained.

[0043] In a second aspect, the present disclosure provides a pixel-level anomaly detection device, comprising:

[0044] The global information extraction module is used to extract the global information of the input image using a preset prompt encoding module;

[0045] A multi-scale feature extraction module, used to extract multi-scale features according to the global information using the residual full-dimensional dynamic convolution module of the omnidirectional state space model of the pre-trained points;

[0046] an optimal prompt generating module, configured to generate an optimal prompt according to the multi-scale features by using the prompt encoding module;

[0047] A visual feature extraction module, used to extract features from a pre-acquired input image using an encoding module of a pre-trained segmentation model to obtain visual features;

[0048] A mask generation module is used to generate an anomaly detection mask based on the visual features and the optimal hint using the decoding module of the segmentation model.

[0049] In a third aspect, the present disclosure provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above aspects.

[0050] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method described in the above aspects when executed by a processor.

[0051] In a fifth aspect, the present disclosure provides a computer program product, including a computer program, which implements the steps of the method described in the above aspects when executed by a processor.

[0052] The present disclosure provides a pixel-level anomaly detection method, device, equipment, storage medium and computer program, which extracts the global information of the input image by using a preset prompt encoding module, extracts multi-scale features according to the global information by using the residual full-dimensional dynamic convolution module of the omnidirectional state space model of the pre-trained points, generates optimal prompts according to the multi-scale features by using the prompt encoding module, extracts features from the pre-acquired input image by using the encoding module of the pre-trained segmentation model to obtain visual features, and generates anomaly detection masks according to the visual features and the optimal prompts by using the decoding module of the segmentation model. The problem of high cost and low efficiency of the existing pixel-level anomaly detection method is solved, and the efficiency of pixel-level anomaly detection is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present disclosure will be described in more detail below based on embodiments and with reference to the accompanying drawings:

[0054] Figure 1 A flowchart of a pixel-level anomaly detection method provided by an embodiment of the present disclosure;

[0055] Figure 2 A functional module diagram of a pixel-level anomaly detection device provided in an embodiment of the present disclosure.

[0056] In the drawings, the same reference numerals are used for the same components, and the drawings are not drawn to scale. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, and to fully understand and implement how the present disclosure applies technical means to solve technical problems and achieve the corresponding technical effects, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only embodiments of a part of the present disclosure, not all of the embodiments. The embodiments of the present disclosure and the various features in the embodiments can be combined with each other without conflict, and the technical solutions formed are all within the scope of protection of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present disclosure.

[0058] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0059] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0060] Embodiment 1

[0061] Figure 1 The following is a flowchart of a pixel-level anomaly detection method provided by an embodiment of the present disclosure. Figure 1 As shown, a pixel-level anomaly detection method includes:

[0062] S1. Using a preset prompt encoding module to extract global information of the input image.

[0063] In the embodiment of the present invention, the preset hint encoding module includes a hint encoder, and the hint encoder is an encoder for generating adaptive hints according to images, and the generated hints are used for subsequent extraction of multi-scale features.

[0064] In the embodiment of the present invention, the hint encoder includes a feature encoder, which can be used to extract features of abnormal targets in the image.

[0065] In detail, the feature encoder is an image encoder based on Vision Transformer (ViT).

[0066] In detail, the Vision Transformer is a deep learning model based on the attention mechanism, which introduces the Transformer architecture from the field of natural language processing into computer vision for processing image data.

[0067] In an embodiment of the present invention, the global information of the input image is extracted by using a preset hint encoding module, which is performed by dividing the input image into image blocks of fixed size, mapping each image block to a low-dimensional feature space through linear changes, assigning position information to each image block through position encoding, and then capturing the global dependency between each image block through the Transformer's multi-head attention mechanism to obtain global information.

[0068] In the embodiment of the present invention, the efficiency of extracting multi-scale features is improved by extracting the global information of the input image using a preset hint encoding module.

[0069] S2. Utilize the residual full-dimensional dynamic convolution module of the omnidirectional state space model of the pre-trained points to extract multi-scale features according to the global information.

[0070] In the embodiment of the present invention, the omnidirectional state space model is a special state space model, which can process and describe the system state changing in multiple directions.

[0071] In an embodiment of the present invention, the residual full-dimensional dynamic convolution module of the omnidirectional state space model using pre-trained points extracts multi-scale features according to the global information, including:

[0072] Discretizing the preset time scale parameter to obtain a discretized time scale parameter;

[0073] The global information is transformed using the discretized time scale parameter to obtain transformed information;

[0074] The residual full-dimensional dynamic convolution module is used to selectively scan the transformation information to obtain multi-scale features.

[0075] In detail, the using the discretized time scale parameter to transform the global information to obtain the transformed information includes:

[0076] The global information is initially transformed using the following formula to obtain initial transformation information:

[0077]

[0078] Wherein, H(t) is the preliminary transformation information, is the first discretization parameter in the discretization time scale parameter, is the second discretization parameter in the discretization time scale parameter, h(t) is a preset hidden state function, and x(t) is the global information;

[0079] The following formula is used to perform final transformation on the preliminary transformation information to obtain transformation information:

[0080] Y(t)=C·H(t)

[0081] Among them, Y(t) is the transformation information, C is the preset projection matrix, and H(t) is the preliminary transformation information.

[0082] In detail, the residual full-dimensional dynamic convolution module is used to selectively scan the transformation information to obtain multi-scale features.

[0083] In the embodiment of the present invention, before the residual full-dimensional dynamic convolution module of the omnidirectional state space model using the pre-trained points extracts multi-scale features according to the global information, the method further includes:

[0084] Utilize a preset neural network model to output a predicted segmentation mask based on pre-acquired training data;

[0085] Calculating a final loss value according to the predicted segmentation mask and a standard segmentation mask preset in the training data using a preset loss function;

[0086] Judging whether the performance of the neural network model meets the requirements according to the final loss value;

[0087] If the performance of the neural network model does not meet the requirements, after optimizing the hyperparameters of the neural network model according to the final loss value, returning to the step of using the preset neural network model to output a predicted segmentation mask according to the pre-acquired training data;

[0088] If the performance of the neural network model meets the requirements, the model training is confirmed to be completed and an omnidirectional state space model is obtained.

[0089] In detail, the using a preset loss function to calculate a final loss value according to the predicted segmentation mask and a standard segmentation mask preset in the training data includes:

[0090] The first loss value is calculated using the following formula:

[0091]

[0092] Among them, L BCE is the first loss value, N is the number of samples, is the i-th prediction result in the predicted segmentation mask, y i is the i-th standard mask in the standard segmentation mask.

[0093] The second loss value is calculated using the following formula:

[0094]

[0095] Among them, L Dice is the first loss value, N is the number of samples, is the i-th prediction result in the predicted segmentation mask, y i is the i-th standard mask in the standard segmentation mask, and ∈ is a preset constant.

[0096] The final loss value is calculated according to the first loss value and the second loss value using the following formula:

[0097] L=λ1L Dice +λ2L BCE

[0098] Wherein, L is the final loss value, L BCE is the first loss value, L Dice is the first loss value, and λ1 and λ2 are both preset balance factors.

[0099] In detail, the first loss value is a binary cross entropy loss value, which is mainly used for binary classification problems. It measures the difference between the probability distribution predicted by the model and the probability distribution of the true label. The second loss value is Dice loss, which is a loss function evolved from the Dice coefficient and is a metric function used to evaluate the similarity of two samples.

[0100] In detail, the preset balance factors may all be 0.5.

[0101] In an embodiment of the present invention, by utilizing the residual full-dimensional dynamic convolution module of the omnidirectional state space model of the pre-trained points to extract multi-scale features according to the global information, the representation strength of edge features is improved, the feature accuracy of multi-scale features is improved, and the accuracy of subsequent generation of optimal prompts is improved.

[0102] S3. Using the prompt encoding module to generate an optimal prompt according to the multi-scale features.

[0103] In the embodiment of the present invention, the optimal hint is a feature representation of an image, which may be a mark of different regions or key points of the image.

[0104] In the embodiment of the present invention, the step of using the hint encoding module to generate an optimal hint according to the multi-scale feature includes:

[0105] generating a preliminary prompt set based on the multi-scale features;

[0106] obtaining quality assessment data of the preliminary prompt set;

[0107] iteratively optimizing the preliminary prompt set based on the quality assessment data;

[0108] When the preset number of iterations is reached, the optimal prompt is obtained.

[0109] In detail, the obtaining of the quality assessment data of the preliminary prompt set may be to calculate indicators such as feature strength or confidence of abnormal regions in the preliminary prompts.

[0110] In detail, iteratively optimizing the preliminary prompt set according to the quality assessment data refers to selecting the best prompts for refinement or generating new prompts according to the quality assessment data.

[0111] In the embodiment of the present invention, by using the prompt encoding module to generate the optimal prompt according to the multi-scale feature, the accuracy of the subsequent generation of the anomaly detection mask is improved.

[0112] S4. Use the encoding module of the pre-trained segmentation model to extract features from the pre-acquired input image to obtain visual features.

[0113] In an embodiment of the present invention, the segmentation model is a Segment Anything Model (SAM), which is a prompt-based image segmentation model that is trained using tens of millions of image-mask data pairs and can complete high-quality image segmentation tasks based on prompts.

[0114] In the embodiment of the present invention, the encoding module using the pre-trained segmentation model performs feature extraction on the pre-acquired input image to obtain the visual feature, and the visual feature can be obtained by extracting the edge of the input image through an edge detection algorithm.

[0115] In the embodiment of the present invention, the encoding module of the pre-trained segmentation model is used to extract features from the pre-acquired input image to obtain visual features, thereby improving the efficiency of generating anomaly detection masks.

[0116] S5. Generate an anomaly detection mask according to the visual features and the optimal prompt using the decoding module of the segmentation model.

[0117] In an embodiment of the present invention, the anomaly detection mask may be a matrix whose size is the same as that of the input image, that is, the number of elements in the matrix is ​​the same as the number of pixels in the input image, and the elements corresponding to the abnormal areas in the anomaly detection mask and the input image are marked as 1 and the rest are marked as 0.

[0118] In an embodiment of the present invention, the decoding module using the segmentation model generates an anomaly detection mask according to the visual features and the optimal prompts, and uses the segmentation model to generate a feature matrix according to the visual features and the optimal prompts, wherein each element in the matrix corresponds to the abnormality probability of each pixel in the input image, and then a threshold is set to convert the elements in the feature matrix into a binary mask, thereby finally obtaining the anomaly detection mask.

[0119] In the embodiment of the present invention, by integrating multi-scale features and global context information, the optimal prompt is automatically generated, and the abnormal area in the image is accurately identified based on the optimal prompt, thereby improving the accuracy and efficiency of anomaly detection.

[0120] Embodiment 2

[0121] Based on the above embodiments, this embodiment provides an application example.

[0122] The pixel-level anomaly detection method proposed in this invention mainly consists of three key parts: SAM encoder Enc SAM , prompt encoder Enc p and SAM decoderDec SAM These three parts work together to realize the whole process from visual feature extraction to final segmentation mask output. First, Enc SAM Responsible for extracting visual features of the input image. These features provide rich image information for subsequent anomaly detection. Next, prompt the encoder Enc pThese features are further refined, especially the visual features of abnormal targets. The hint encoder contains a feature encoder for extracting features of abnormal targets. In order to fully mine the global information of abnormal targets, the present invention designs a residual full-dimensional dynamic convolution module based on the state-space model. This module performs selective scanning in multiple directions, enhancing the global effective receptive field of the model in multiple directions. In addition, through multi-scale representation, the module can effectively capture the edge details of abnormal targets, which is crucial for accurate anomaly detection. The hint encoder uses these multi-scale features to iteratively search for the optimal hint in the hint space to achieve more accurate pixel-level anomaly detection results. This process enables the model to deeply understand the abnormal features of different scales, thereby improving the accuracy of detection. Finally, the optimal hint is input into Dec together with the extracted visual features. SAM Generate the final pixel-level anomaly detection mask. In the training phase, in order to maintain the powerful feature extraction ability and excellent spatial reasoning ability of the SAM model, the present invention freezes this Enc SAM and Dec SAM network parameters.

[0123] In detail, previous studies have shown that using multi-scale representation to capture contextual information can effectively improve the accuracy of image edge segmentation. In addition, global information can provide a more comprehensive scene understanding and plays a vital role in detecting the overall structure of abnormal objects. However, current pixel-level anomaly detection methods fail to effectively integrate multi-scale information and global context information in images. Therefore, it is difficult for existing methods to accurately segment abnormal objects in complex scenes, especially when both local details and global context need to be considered comprehensively. In order to solve the above limitations, the present invention proposes a residual full-dimensional dynamic convolution module based on a state-space model, which encodes local details and global context information as prompts and inputs them into the SAM model for pixel-level anomaly detection. Specifically, the present invention adopts a state-space model Mamba with a special structure to extract global information. Mamba can process sequence data with linear complexity, significantly improving the efficiency of sequence processing. However, due to the fixed scanning direction, Mamba is difficult to capture global information. In addition, Mamba shows a tendency to easily forget previous inputs, causing the model to prioritize the most recent inputs during processing and ignore early information in the sequence. This weakens the model's ability to fully focus on the middle area of ​​the image in the image segmentation task and leads to the neglect of abnormal information.

[0124] Further, in order to address these limitations, the present invention introduces an omni-directional state space model (Omni Dimensional-SSM, OD-SSM). OD-SSM improves the extraction of global information by scanning the input sequence in multiple directions, and it can focus on the middle and edge areas of the image at the same time. Therefore, it can effectively avoid the omission of abnormal information. In addition, the present invention uses a depthwise convolution (DW Conv) layer before OD-SSM to retain local details. It can extract neighbor features before modeling global information. The use of DW Conv can effectively capture the significant features of local areas. It is worth noting that this improvement does not prevent the proposed method from extracting global information, thereby ensuring the integration of local details with global information. The present invention uses an omni-dimensional dynamic convolution module (Omni Dimensional Dynamic Convolution, OD Conv) with different expansion rates to extract multi-scale information. OD Conv combines a multi-dimensional dynamic attention mechanism to optimize the shape of the convolution kernel and enhance feature sensitivity. This mechanism enables the convolution kernel to adaptively focus on key information in the input data, thereby enhancing the representation ability of the model. In addition, the model effectively captures multi-scale features by adopting OD-Conv with different expansion rates, which enables the model to learn features that contain local and global contexts. This complements the global information extracted by SSM and provides more comprehensive hints for the decoder.

[0125] In detail, the present invention adopts a composite loss function in training, which combines binary cross entropy (BCE) loss and Dice loss in a weighted manner. This combined function can balance prediction accuracy and boundary alignment, thereby improving the training efficiency and accuracy of pixel-level anomaly detection tasks. The BCE and Dice loss functions are defined as follows:

[0126]

[0127] Where N is the total number of samples, y i is the true label of the ith pixel, is the predicted probability of the i-th pixel. In addition, ∈ is a small constant for numerical stability. The final loss function is The definition is as follows:

[0128]

[0129] Among them, λ1 and λ2 are two balancing factors, both set to 0.5.

[0130] The state space model (SSM) originates from the continuous system, which uses the hidden state function Convert the sequence x(t) to y(t). It can be expressed as the following equation:

[0131] h′(t)=Ah(t)+Bx(t),

[0132] y(t)=Ch(t),

[0133] in is the state matrix, is the projection matrix.

[0134] The Mamba model is a discretized version of the continuous system described above. They introduce time scale parameters ΔA and ΔB to convert continuous parameters A and B into discrete parameters and The discretization rule uses zero-order hold, which is defined as follows:

[0135]

[0136] Where J is the identity matrix.

[0137] After discretizing matrices A and B, the formula can be rewritten as:

[0138]

[0139] y(t)=Ch(t).

[0140] In addition, a global convolution is used to compute the output, which is defined as follows:

[0141]

[0142] Where L represents the length of the input sequence, is a structured convolution kernel.

[0143] Embodiment 3

[0144] like Figure 2 , which is a functional module diagram of a pixel-level anomaly detection device provided for this embodiment.

[0145] The pixel-level anomaly detection device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the pixel-level anomaly detection device 100 can include a global information extraction module 101, a multi-scale feature extraction module 102, an optimal prompt generation module 103, a visual feature extraction module 104, and a mask generation module 105. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0146] In this embodiment, the functions of each module / unit are as follows:

[0147] The global information extraction module 101 is used to extract the global information of the input image using a preset prompt encoding module;

[0148] The multi-scale feature extraction module 102 is used to extract multi-scale features according to the global information using the residual full-dimensional dynamic convolution module of the omnidirectional state space model of the pre-trained points;

[0149] The optimal prompt generating module 103 is used to generate the optimal prompt according to the multi-scale feature by using the prompt encoding module;

[0150] The visual feature extraction module 104 is used to extract features from a pre-acquired input image using an encoding module of a pre-trained segmentation model to obtain visual features;

[0151] The mask generation module 105 is used to generate an anomaly detection mask according to the visual features and the optimal hint using the decoding module of the segmentation model.

[0152] Embodiment 4

[0153] Based on the above embodiments, this embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the following steps:

[0154] Extracting global information of the input image using a preset hint encoding module;

[0155] Extracting multi-scale features based on the global information using a residual full-dimensional dynamic convolution module of an omnidirectional state space model of pre-trained points;

[0156] generating an optimal prompt based on the multi-scale features using the prompt encoding module;

[0157] Using the encoding module of the pre-trained segmentation model to extract features from the pre-acquired input image, obtaining visual features;

[0158] An anomaly detection mask is generated using a decoding module of the segmentation model based on the visual features and the optimal hint.

[0159] In some implementations of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the following steps are implemented:

[0160] Extracting global information of the input image using a preset hint encoding module;

[0161] Extracting multi-scale features based on the global information using a residual full-dimensional dynamic convolution module of an omnidirectional state space model of pre-trained points;

[0162] generating an optimal prompt based on the multi-scale features using the prompt encoding module;

[0163] Using the encoding module of the pre-trained segmentation model to extract features from the pre-acquired input image, obtaining visual features;

[0164] An anomaly detection mask is generated using a decoding module of the segmentation model based on the visual features and the optimal hint.

[0165] In some implementations of this embodiment, a computer program product is provided, including a computer program, characterized in that when the computer program is executed by a processor, the steps of the method described in the above embodiment are implemented.

[0166] The processor may include, but is not limited to, one or more processors or microprocessors, etc. Each processor may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components to execute the methods in the above embodiments.

[0167] Computer-readable storage media can be implemented by any type of volatile or non-volatile storage device or a combination thereof. Computer-readable storage media may include, but are not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (such as hard disks, floppy disks, solid-state drives, removable disks, Blu-ray disks, etc.).

[0168] The computer-readable storage medium may also store at least one computer executable program, such as a computer-readable instruction. The computer-readable storage medium includes, but is not limited to, for example, a volatile memory and / or a non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The computer-readable storage medium may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0169] In addition, the computer device may also include (but not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.), etc.

[0170] The processor may communicate with external devices via an I / O bus via a wired or wireless network.

[0171] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.

[0172] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0173] It should be noted that in the present disclosure, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element limited by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0174] Although the embodiments disclosed in the present disclosure are as above, the above contents are only embodiments adopted for facilitating the understanding of the present disclosure and are not intended to limit the present disclosure. Any technician in the technical field to which the present disclosure belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present disclosure, but the scope of patent protection of the present disclosure shall still be subject to the scope defined in the attached claims.

Claims

1. A pixel-level anomaly detection method, characterized in that: include: Extracting global information of the input image using a preset hint encoding module; Extracting multi-scale features based on the global information using a residual full-dimensional dynamic convolution module of an omnidirectional state space model of pre-trained points; generating an optimal prompt based on the multi-scale features using the prompt encoding module; Using the encoding module of the pre-trained segmentation model to extract features from the pre-acquired input image, obtaining visual features; An anomaly detection mask is generated using a decoding module of the segmentation model based on the visual features and the optimal hint.

2. The method according to claim 1, characterized in that The residual full-dimensional dynamic convolution module of the omnidirectional state space model using pre-trained points extracts multi-scale features according to the global information, including: Discretizing the preset time scale parameter to obtain a discretized time scale parameter; The global information is transformed using the discretized time scale parameter to obtain transformed information; The residual full-dimensional dynamic convolution module is used to selectively scan the transformation information to obtain multi-scale features.

3. The method according to claim 2, characterized in that The step of transforming the global information by using the discretized time scale parameter to obtain the transformed information includes: The global information is initially transformed using the following formula to obtain initial transformation information: Wherein, H(t) is the preliminary transformation information, is the first discretization parameter in the discretization time scale parameter, is the second discretization parameter in the discretization time scale parameter, h(t) is a preset hidden state function, and x(t) is the global information; The following formula is used to perform final transformation on the preliminary transformation information to obtain transformation information: Y(t)=C·H(t) Among them, Y(t) is the transformation information, C is the preset projection matrix, and H(t) is the preliminary transformation information.

4. The method according to claim 1, characterized in that: Before extracting multi-scale features according to the global information using the residual full-dimensional dynamic convolution module of the omnidirectional state space model of the pre-trained points, the method further includes: Utilize a preset neural network model to output a predicted segmentation mask based on pre-acquired training data; Calculating a final loss value according to the predicted segmentation mask and a standard segmentation mask preset in the training data using a preset loss function; Judging whether the performance of the neural network model meets the requirements according to the final loss value; If the performance of the neural network model does not meet the requirements, after optimizing the hyperparameters of the neural network model according to the final loss value, returning to the step of using the preset neural network model to output a predicted segmentation mask according to the pre-acquired training data; If the performance of the neural network model meets the requirements, the model training is confirmed to be completed and an omnidirectional state space model is obtained.

5. The method according to claim 4, characterized in that The method uses a preset loss function to calculate a final loss value according to the predicted segmentation mask and a standard segmentation mask preset in the training data, include: The first loss value is calculated using the following formula: Among them, L BCE is the first loss value, N is the number of samples, is the i-th prediction result in the predicted segmentation mask, y i is the i-th standard mask in the standard segmentation mask. The second loss value is calculated using the following formula: Among them, L Dice is the first loss value, N is the number of samples, is the i-th prediction result in the predicted segmentation mask, y i is the i-th standard mask in the standard segmentation mask, and ∈ is a preset constant. The final loss value is calculated according to the first loss value and the second loss value using the following formula: L=λ1L Dice +λ2L BCE Wherein, L is the final loss value, L BCE is the first loss value, L Dice is the first loss value, and λ1 and λ2 are both preset balance factors.

6. The method according to claim 1, characterized in that The using the hint encoding module to generate an optimal hint according to the multi-scale feature includes: generating a preliminary prompt set based on the multi-scale features; obtaining quality assessment data of the preliminary prompt set; iteratively optimizing the preliminary prompt set based on the quality assessment data; When the preset number of iterations is reached, the optimal prompt is obtained.

7. A pixel-level anomaly detection device, characterized in that: include: The global information extraction module is used to extract the global information of the input image using a preset prompt encoding module; A multi-scale feature extraction module, used to extract multi-scale features according to the global information using the residual full-dimensional dynamic convolution module of the omnidirectional state space model of the pre-trained points; an optimal prompt generating module, configured to generate an optimal prompt according to the multi-scale features by using the prompt encoding module; A visual feature extraction module, used to extract features from a pre-acquired input image using an encoding module of a pre-trained segmentation model to obtain visual features; A mask generation module is used to generate an anomaly detection mask based on the visual features and the optimal hint using the decoding module of the segmentation model.

8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images