An industrial scene anomaly detection method based on multi-dimensional feature decoupling and double-track state machine

Abstract

The application discloses an industrial scene anomaly detection method based on multi-dimensional feature decoupling and a double-track state machine, which comprises the following steps: taking an industrial field monitoring video stream as data input, firstly, using a dynamic registration algorithm based on a feature anchor point and a homography matrix to perform spatial correction on the video stream, so that background shaking is eliminated; secondly, inputting the corrected frame into a heterogeneous channel in parallel, generating an initial foreground through adaptive Gaussian mixture and HSV joint filtering; then, using morphological filtering to reconstruct a target, inputting the target into a double-track timing module to be shunted to an existence track or a persistence track for judgment; finally, comprehensively integrating timing cumulative results of the tracks, and outputting an alarm signal containing an abnormal type, coordinates and a confidence degree. The application is suitable for complex scenes with many interferences, effectively solves the problems of shaking false positives and mechanical retention missed detection, and improves the detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of computer graphics and image processing, and relates to an industrial scene anomaly detection method based on multidimensional feature decoupling and dual-track state machine. Background Technology

[0002] Anomaly detection in industrial settings has become a popular research area in intelligent manufacturing and safety monitoring in recent years, with significant applications in oil extraction, chemical production, and the monitoring of large machinery. Anomaly detection can be defined as identifying events or targets that deviate from normal patterns in a continuous video stream. It typically detects the most risky and unexpected changes in the entire monitoring sequence, such as equipment leaks, smoke emissions, or mechanical component failures. While anomaly detection technologies vary, they can be categorized into three types based on their implementation path: reconstruction-based methods, prediction-based methods, and methods based on traditional image processing. Reconstruction- and prediction-based methods typically rely on deep learning models to learn the distribution of normal samples, resulting in high computational costs and strong dependence on sample data, making them difficult to implement in real-time on edge devices. In contrast, methods based on traditional image processing do not require a large number of training samples. By establishing a background model to separate the moving foreground from the background, this method is widely used due to its good real-time performance.

[0003] Existing algorithms, such as inter-frame difference, Gaussian mixture background modeling, and optical flow, are similar to static image detection. They extract features from static information such as color, contour, and texture to build a foreground detection model, and then segment abnormal objects based on this information. Considering that the essential difference between industrial video surveillance and static images is that video contains rich temporal motion information, some methods attempt to complement motion information with appearance feature information, establishing a model that combines motion information. However, current motion information acquisition is often easily affected by camera shake. For example, the swaying of outdoor poles caused by wind can lead to a large number of false edge detections in background difference. Moreover, existing methods mostly use a single parameter configuration for global detection, which cannot simultaneously adapt to multimodal targets with huge differences in physical form, such as the diffusion of smoke, the rapid jetting of fluids, and the persistence of mechanical faults. In addition, existing technologies focus more on existence detection and ignore the logical judgment of persistent states, making it difficult to identify faults such as broken or jammed links. Therefore, how to fully utilize temporal and spatial information to decouple multidimensional features in scenarios with numerous and extremely complex interference targets, and achieve accurate and efficient detection of various anomalies, has become an urgent problem to be solved. Summary of the Invention

[0004] To overcome the aforementioned shortcomings, this invention proposes an anomaly detection method for industrial scenarios based on multidimensional feature decoupling and a dual-track state machine. The specific steps of this invention are as follows:

[0005] S1, acquire the monitoring video stream of the industrial site, and initialize a heterogeneous detector containing multiple parallel detection channels based on the system preset file. The multiple parallel detection channels include a sky smoke detection channel, an equipment puncture detection channel, and a mechanical stagnation detection channel. Each channel is configured with independent detection parameters for abnormal physical forms. The independent detection parameters include a region of interest mask, a timing logic type, and a time threshold.

[0006] S2, extract the feature key points of the current video frame and perform feature matching with the reference frame, calculate the homography transformation matrix, and perform spatial correction on the current video frame to eliminate background jitter;

[0007] S3, the corrected video frames are input in parallel to each detection channel, adaptive background modeling and color space filtering are performed respectively, and an initial foreground mask is generated by feature fusion;

[0008] S4. Based on the preset physical characteristic parameters of each detection channel, the initial foreground mask is subjected to adaptive morphological filtering, and spatial constraints are applied in conjunction with the region of interest mask to obtain the target binary mask.

[0009] S5, input the target binary mask of each channel into the dual-track timing logic module, and divert the detection results to the existence detection track or the persistence detection track according to the preset logic type; wherein, the preset logic type refers to: diverting the targets of the sky smoke detection channel and the equipment puncture detection channel to the existence detection track, and diverting the targets of the mechanical stagnation detection channel to the persistence detection track; the existence detection track determines anomalies based on the total pixel area of ​​the connected domain and the time window, and the persistence detection track determines stagnation type faults based on the duration of the static foreground in a specific area;

[0010] S6, cumulatively monitors the timing status of each track and determines whether the target meets the preset alarm trigger frame number or dwell time threshold.

[0011] S7 integrates the judgment results of each track, generates and outputs an alarm signal containing the anomaly type, target coordinates and confidence level.

[0012] The technical features and improvements of this invention are as follows:

[0013] For step S2, the dynamic image stabilization preprocessing used in this invention employs a registration algorithm based on feature anchor points. This allows for the extraction of rigid feature points in the image, resolving the background difference false alarm problem caused by high-frequency camera jitter due to strong outdoor winds. In complex scenarios such as outdoor oil pumping units or offshore oil platforms, cameras are typically mounted on high poles or towers, making them highly susceptible to slight swaying caused by natural wind or equipment vibration. This subpixel-level displacement, in traditional background subtraction methods, results in a large number of bright "pseudo-foreground" noise points at the background edges, severely impacting detection accuracy. This invention introduces feature-level anchor points, allowing each frame to dynamically adjust its coordinate system based on the relative positions of feature points, "pulling" the current frame back to the coordinate system of the reference frame, thus providing a stable pixel alignment basis for the next level of detection. Specifically, firstly, an ORB feature extractor is constructed for the current frame... and reference frame Perform multi-scale pyramid decomposition, and use the FAST algorithm to detect corner points on each pyramid layer image; for any detected feature point... Define the gray centroid within its neighborhood. And calculate the principal direction of the feature points. The formula is as follows:

[0014]

[0015]

[0016]

[0017] In equations (1) to (3), Representing an image First-order geometric moments, Represents the local coordinates of the image The grayscale intensity value at that location. The sum of the total gray levels of the corresponding local area, and These correspond to the first-order moments in the horizontal and vertical directions, respectively. Equation (2) uses these moments to calculate the intensity centroid of the neighborhood of the feature point. The centroid and the geometric center (center of the circle) of the feature point usually have a fixed offset vector. Equation (3) determines the unique principal direction of the feature point by calculating the angle of this offset vector. This step is crucial because it imparts rotation invariance to the feature points, ensuring the algorithm accurately matches them even with slight camera shake. Subsequently, binary feature vectors are constructed using the BRIEF descriptor, and Hamming distance is employed to measure the similarity of feature point pairs, selecting a candidate matching set. To eliminate false matches caused by moving objects in the background (such as birds or vehicles), this invention again utilizes a random sampling consensus algorithm to... In the intermediate iteration, four pairs of random points are selected to calculate the homography matrix. This satisfies the transformation model The number of interior points is maximized. The homography matrix... Defined as:

[0018]

[0019] In equation (4), For one The perspective transformation matrix, where This determines the rotation and scaling transformations of the image. The translation transformation is determined. This determines the perspective distortion correction. This matrix can fully describe the geometric changes in the image plane when the camera shakes in three-dimensional space. Finally, for each pixel coordinate in the current frame image... Apply perspective transformation to calculate its new coordinates in the corrected coordinate system. :

[0020]

[0021] In equation (5), the original pixel coordinates are mapped to the target coordinate system through matrix multiplication, and the final coordinates are obtained by homogeneous coordinate normalization. Since the calculated new coordinates are usually floating-point numbers, this invention further employs a bilinear interpolation algorithm to obtain the pixel value at that location, thereby generating a spatially corrected video frame. This processing enables the temporal background modeling to resist environmental interference, greatly improving the detection robustness in complex scenes.

[0022] For step S3, this invention employs a parallel heterogeneous Gaussian mixture background modeling strategy. For abnormal targets with different physical forms, each detection channel independently maintains an adaptive background model. This is a method that fully utilizes temporal distribution information to separate moving foregrounds from static backgrounds. In industrial scenarios, the background is not completely static (e.g., swaying leaves, water ripples), making a single background model difficult to adapt. This invention, through multi-channel parallel processing, allows for setting different model parameters for different monitoring targets. For detection channels... Each pixel in ,use A mixture model with a Gaussian distribution is used to fit the historical pixel value distribution, with a probability density function. Defined as:

[0023]

[0024] In equation (6), The number of Gaussian components. Indicates the first Each Gaussian component in The weight of a time period reflects the frequency of occurrence of the scene corresponding to that component; Let be the mean vector of this component. Let covariance matrix be the variance matrix. The probability density function is Gaussian.

[0025]

[0026] Equation (7) describes the pixel value The probability of belonging to a certain Gaussian distribution. During each frame update, first determine the current pixel... Is it compatible with existing ones? Matching of Gaussian components, the matching condition being that the pixel value lies within a Gaussian distribution. Within 1000 standard deviations:

[0027]

[0028] In equation (8), The standard deviation is denoted as . If a match is successful, it means the pixel belongs to an existing background pattern (such as a background after a change in lighting), and the parameters of the Gaussian component need to be updated to adapt to the new environment; if there is no match, it may be a new foreground object or a new background pattern. The weight update follows the formula:

[0029]

[0030]

[0031]

[0032] In equations (9) to (11), The global learning rate determines how quickly the model adapts to changes in the background; The coefficients for updating parameters are defined as follows: The core improvement of this invention lies in the differentiated configuration of the learning rate (alpha) and variance threshold for different detection channels. For example, for the "equipment puncture leak" channel, due to the extremely high speed and short duration of the jet fluid, a larger learning rate and a lower variance threshold are set to improve sensitivity; while for the "oil accumulation on the ground" channel, since the oil accumulation is gradual, a smaller learning rate and a higher variance threshold are set to suppress false alarms caused by sudden changes in illumination. Finally, all Gaussian components are... Sort the ratios in descending order and select the first... One component is used as the background model:

[0033]

[0034] In equation (12), This is set as the background ratio threshold. This ensures that the most frequent and least variance (most stable) pixel distribution is identified as the background, while the remaining distributions are identified as the foreground, generating a binary mask. .

[0035] For step S4, in order to distinguish abnormal targets with different physical properties and solve the problem that the RGB color space has difficulty distinguishing shadows and dark objects (such as black smoke and crude oil) in industrial scenarios, this invention introduces an HSV color space joint filtering mechanism in parallel channels. First, the corrected pixels... Converted to HSV space In the conversion formula, let , , Then V and S are calculated as follows:

[0036]

[0037]

[0038] In equations (13) and (14), V (lightness) is taken directly from the maximum value in RGB, and S (saturation) reflects the purity of the color. The formula for calculating hue H is:

[0039]

[0040] Equation (15) maps color information onto a 0-360 degree color wheel, achieving decoupling of brightness and chromaticity. Subsequently, for each detection channel... Define its own HSV threshold vector and For each pixel in the image Generate color mask The discriminant formula is:

[0041]

[0042] In formula (16), the present invention has made fine-tuned settings for specific anomalies. For example, for "black oil detection", it is specifically set that... and To strictly filter low-brightness black areas and prevent the shadows of dark devices from being falsely reported as oil accumulation; for "dark smoke detection", settings are configured... and This is done to exclude highly saturated blue sky backgrounds and pure black non-smoky objects (such as birds). The final feature fusion mask is obtained through... The calculation enabled the recombination of spatiotemporal motion features and surface color features.

[0043] For step S5, this invention applies an adaptive morphological filtering strategy based on the target's physical shape, and performs refined reconstruction of the foreground mask through set operations of mathematical morphology. Addressing the contradiction between "fragile smoke" and "corrosive jet fluid" in industrial scenarios, this invention employs differentiated structural elements. (Definition of dilation operation) Erosion operation For the "smoke detection channel", a large-size elliptical kernel is used. (For example or ), perform the closing operation:

[0044]

[0045] In equation (17), the closing operation is a process of expansion followed by erosion. Since smoke is often visually diffuse, semi-transparent, and uneven, the initial detection results often contain numerous voids and breaks. The closing operation can fill these voids, fusing discrete smoke pixel clusters into a complete connected region, thereby avoiding missed detections due to the thinness of the smoke. For the "high-pressure jet detection channel," a small-sized elliptical kernel is used. (For example Only the opening operation is performed:

[0046]

[0047] In equation (18), the opening operation is a process of erosion followed by expansion. High-pressure jet fluids (such as oil mist and water mist) are usually fine lines or mists with rich texture details but fragile structures. If a large kernel is used for erosion, these details will be directly erased. Therefore, this invention only uses the small kernel opening operation to remove random noise from a single pixel, preserving the fine edge features of the fluid to the greatest extent.

[0048] For step S6, this invention innovatively introduces a dual-track timing state logic to solve the problem that traditional methods cannot detect "stagnation-type" faults. The "existence detection track" is mainly applied to anomalies such as liquid leaks and gas emissions. First, the total pixel area of ​​all connected components is calculated:

[0049]

[0050]

[0051] Equations (19) and (20) define the abnormal state of a single frame when the detected area exceeds a threshold. The time stamp is marked as abnormal. To suppress transient interference (such as light flicker), a length of [length missing] is introduced. Sliding time window queue :

[0052]

[0053]

[0054] Equations (21) and (22) obtain the alarm confidence level by calculating the proportion of abnormal frames within the window. Only when the confidence level rises smoothly and exceeds the threshold The final alarm is triggered only at certain times. The "persistent detection track" is specifically designed for detecting mechanical component breakage, jamming, or foreign object retention (such as broken and accumulated debris in a pumping unit connecting rod). The system maintains a global persistent counter. Its state transition equation is as follows:

[0055]

[0056] In equation (23), This indicates whether a stationary foreground exists within the region of interest (ROI) in the current frame. The physical meaning of this logic is: if a stationary foreground (i.e., a mechanical component stops moving or broken parts accumulate) is continuously detected within the monitored ROI, the counter increments; if the region returns to its background state (i.e., the mechanical component periodically moves out of the region as expected), the counter resets to zero. A time threshold is set. When the conditions are met When a "delayed fault" is detected, an alarm is activated. Set to True, and the alarm type will be labeled "STUCK / BROKEN".

[0057] The industrial scene anomaly detection method based on multidimensional feature decoupling and dual-track state machine of the present invention solves the problems of large differences in the morphology of abnormal targets and poor real-time performance in complex scenes in existing technologies, and has the following advantages:

[0058] (1) The method of the present invention introduces dynamic image stabilization technology based on feature anchor points in scenes with many interfering targets and extremely complex scenes. This allows for the extraction of rigid feature points in the image, solving the problem of false alarms in background difference caused by high-frequency camera shaking due to strong outdoor winds, and greatly improving the accuracy of target object detection and segmentation in complex scenes.

[0059] (2) The method of the present invention can be applied to the complex scenario of offshore platforms or field oil well sites, realizing the compatible detection of multimodal anomaly targets. Through the parallel heterogeneous parameter decoupling mechanism, the accurate capture of diffuse black smoke, textured high-pressure leaks and static oil accumulation is realized in the same system at the same time, breaking the limitation that the traditional single model cannot adapt to multiple abnormal physical forms.

[0060] (3) The method of the present invention can make full use of the temporal structure and contextual information of the video, and innovatively proposes the "dual-track temporal state machine" logic, which improves the defect that traditional motion detection can only detect moving targets. It successfully solves the problem of automatic judgment of "stagnant" faults such as mechanical component breakage or jamming through persistent counter, and can adapt to video input of any length and provide stable alarm output. Attached Figure Description

[0061] Figure 1 This is a schematic diagram illustrating the dynamic image stabilization and registration principle based on feature anchor point matching in this invention.

[0062] Figure 2 This is a schematic diagram of the internal processing architecture of the parallel heterogeneous detection channel in this invention;

[0063] Figure 3 This is a comparative schematic diagram of the logic decision process of the dual-track sequential state machine in this invention. Detailed Implementation

[0064] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0065] An industrial scene anomaly detection method based on multidimensional feature decoupling and dual-track state machine, such as... Figure 1 The diagram shown is an overall flowchart of the present invention, and the method includes the following specific implementation steps:

[0066] S1, Industrial Video Stream Data Acquisition and Multi-Channel Detection Environment Initialization: Using industrial monitoring video information as input, a sequence preprocessing is performed on a continuous monitoring video stream. In this embodiment, the system is deployed in an edge computing terminal at an oil extraction site or chemical industrial park for 24 / 7 intelligent monitoring of the production area. First, monitoring video streams are acquired using industrial-grade high-definition cameras, and the video data is decoded into a continuous digital image sequence. To balance real-time performance and detection accuracy with limited computing resources, this embodiment sets a specific processing frame rate of five frames per second and a standard input image resolution. During the initialization phase, the system reads a preset configuration file, constructs and instantiates a heterogeneous detector containing multiple parallel branches. Unlike traditional single-mode detectors, this detector internally divides into three independent detection channels for different physical anomalies. The first channel is for detecting black smoke in the sky, with its area of ​​interest set to the sky background at the top of the monitoring screen. It is specifically used to detect plumes of black smoke produced by incomplete combustion. The second channel is for detecting equipment leaks, with its area of ​​interest set to the wellhead, valves, and surface manifold areas. It is specifically used to detect mist-like anomalies caused by high-pressure fluid jets. The third channel is for detecting mechanical stagnation, with its area of ​​interest precisely defined as the rectangular space at the bottom of the pumping unit's stroke. It is specifically used to detect mechanical failures such as broken connecting rods or equipment jamming. Through this initial channel isolation, the system can simultaneously process three distinctly different anomalies—gas, liquid, and solid—within the same video stream, providing an architectural foundation for subsequent feature decoupling.

[0067] S2, Dynamic Image Stabilization and Registration Based on Feature Anchor Point Matching: This step extracts key feature points from the current video frame and performs feature matching with a reference frame. Addressing the issue that cameras in outdoor scenes are often mounted on tall poles or towers, making them highly susceptible to high-frequency, minute displacements caused by wind or mechanical vibrations, this embodiment introduces a dynamic image registration step based on feature anchor points before background modeling. This step aims to establish a stable spatiotemporal coordinate system and eliminate false alarms in background difference caused by camera shake. Specifically, the system selects a historical image with good lighting conditions and no abnormal occlusion as the reference frame. For the current video frame input in real time, the system first converts it to a grayscale image and uses a fast corner detection and rotation-robust feature description algorithm to extract key feature points from the image. These feature points are typically located in textured and rigid areas such as equipment edges and building corners. To calculate the orientation of the feature points with rotation invariance, the algorithm calculates the grayscale centroid within the neighborhood of the feature point and defines the vector pointing from the geometric center to the grayscale centroid as the principal direction. Subsequently, the Hamming distance of the binary descriptor is used to match feature points between the current frame and the reference frame, with a matching threshold of 0.75 to ensure matching quality. Considering the possibility of mismatches caused by moving objects such as birds and vehicles in complex industrial backgrounds, this embodiment further introduces a random sampling consensus algorithm to eliminate outlier point pairs, setting a reprojection error threshold of 5.0 pixels. This algorithm calculates the optimal homography transformation matrix that can contain the most interior points through iterative sampling. This matrix describes the rotation, translation, and scaling relationship of the current frame relative to the reference frame in the plane. Finally, perspective projection correction is performed on the current video frame using the inverse transformation of this matrix, and bilinear interpolation is used to fill the pixels. After this step, even if the camera shakes, the corrected video frame can still maintain strict alignment with the reference frame in pixel coordinates, thus ensuring that the subsequent background subtraction algorithm can accurately distinguish between real moving targets and background shaking noise.

[0068] S3, Parallel Heterogeneous Gaussian Mixture Background Modeling and Foreground Segmentation, inputs the corrected video frames in parallel to each detection channel for adaptive background modeling. The core of this embodiment lies in configuring drastically different model update parameters for the physical target characteristics of different channels, achieving decoupling at the feature extraction level. In the sky smoke detection channel, considering that smoke is usually thin in its early stages, has low contrast with the sky background, and easily casts shadows, the system is configured with a high sensitivity parameter, i.e., a low variance threshold, allowing the algorithm to capture minute grayscale changes. Simultaneously, this channel enables a shadow detection mechanism, distinguishing the shadows cast by clouds or smoke from the smoke itself by evaluating the brightness attenuation and chromaticity preservation characteristics of the foreground region. In the device leak detection channel, considering that high-pressure fluid jets typically have sharp edges, high speed, and high opacity, the system is configured with a low sensitivity parameter, i.e., a high variance threshold. This strategy effectively filters out minute noise such as airborne dust and insects, retaining only jets with significant motion characteristics. Furthermore, shadow detection is disabled in this channel to reduce computational load and avoid misclassifying dark oil as shadows. In the mechanical stagnation detection channel, the system employs a moderate sensitivity setting and utilizes a long history window to adapt to slow changes in ground lighting, ensuring the stability of the background model and providing an accurate static foreground area for subsequent logical judgments.

[0069] S4, Multi-dimensional feature decoupling and color space joint filtering, generates an initial foreground mask through feature fusion. Relying solely on motion features is insufficient to distinguish targets with different physical properties; for example, white high-temperature steam and black combustion smoke may appear similar in motion, but their hazard levels are drastically different. Therefore, this embodiment further introduces a color space filtering mechanism in each detection channel. The system converts the corrected image from the red-green-blue color space to the hue-saturation-brightness color space, utilizing the brightness and chroma separation characteristic of this space for precise segmentation. For black smoke detection, the system sets strict upper limits for saturation and brightness. Since the background sky typically exhibits high saturation blue, while smoke appears low saturation gray, limiting the saturation threshold effectively eliminates the sky background; simultaneously, limiting the brightness threshold excludes white water vapor, ensuring that the detected target is dark gray smoke. For detecting oil or dark fluids on the ground, the system sets extremely low brightness thresholds and a full range of saturation thresholds, aiming to lock in areas of high light absorption and low reflectivity in the image, thereby eliminating interference from device shadows or wet ground. By performing a logical AND operation between the moving foreground mask obtained from background modeling and the color foreground mask obtained from color filtering, only pixels that simultaneously satisfy the conditions of "being in motion" and "having a color that conforms to specific physical properties" are retained. This fusion of spatiotemporal and spectral features significantly reduces the false alarm rate and enables the accurate capture of specific anomalous targets.

[0070] S5, Adaptive Morphological Filtering and Target Connected Part Reconstruction: Based on the preset physical characteristic parameters of each detection channel, adaptive morphological filtering is performed on the initial foreground mask. The initial foreground mask generated after the above steps often contains noise or broken areas and cannot be directly used for alarm determination. This embodiment applies differentiated mathematical morphological processing strategies according to the topological morphology of different abnormal targets. For diffuse targets such as smoke, it often appears visually as discontinuous clumps with blurred edges. Therefore, in the smoke detection channel, the system uses large-sized elliptical structuring elements to perform closing operations. The closing operation involves expansion followed by erosion, which effectively fills the voids inside the smoke clumps and connects neighboring discrete pixels into a complete connected part, thereby not only improving the visual effect of detection but also facilitating the subsequent calculation of the overall area of ​​the smoke clump. Conversely, for textured targets such as high-pressure leaks, the ejected fluid usually appears as fine lines or mist, with a relatively fragile structure. If large-sized kernels are used for processing, their detailed features are easily destroyed. Therefore, in the leak detection channel, the system uses extremely small-sized structuring elements to perform only opening operations. The opening operation involves erosion followed by dilation, aiming to remove random electronic noise from individual pixels while preserving the subtle edges and textures of the jet fluid to the greatest extent possible, preventing missed detections due to over-smoothing. The final binary mask is then intersected with a pre-defined region of interest mask to ensure all alarm signals are strictly confined to the hazardous areas specified in the process flow.

[0071] S6, Dual-track temporal state machine logic judgment and anomaly classification, inputs the target binary mask of each channel into the dual-track temporal logic module. This is a key step in solving complex logic anomalies in this embodiment. The post-processing module receives the detection results of each channel and splits them into two independent temporal tracks according to the preset logic type. The first track is the existence detection track, mainly used for black smoke and puncture leakage channels. In this track, the algorithm calculates the total pixel area of ​​the target connected region in the current frame. If the area exceeds the preset minimum threshold, the frame is marked as a suspected anomaly. To suppress instantaneous interference caused by sudden changes in light or camera autofocus, the system maintains a sliding time window queue, for example, with a length of thirty frames. The system only confirms an anomaly when the proportion of accumulated abnormal frames or the number of consecutive abnormal frames in the queue exceeds the set confidence threshold. The second track is the persistence detection track, specifically used to solve the problem of mechanical fault detection. Taking an oil pump as an example, its pump head makes periodic up-and-down reciprocating movements during normal operation. The system delineates the detection area at the bottom of the pump head's movement stroke. Under normal operating conditions, the donkey head periodically enters and leaves the area, causing the foreground mask in that area to appear and disappear periodically, resulting in the persistence counter being continuously reset to zero. However, when a connecting rod breaks and falls, or the pumping unit jams at the bottom, a stationary foreground object will remain in the area. In this case, the persistence counter will accumulate linearly over time. The system has a time threshold set that is longer than the normal motion cycle, for example, one hundred frames. Once the counter value exceeds this threshold, the system determines that an "unexpected prolonged stay" has occurred, and thus outputs a mechanical fault alarm.

[0072] S7, multi-dimensional anomaly information fusion and visualized alarm output, integrates the judgment results of each track to generate an alarm signal containing anomaly type, target coordinates, and confidence level. After completing all the above logical judgments, the system integrates the output results of each track. If any channel triggers an alarm condition, the system will generate a structured alarm event containing anomaly type, occurrence time, target location coordinates, and confidence level. To facilitate quick confirmation by monitoring personnel, the system overlays visualized enhanced information onto the original video stream. For different types of anomalies, the system draws different colored bounding boxes for identification; for example, a red border is used to identify high-risk puncture leaks, a purple border to identify oil spills on the ground, and a yellow border and text prompts to identify mechanical component breakage or jamming faults. Simultaneously, the system extracts and saves alarm segments to local storage or uploads them to a cloud server, forming a complete anomaly event log, providing detailed data support for subsequent safety production traceability. In summary, the method in this embodiment constitutes a closed-loop, robust intelligent monitoring system with extremely high application value in high-risk industrial scenarios such as petrochemicals.

[0073] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. An industrial scene anomaly detection method based on multi-dimensional feature decoupling and dual-track state machine, characterized in that, The method comprises the following steps: S1, obtaining a monitoring video stream of an industrial site, and initializing a heterogeneous detector comprising multiple parallel detection channels based on a system preset file, the multiple parallel detection channels comprising a sky smoke detection channel, a device leakage detection channel, and a mechanical stagnation detection channel; each channel is configured with independent detection parameters for abnormal physical morphology, the independent detection parameters comprising a region of interest mask, a timing logic type, and a time threshold; S2, extracting feature key points of a current video frame and performing feature matching with a reference frame, calculating a homography transformation matrix, and performing spatial correction on the current video frame to eliminate background jitter; S3, inputting the corrected video frame into each detection channel in parallel, respectively performing adaptive background modeling and color space filtering, and generating an initial foreground mask through feature fusion; S4, performing adaptive morphological filtering on the initial foreground mask according to the physical characteristic parameters preset by each detection channel, and combining the region of interest mask to perform spatial constraint to obtain a target binary mask; S5, inputting the target binary mask of each channel into a double-track timing logic module, and according to the preset logic type, the detection results are divided into an existence detection track or a persistence detection track; wherein, the preset logic type refers to: the target of the sky smoke detection channel and the device leakage detection channel is divided into the existence detection track, and the target of the mechanical stagnation detection channel is divided into the persistence detection track; the existence detection track determines the abnormality based on the total pixel area of connected domains and a time window, and the persistence detection track determines the stagnation type fault based on the duration of stationary foreground in a specific region; S6, accumulating monitoring of the timing state of each track, and determining whether the target meets the preset alarm trigger frame number or stagnation duration threshold; S7, comprehensively determining the results of each track, generating an alarm signal comprising an abnormal type, target coordinates, and confidence, and outputting.

2. The industrial scene anomaly detection method based on multi-dimensional feature decoupling and dual rail state machine according to claim 1, characterized in that, For step S2, the spatial correction of the current video frame is specifically implemented by using a dynamic image stabilization and registration method based on feature anchor point matching, and the specific mathematical implementation process is as follows: first, an ORB feature extractor is constructed, and the current frame and the reference frame are subjected to multi-scale pyramid decomposition, and the FAST algorithm is used to detect the corner points on each layer of the pyramid image; for any detected feature point , the gray centroid in the neighborhood thereof is defined , and the main direction of the feature point is calculated , and the formula is as follows: wherein denotes the image of denotes the image of the gray intensity value at the local coordinate denotes the total gray sum of the corresponding local region; and denotes the first moment of the horizontal and vertical direction, respectively; then the binary feature vector is constructed by using the BRIEF descriptor;​​ Secondly, the Hamming distance between two sets of feature points is calculated by using the brute force matcher, and the point pairs with distance less than a pre-set threshold are selected as the candidate matching set . ​ Again, we use RANSAC algorithm to select 4 pairs of random points from and compute the homography matrix that maximizes the number of inliers that satisfy the transformation model where denotes the source feature point coordinates in the candidate matching set; the homography matrix is the matrix: Finally, for each pixel coordinate in the current frame image Apply the perspective transformation to compute its new coordinates in the rectified coordinate system : The corrected pixel value is obtained through bilinear interpolation, thereby completing image registration.

3. The industrial scene anomaly detection method based on multi-dimensional feature decoupling and dual rail state machine according to claim 1, characterized in that, For step S3, the adaptive background modeling specifically adopts a parallel heterogeneous Gaussian mixture background modeling algorithm, whose specific algorithm logic is as follows: for each pixel in the detection channel , a mixture model of Gaussian distributions is used to fit its historical pixel value distribution, and the probability density function is defined as:​ where, is the weight of the th Gaussian component at time , is the mean, is the covariance matrix, is the Gaussian probability density function: At each frame update, first determine whether the current pixel matches any of the existing Gaussian components, which is the case if the pixel value lies within times the standard deviation of the Gaussian distribution: If the match is successful, the weight, mean and variance of the Gaussian component are updated; the update of the weight follows the formula below, where is the learning rate: wherein If not, a new Gaussian component is created to replace the one with the smallest weight. Finally, all Gaussian components are sorted in descending order of the ratio of , and the first components are selected as the background model, and the rest as foreground. wherein is a background ratio threshold; if the component matched by the current pixel belongs to the background for the previous n frames, then it is determined as background, otherwise as foreground, generating a binary mask .

4. The industrial scene anomaly detection method based on multi-dimensional feature decoupling and dual rail state machine according to claim 1, characterized in that, For step S4, the color conversion and threshold segmentation logic in the color space joint filtering is as follows: first, the corrected pixel converted into HSV space , the conversion formula is: let , , then: Subsequently, for each detection channel , a dedicated HSV threshold vector and is defined; for each pixel in the image, a color mask is generated For black oil liquid detection, special settings are made and to strictly screen low brightness areas; For dark smoke detection, set and to exclude high-saturation blue sky backgrounds and non-smoke objects that are pure black.

5. The industrial scene anomaly detection method based on multi-dimensional feature decoupling and dual rail state machine according to claim 1, characterized in that, For step S5, the mathematical expression and application strategy of the adaptive morphological filter are as follows: define the original binary mask image as , the structural element as , the dilation operation as , the erosion operation as , the open operation as , which is used to remove isolated noise points smaller than the structural element , the close operation as , which is used to fill the internal cavities of objects and connect adjacent broken areas, and different structural elements are adopted for different detection channels: (1) for the smoke detection channel, a large-size elliptical kernel is adopted to perform the close operation: wherein, a target binary mask representing the smoke detection channel, a feature fusion mask representing the smoke detection channel, a region of interest mask representing the smoke detection channel; the purpose is to fuse the diffuse and semi-transparent smoke pixel blocks into a complete connected domain, and prevent detection fragmentation caused by thin smoke; (2) for the high-pressure jet detection channel, a small size elliptical kernel is only executed open operation. wherein, represents a target binary mask of the high-pressure jet detection channel, represents a feature fusion mask of the high-pressure jet detection channel, represents a region of interest mask of the high-pressure jet detection channel; the purpose is to retain the fine edge features of the high-pressure jet fluid to the greatest extent while removing single-pixel random noise, preventing missed detection due to excessive corrosion; (3) the final target binary mask is the intersection of the morphological processing result and the static ROI mask, ensuring that the detection result is strictly limited within the preset physical space range.

6. The industrial scene anomaly detection method based on multi-dimensional feature decoupling and dual rail state machine according to claim 1, characterized in that, For step S5, the specific logical decision process of the "existence detection track" is as follows: the track is mainly applied to liquid leakage, gas emission and other abnormalities; for the first Frame detection result , first calculate the total pixel area of all connected domains : Defining single frame abnormal state variables : wherein a minimum effective area threshold preset for the channel; to suppress transient interference, a sliding time window queue is introduced , with a length of ; Computing an alarm confidence : When wherein is an alarm trigger threshold, and the current frame status the final alarm signal is activated.

7. The industrial scene anomaly detection method based on multi-dimensional feature decoupling and dual rail state machine according to claim 1, characterized in that, For step S5, the specific logic decision process of the "persistent detection track" is as follows: the track is mainly applied to the detection of mechanical part fracture, jamming or foreign matter retention; the system maintains a global persistent counter , the initial value of which is 0; at the moment , the target existence state is calculated according to the detection mask ; the state transition equation is as follows: The physical meaning of this logic is: if stationary foreground is continuously detected in the specific mechanical motion area under monitoring, the counter is accumulated; if the area returns to background state, the counter is cleared; a time threshold is set ; when the condition is met, it is determined that a "stuck fault" occurs, at which time the alarm state is set to True and the alarm type is marked as "STUCK / BROKEN".

8. The industrial scene anomaly detection method based on multi-dimensional feature decoupling and dual rail state machine according to claim 1, characterized in that, The system preset file in step S1 is a system configuration parameter set In the set, the specific configuration parameters for "pumping unit connecting rod detection" Have the following characteristics: its region of interest mask Is accurately drawn as the rectangular space where the connecting rod device is located when the pumping unit horse head moves to the bottom end Type of its sequential logic Set as "persistent detection track"; its time threshold Set as greater than the maximum duration of the horse head staying at the bottom end in the normal operation cycle of the pumping unit The working principle is: using the mixed Gaussian background model to adapt to the slow changing background, and continuously capture the broken connecting rod which suddenly appears and long stays; if the connecting rod moves normally, its stay time in ROI is not only shorter than , but also will periodically leave to make zero, so as to reset the counter and avoid false positives; only when the connecting rod is broken and falls and accumulates in the ROI, or the machine is stuck at the bottom and does not move, the counter will break through the threshold to trigger the alarm.

Images