Multi-type image quality anomaly detection system and method

Abstract

The invention relates to a multi-type image quality anomaly detection system and method. The method comprises the following steps: acquiring a to-be-detected image or video stream, and preprocessing; carrying out at least two types of image quality anomaly detection, the types of image quality anomaly detection comprise jitter anomaly detection, scene change detection, shielding anomaly detection, lagging anomaly detection, freezing anomaly detection, brightness anomaly detection, strong light anomaly detection, black and white anomaly detection, color cast anomaly detection, stripe anomaly detection, fuzzy anomaly detection and random stripe noise anomaly detection; and summarizing and outputting image quality anomaly detection results. According to the technical scheme provided by the invention, 12 core image quality anomaly types are covered, the detection comprehensiveness is high, each anomaly detection module adopts targeted optimization logic, the method can adapt to a complex scene, the false detection and omission ratio is remarkably reduced, each anomaly detection algorithm can be independently deployed or integrated due to the modular design, the expansibility is high, and different monitoring scene requirements are met.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, specifically to a multi-type image quality anomaly detection system and method applicable to video surveillance scenarios (especially complex industrial scenarios such as mines). Background Technology

[0002] In video surveillance systems, image quality anomalies can severely impact the validity of monitoring data, potentially leading to overlooked security risks and misjudgments of equipment malfunctions. Existing image quality anomaly detection technologies have the following shortcomings: 1. Single anomaly detection algorithms have poor adaptability and are prone to false detection and missed detection in complex scenarios (such as the coexistence of light and dark in mines and interference from large machinery); 2. The integrated detection of multiple anomaly types consumes a lot of computing power, making it difficult to meet the needs of real-time monitoring; 3. The detection logic is simple (such as the traditional frame difference method), and it cannot distinguish similar abnormal features (such as jitter and scene change). 4. The system deployment is cumbersome, requiring manual configuration of parameters and location of key areas (such as timestamps), and has poor compatibility; 5. For some types of anomalies (such as strong light anomalies and random stripe noise), there is a lack of targeted detection schemes, and existing methods are not robust enough.

[0003] Therefore, there is an urgent need for an intelligent detection technology that can cover multiple types of image quality anomalies, has high detection accuracy, low computing power consumption, is easy to deploy, and can adapt to complex scenarios. Summary of the Invention

[0004] This invention provides a multi-type image quality anomaly detection system and method. The system adopts a modular design, with each anomaly detection module being independent yet integrated. Furthermore, efficient deployment is achieved through system optimization. The specific technical solution is as follows: According to a first aspect of the present invention, a multi-type image quality anomaly detection system is provided, comprising: The data acquisition module is used to acquire the image or video stream to be detected and perform preprocessing. The multi-anomaly detection module includes at least two types of image quality anomaly detection sub-modules, which are selected from the following sub-modules: jitter anomaly detection sub-module, scene change detection sub-module, occlusion anomaly detection sub-module, stuttering anomaly detection sub-module, freeze anomaly detection sub-module, brightness anomaly detection sub-module, strong light anomaly detection sub-module, black and white anomaly detection sub-module, color cast anomaly detection sub-module, stripe anomaly detection sub-module, blur anomaly detection sub-module, and random stripe noise anomaly detection sub-module. The results output module is used to summarize and output the detection results of multiple anomaly detection modules.

[0005] Furthermore, the jitter anomaly detection submodule includes a corner extraction unit, an optical flow tracking unit, a reciprocating motion analysis unit, and a jitter determination unit. The corner extraction unit is used to extract corners using the Shi-Tomasi algorithm. The optical flow tracking unit is used to determine the motion trajectory of the corners using the LK optical flow method. The reciprocating motion analysis unit is used to determine the direction reversal by calculating the angle between the motion vectors of the corners. The jitter determination unit is used to determine the occurrence of jitter anomalies by the ratio of the number of points with direction reversal to the total number of tracked corners.

[0006] Furthermore, the scene change detection submodule is used to determine scene changes based on three indicators: motion consistency of corner points in the image, motion participation, and frequency of direction change.

[0007] Furthermore, the occlusion anomaly detection submodule is used to determine occlusion based on the temporal features of edge density from Canny edge detection, triggered by scene changes or timed detection.

[0008] Furthermore, the stuttering anomaly detection submodule is used to extract timestamps using an OCR model, optimize recognition accuracy through bilateral filtering and Laplacian sharpening, and determine stuttering through timestamp difference analysis.

[0009] Furthermore, the freeze anomaly detection submodule is used to extract timestamps using an OCR model. When the extracted timestamp "second" value remains unchanged for multiple consecutive periods, it is determined that the image is frozen.

[0010] Furthermore, the brightness anomaly detection submodule is used to calculate the average brightness of the grayscale image. When the average brightness exceeds the overbrightness threshold, it is determined to be an overbrightness anomaly; when the average brightness is below the underbrightness threshold, it is determined to be an underbrightness anomaly.

[0011] Furthermore, the strong light anomaly detection submodule is used to set a dynamic highlight threshold based on the maximum grayscale value of the image; perform binarization and morphological operations on the image grayscale image; determine the largest connected highlight region and calculate the proportion of its area to the total image area; when the proportion exceeds the dynamic highlight threshold, it is determined to be a strong light anomaly.

[0012] Furthermore, the black-and-white anomaly detection submodule is used to detect anomalies by judging the number of channels and analyzing the consistency of pixel values ​​in the three channels. Single-channel images are directly judged as black and white, while in three-channel images, when the absolute difference of the R, G, and B channel values ​​of more than 95% of the pixels is less than a preset fault tolerance threshold, it is judged as a black-and-white anomaly.

[0013] Furthermore, the color cast anomaly detection submodule is used to calculate the ratio of the maximum to the minimum value of the mean values ​​of the R, G, and B channels. When the ratio exceeds a preset threshold, it is determined to be a color cast anomaly.

[0014] Furthermore, the stripe anomaly detection submodule is used to calculate the absolute difference between two adjacent rows of pixels in the image to obtain an absolute difference matrix; it calculates the proportion of zero values ​​in each row of the absolute difference matrix and the sum of each row; it filters similar rows based on the sum and the proportion of zero values, and if the proportion of similar rows to the total number of rows is less than a set threshold, it is determined that there is no stripe anomaly; it performs a blackening operation on the areas of dissimilar rows, uses the Canny operator to extract edges, and uses the Hough transform to detect straight lines in the image; it filters straight lines with an angle close to 90°, and when the number of filtered straight lines exceeds a threshold, it is determined to be a stripe anomaly.

[0015] Furthermore, the blur anomaly detection submodule is used to convert the image into a grayscale image, expand the image boundary using boundary copying, calculate the image contrast and perform normalization processing, and determine the blur anomaly when the normalized contrast is lower than a preset threshold.

[0016] Furthermore, the random stripe noise anomaly detection submodule is used to smooth the image using a 3x3 median filter to obtain a smoothed image; the proportion of noise pixels is obtained by calculating the difference between the original image and the smoothed image, and when the proportion exceeds a threshold, it is determined to be a random stripe noise anomaly.

[0017] According to a second aspect of the present invention, a method for detecting multiple types of image quality anomalies is provided, comprising: Acquire the image or video stream to be detected and perform preprocessing; Perform at least two types of image quality anomaly detection, including jitter anomaly detection, scene change detection, occlusion anomaly detection, stuttering anomaly detection, freeze anomaly detection, brightness anomaly detection, strong light anomaly detection, black and white anomaly detection, color cast anomaly detection, stripe anomaly detection, blur anomaly detection, and random stripe noise anomaly detection. Summarize and output the results of image quality anomaly detection.

[0018] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: It covers 12 core image quality anomaly types, with strong detection comprehensiveness. Each anomaly detection module adopts targeted optimization logic, which can adapt to complex scenarios and significantly reduce the false detection and false negative rates. The modular design allows each anomaly detection algorithm to be deployed or integrated independently, with strong scalability and adaptability to different monitoring scenario requirements.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0020] The above and other objects, features and advantages of the present invention will become more apparent from the more detailed description of exemplary embodiments of the invention in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the invention.

[0021] Figure 1 This is a structural block diagram of a multi-type image quality anomaly detection system according to an exemplary embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for detecting multiple types of image quality anomalies according to an exemplary embodiment of the present invention. Detailed Implementation

[0022] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0023] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0024] It should be understood that although the terms "first," "second," "third," etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, features defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0025] This system can be deployed on Linux or Windows operating systems, supports Intel CPU platforms, and accesses real-time video streams or offline video files from surveillance cameras via a data acquisition module. After system startup, it automatically detects the operating environment and the number of CPU cores, configures the number of MKL and OMP threads, and automatically locates the timestamp region in the video. Users can confirm or adjust this information through the interface. The technical solution of this invention is described in detail below with reference to the accompanying drawings.

[0026] Figure 1 This is a structural block diagram of a multi-type image quality anomaly detection system according to an exemplary embodiment of the present invention.

[0027] See Figure 1 The system includes: The data acquisition module is used to acquire the image or video stream to be detected and perform preprocessing. The multi-anomaly detection module includes at least two types of image quality anomaly detection sub-modules, which are selected from the following sub-modules: jitter anomaly detection sub-module, scene change detection sub-module, occlusion anomaly detection sub-module, stuttering anomaly detection sub-module, freeze anomaly detection sub-module, brightness anomaly detection sub-module, strong light anomaly detection sub-module, black and white anomaly detection sub-module, color cast anomaly detection sub-module, stripe anomaly detection sub-module, blur anomaly detection sub-module, and random stripe noise anomaly detection sub-module. The results output module is used to summarize and output the detection results of multiple anomaly detection modules.

[0028] Optionally, in this embodiment, the jitter anomaly detection submodule includes a corner point extraction unit, an optical flow tracking unit, a reciprocating motion analysis unit, and a jitter determination unit. The corner point extraction unit is used to extract corner points using the Shi-Tomasi algorithm. The optical flow tracking unit is used to determine the motion trajectory of the corner points using the LK optical flow method. The reciprocating motion analysis unit is used to determine the direction reversal by calculating the angle between the motion vectors of the corner points. The jitter determination unit is used to determine the occurrence of jitter anomalies by the ratio of the number of points with direction reversal to the total number of tracked corner points.

[0029] Specifically, the key difference between jitter and normal motion lies in its reciprocating nature. Therefore, this application, in addition to the previous analysis of motion displacement characteristics, further adds the analysis of corner motion patterns.

[0030] To capture jitter of varying amplitudes (from minute vibrations to violent shaking), this application introduces an adaptive multi-scale detection mechanism. This mechanism dynamically adjusts the distance threshold used to determine effective motion based on the average amplitude of corner point motion, thereby achieving sensitive capture of jitter signals with different energy levels.

[0031] This application employs the Shi-Tomasi corner detection algorithm to find stable feature points in video frames. This algorithm identifies corner points by calculating the gradient covariance matrix of a local image region. For a pixel (x, y), the gradient covariance matrix M within its surrounding window is defined as:

[0032] Where I represents the gradient of the image in the x and y directions. This is achieved by calculating the eigenvalues ​​of matrix M. and When both eigenvalues ​​are large, the point is identified as a corner. Specifically, the mass R of a corner is defined as:

[0033] When R is greater than a preset threshold, the point is identified as a corner point.

[0034] The Lucas-Kanade (LK) optical flow method is used to track corner points across consecutive frames, forming motion trajectories. The basic assumption of the LK optical flow method is that the grayscale value of a pixel remains constant within a small spatiotemporal neighborhood. For a point (x, y) with a grayscale value I(x, y, t) at time t, after moving (dx, dy) at time t+dt, we have:

[0035] By Taylor expansion and neglecting higher-order terms, we can obtain the optical flow constraint equation:

[0036] in and It is the optical flow vector. A robust optical flow estimate can be obtained by solving the overdetermined equations within a small window.

[0037] The following analysis examines the directional changes in the trajectory of each corner point. When a corner point undergoes a near 180-degree "turnaround" motion consecutively within a short period, it is marked as a "reciprocating motion point." The criterion for determining a "turnaround" is two consecutive motion vectors. and The angle between :

[0038] when When the angle exceeds a threshold (e.g., 130 degrees), it is considered that a direction reversal has occurred.

[0039] When the proportion of "reciprocating motion points" in the image to the total number of tracking corner points exceeds a preset threshold T. jitter Furthermore, their absolute number also exceeds a minimum threshold N. min When this occurs, the system determines that a jitter anomaly has occurred, and the judgment logic is as follows:

[0040] Optionally, in this embodiment, the scene change detection submodule is used to determine scene changes based on three indicators: motion consistency of corner points in the image, motion participation, and frequency of direction change.

[0041] Specifically, scene change anomalies are usually caused by the camera lens actively or passively turning or moving, resulting in image changes. Current technology generally uses frame difference to detect image changes, specifically calculating the similarity and translation value between adjacent frames. If a certain threshold is reached, it is considered an anomaly. However, this detection logic is too simplistic and can lead to false positives when there are significant changes in the image or other image anomalies (such as flashes). It can also miss detections if the image is moving slowly. A filtering logic is added to the frame difference method: the frame to be analyzed is inspected for image quality. If an anomaly is found, the 10th frame is selected for inspection. This mechanism selects a maximum of three consecutive frames. If three consecutive frames show image anomalies, the third frame is selected for scene change detection. This logic reduces false positives caused by sudden flickering or halos, but false alarms still exist.

[0042] To address the problems with the above detection methods, the technical solution of this invention analyzes the motion patterns of corner points. Analysis of a large amount of data reveals that the motion patterns of scene changes have significant characteristics: the vast majority of corner points in the image move along a continuous, segmented, and unidirectional direction, and the frequency of direction changes is very low. The average motion direction of all effective optical flow vectors is calculated using the following formula:

[0043] The consistency of motion direction is quantified by calculating the mean of the cosine of the angle between each motion vector and the average direction. The calculation formula is as follows:

[0044] An image is considered a scene change when it meets the following conditions for a certain number of frames: 1. High motion consistency: directional consistency is higher than the threshold.

[0045] 2. High motion participation: The proportion of corners whose motion direction is basically consistent with the average motion direction exceeds the threshold.

[0046] 3. Low frequency of directional change: The average direction of motion remains stable within a continuous time window.

[0047] This algorithm can effectively distinguish between irregular jitter and regular scene changes, significantly improving accuracy and basically eliminating false alarms.

[0048] Optionally, in this embodiment, the occlusion anomaly detection submodule is used to determine occlusion based on the temporal features of edge density from Canny edge detection, triggered by scene changes or timed detection.

[0049] Specifically, the cause of the obstruction is usually that the camera lens is blocked by an object, such as a foreign object or water droplet that is close to or attached to it.

[0050] The technical solution in this embodiment uses the Canny edge detection algorithm to extract the edge map of the frame. The edge density Dedge is defined as the ratio of the number of edge pixels to the total number of pixels.

[0051]

[0052] Where W and H are the width and height of the image.

[0053] The triggering and judgment mechanism is as follows: 1. Triggering: There are two triggering methods: 1) Scene change detection as the trigger. Once a scene change is detected, the system starts monitoring edge density. 2) Slow detection once per minute; once the edge density falls below a threshold, formal content detection begins.

[0054] 2. Occlusion begins: If the edge density remains below a baseline threshold T for a continuous period of time, occlusion is determined to have occurred.

[0055] 3. Occlusion ends: If the edge density continues to recover to a high level, the occlusion is considered to have ended.

[0056] This dynamic detection method based on temporal feature analysis effectively solves the interference caused by changes in the light and shadow of occluders, and is more robust.

[0057] Optionally, in this embodiment, the stuttering anomaly detection submodule is used to extract timestamps using an OCR model, optimize recognition accuracy through bilateral filtering and Laplacian sharpening, and determine stuttering through timestamp difference analysis.

[0058] Specifically, the cause of stuttering is usually that the camera overheats due to continuous operation, leading to abnormal input.

[0059] Video stuttering, as a dynamic anomaly, is characterized by unstable, flickering, teleporting, and pixelated blurring of moving parts of the screen, followed by time jumps. The screen features are numerous, unstable, and random. Therefore, we chose to use timestamp information as a detection feature, employing an OCR model to extract timestamp information and analyzing changes in timestamps to determine if stuttering has occurred. Suitable OCR models include PaddleOCR, EasyOCR, and TesseractOCR. To improve efficiency, an interval sampling strategy is preferred, performing OCR recognition on only four keyframes per second (e.g., 25 frames).

[0060] To address the reduced recognition rate caused by downsampling, this embodiment conducted extensive testing of various image enhancement methods on the extracted timestamp image region, including binarization, bilateral filtering, median filtering, Gaussian filtering, histogram equalization, gamma correction, CLAHE, edge sharpening, and connected component analysis. The final test results showed that the combination of bilateral filtering and image sharpening maximizes the accuracy of OCR.

[0061] Bilateral filtering is a non-linear filtering method that effectively removes noise while maintaining sharp edges. The core formula of bilateral filtering involves a weighted average of pixels, combining spatial domain weights and value domain weights:

[0062] Among them: the first exponential term is the spatial domain weight, which uses the positional relationship of pixels to calculate the weight, making the filtering more in line with the spatial structure of the image; The second index term is the range weight, which focuses on the difference in grayscale values ​​of pixels in the image. It is calculated based on the difference in pixel values ​​between the center pixel and its neighboring pixels. It is used to measure the impact of different pixel values ​​and helps to better preserve edges during filtering (the pixel values ​​at the edges are large, and the range weight allows edge pixels to be affected by themselves and pixels with similar values, avoiding over-smoothing). W p It is the normalization factor, which is the sum of all weights: Image sharpening enhances image edges and makes character outlines clearer through kernel functions such as the Laplacian operator. In a specific embodiment, image sharpening technology based on the discrete Laplacian operator is preferably employed. Specifically, the following convolution kernels are used to perform convolution operations on the image:

[0063] This operation amplifies the pixel value differences in edge areas, making edge structures such as character outlines clearer and details more prominent.

[0064] Calculate the timestamp difference Δt between the identified two seconds. Under normal playback conditions, Δt should be 1 or 0 (considering cycle switching). When Δt exceeds a normal threshold, it is determined that a stutter has occurred.

[0065] Optionally, in this embodiment, the freeze anomaly detection submodule is used to extract timestamps using an OCR model. When the extracted timestamp "second" value remains unchanged for multiple consecutive periods, it is determined that the image is frozen.

[0066] Specifically, the cause of freezing anomalies is usually that the camera overheats due to continuous operation, leading to abnormal input. If frame difference detection is used, the similarity of each frame is checked, and if they are completely identical, an anomaly is considered to have occurred. However, the drawback of this detection method is that it is extremely performance-intensive.

[0067] In this embodiment, the timestamp information extracted during the aforementioned stuttering anomaly detection process is reused. If the timestamp "second" value extracted in multiple consecutive cycles remains unchanged, it is determined that the screen is frozen.

[0068] This method reuses OCR results, avoiding the huge performance overhead of frame-by-frame comparison in the traditional frame difference method, and greatly saves computing resources while ensuring high accuracy.

[0069] Optionally, in this embodiment, the brightness anomaly detection submodule is used to calculate the average brightness of the grayscale image. When the average brightness exceeds the overbrightness threshold, it is determined to be an overbrightness anomaly; when the average brightness is below the underbrightness threshold, it is determined to be an underbrightness anomaly.

[0070] Specifically, abnormal brightness is usually caused by abnormal scene lighting, such as underexposure or overexposure. Such abnormal brightness can seriously affect the monitoring personnel's judgment of the underground environment and may lead to the neglect of safety hazards in emergency situations. Therefore, timely detection of abnormal brightness is crucial to ensuring mine safety.

[0071] Analysis suggests that the core indicator for detecting brightness anomalies is the average brightness of the grayscale image. When the average brightness exceeds the overbrightness threshold, it is judged as an overbrightness anomaly; when the average brightness is below the underbrightness threshold, it is judged as an underbrightness anomaly.

[0072] Optionally, in this embodiment, the strong light anomaly detection submodule is used to set a dynamic highlight threshold based on the maximum grayscale value of the image; perform binarization and morphological operations on the image grayscale image; determine the largest connected highlight region and calculate the proportion of its area to the total image area; when the proportion exceeds the dynamic highlight threshold, it is determined to be a strong light anomaly.

[0073] Specifically, strong light can completely obscure the surrounding area, making it impossible for monitoring personnel to see the environmental conditions behind the light. This could lead to the missed detection of important safety hazards during safety monitoring. Therefore, strong light detection is an important part of mine safety monitoring.

[0074] In this embodiment, the dynamic highlight threshold is first set based on the maximum grayscale value of the image.

[0075] The image is binarized and morphologically processed. Specifically, the grayscale image is binarized, and noise is removed using erosion and dilation operations, while significant highlight areas are preserved to ensure the halo characteristics of the detection area.

[0076]

[0077] Connectivity analysis: Locate the largest connected specular region and calculate its area as a percentage of the total image area (AreaRatio).

[0078] Eccentricity filtering: To exclude interference from elongated, bright objects (rather than halos), the eccentricity e of the region is calculated. Further evaluation only continues if the eccentricity e is below a specific threshold (e.g., 0.93, indicating the region is closer to a circle).

[0079]

[0080] Where a and b are the major and minor semi-axes of the ellipse.

[0081] If AreaRatio exceeds the threshold, it is determined to be an abnormal strong light.

[0082] Optionally, in this embodiment, the black-and-white anomaly detection submodule is used to detect anomalies by judging the number of channels and analyzing the consistency of pixel values ​​in the three channels. A single-channel image is directly determined to be black and white, and a black-and-white anomaly is determined when the absolute difference of the R, G, and B channel values ​​of more than 95% of the pixels in the three-channel image is less than a preset fault tolerance threshold.

[0083] Specifically, the cause of black-and-white anomalies is insufficient ambient light, as the camera uses infrared black-and-white imaging. Black-and-white images lose crucial color information, affecting monitoring personnel's accurate assessment of equipment status and potentially masking early signs of equipment malfunction. Therefore, black-and-white image detection helps in the timely detection of equipment problems and the implementation of maintenance measures.

[0084] Analysis shows that for any pixel in a true-color image, the values ​​of its R, G, and B channels should satisfy the following:

[0085] If the image is a single-channel image, it is directly classified as black and white.

[0086] For a three-channel image, calculate the absolute difference between the pixel values ​​of each channel. If the three-channel values ​​of the vast majority of pixels are equal, then it is determined to be a black and white image.

[0087] Optionally, in this embodiment, the color cast anomaly detection submodule is used to calculate the ratio of the maximum to the minimum value of the mean values ​​of the three channels R, G, and B. When the ratio exceeds a preset threshold, it is determined to be a color cast anomaly.

[0088] Specifically, color cast anomalies are usually caused by abnormal scene lighting. Color cast can affect the monitoring personnel's accurate judgment of the equipment status and may mask early signs of equipment failure. Therefore, accurate color cast detection helps to detect equipment problems in a timely manner.

[0089] The detection principle of this embodiment is to determine whether there is color cast in the image by calculating the average ratio of the R, G, and B channels. The determination logic is as follows: 1. Calculate the average value of the three channels:

[0090] 2. Calculate the ratio between the maximum average and the minimum average.

[0091]

[0092] 3. If the ratio exceeds the preset threshold (set to 1.5 here), it is determined to be a color cast.

[0093] Optionally, in this embodiment, the stripe anomaly detection submodule is used to calculate the absolute difference between two adjacent rows of pixels in the image to obtain an absolute difference matrix; to calculate the proportion of zero values ​​in each row of the absolute difference matrix and the sum of each row; to filter similar rows based on the sum and the proportion of zero values; if the proportion of similar rows to the total number of rows is less than a set threshold, then it is determined that there is no stripe anomaly; to blacken the areas of dissimilar rows, to extract edges using the Canny operator, and to detect straight lines in the image using the Hough transform; to filter straight lines with an angle close to 90°; when the number of filtered straight lines exceeds a threshold, it is determined to be a stripe anomaly.

[0094] Specifically, the cause of stripe anomalies is usually overheating of the camera hardware or abnormal input. These stripes can interfere with the continuity of the monitoring image, affect the monitoring personnel's observation of dynamic events, and may affect the response speed in emergency situations. Therefore, vertical stripe detection is an important means to ensure the reliability of the monitoring system.

[0095] Image feature analysis of the stripe anomaly revealed that the pixel values ​​of adjacent rows in the stripe region are very similar, or even completely identical. Based on these findings, the stripe anomaly detection scheme in this embodiment combines row difference analysis and Hough transform for comprehensive judgment, and includes the following parts: 1. Row Difference: Calculates the absolute difference between two adjacent rows of pixels in an image.

[0096] 2. Zero-value ratio analysis: Statistically calculate the proportion of zero-value pixels in each row of the absolute difference matrix.

[0097] 3. Row sum statistics: Calculate the sum of each row to exclude special cases where there are many zero values ​​but some are very different, thus enhancing the requirements for distinguishing similar rows.

[0098] 4. Filter by criteria: Filter similar rows that meet the criteria: total row sum < 0.9 × width and zero value ratio > 0.6.

[0099] 5. Global ratio: If the ratio of rows that meet the criteria to the total number of rows is less than 0.14, return no exception.

[0100] 6. To eliminate interference from large areas of solid color (which would satisfy the requirement of similar rows), the areas of dissimilar rows are first blackened (to avoid noise in dissimilar row areas interfering with line detection). Then, the edges of the processed image are extracted using the Canny operator, and the lines in the image are detected using the Hough transform.

[0101] Canny edge detection includes Gaussian filtering, gradient calculation, nonmaximum suppression, and double-threshold hysteresis connections.

[0102] (1) Gaussian filtering (its function is to blur the image, remove noise, and avoid subsequent false detection of "noise edges") (2) Gradient calculation (calculate the gradient using the Sobel operator) to identify areas where the edges "change rapidly". Gradient magnitude and direction.

[0103] (3) Non-maximum suppression (built-in steps of the algorithm): For each pixel, compare the gradient strength of the two adjacent points in its gradient direction and retain only the local maximum value.

[0104] (4) Dual threshold detection (the thresholds here are set to 10 and 30). Strong edges are always retained, weak edges are only retained in the part connected to the strong edges, and non-edges are directly discarded.

[0105] Hough transform: Maps points in image space to a parameter space (such as polar coordinates) and uses a voting process to select the most likely line. The polar coordinates of the line are represented as: 7. Comprehensive judgment: Filter out vertical lines with an angle close to 90 degrees. If the number of them exceeds the threshold (set to 150 here), then vertical stripes are determined to exist.

[0106] Optionally, in this embodiment, the blur anomaly detection submodule is used to convert the image into a grayscale image, expand the image boundary using boundary copying, calculate the image contrast and perform normalization processing, and determine the image as blur anomaly when the normalized contrast is lower than a preset threshold.

[0107] Specifically, blurriness is generally caused by camera hardware overheating, slow input speed, or decreased bitrate. Blurritting disrupts image continuity, preventing monitoring personnel from seeing the complete scene and potentially causing them to miss important dynamic information in security monitoring. Therefore, blur detection is crucial for ensuring the integrity of monitoring systems. The detection principle of this scheme is that blurred images lose local details, resulting in significantly lower local contrast compared to normal images. The judgment logic is as follows: 1. Convert the image to grayscale.

[0108] 2. Use boundary copying to extend the image boundary to prevent out-of-bounds errors when calculating surrounding neighboring pixels later.

[0109] 3. Calculate the sum of squared differences between each pixel and its four adjacent pixels (up, down, left, and right).

[0110] 4. The calculated contrast ratio C needs to be normalized or standardized to obtain a normalized contrast ratio. Then, the normalized contrast ratio is judged. If the contrast ratio is lower than a preset threshold (this threshold should be derived from the statistical values ​​of normal clear images; here, 160 is used as the threshold), then a blur anomaly is determined to exist. If the image is higher than the set threshold, then the image is considered a normal clear image.

[0111] Optionally, in this embodiment, the random stripe noise anomaly detection submodule is used to smooth the image using a 3x3 median filter to obtain a smoothed image; the proportion of noise pixels is obtained by calculating the difference between the original image and the smoothed image, and when the proportion exceeds a threshold, it is determined to be a random stripe noise anomaly.

[0112] Specifically, random stripe noise anomaly is a type of updated stripe anomaly. This noise significantly reduces image clarity and affects the observation of details by monitoring personnel. In security monitoring, it may cause key information to be missed. Therefore, noise detection is of great significance for ensuring monitoring quality.

[0113] In this application's technical solution, the detection principle for random stripe anomalies is to identify noise by comparing the difference between the original image and the smoothed image. The specific judgment logic is as follows: 1. Smoothing: A 3x3 median filter is used to smooth the image, resulting in a smoothed image. Median filtering effectively removes salt-and-pepper noise while preserving edges relatively well.

[0114] 2. Difference Calculation and Binarization: Calculate the absolute difference between the original image and the smoothed image, and perform thresholding on the difference image.

[0115] 3. Noise ratio judgment: Calculate the proportion of noise pixels to total pixels. If it exceeds the threshold (set to 0.15 here), it is determined that there is random stripe noise.

[0116] Figure 2 This is a flowchart illustrating a method for detecting multiple types of image quality anomalies according to an exemplary embodiment of the present invention.

[0117] See Figure 2 The method includes: S1. Acquire the image or video stream to be detected and perform preprocessing; S2. Perform at least two types of image quality anomaly detection, including jitter anomaly detection, scene change detection, occlusion anomaly detection, stuttering anomaly detection, freeze anomaly detection, brightness anomaly detection, strong light anomaly detection, black and white anomaly detection, color cast anomaly detection, stripe anomaly detection, blur anomaly detection, and random stripe noise anomaly detection. S3. Summarize and output the results of image quality anomaly detection.

[0118] The specific execution methods of each step in the above embodiments have been described in detail in the embodiments of the system, and will not be elaborated further here.

[0119] We tested the image quality anomaly detection algorithm provided in this embodiment of the invention on a targeted dataset, and the results are shown in the table below: Table 1 Video Test Results

[0120] Table 2 Image Quality Test Results

[0121] Then, seven data sets—overly bright, overly dark, black and white, striped, blurry, strong light, and color cast—were merged, and all abnormal data were mixed into one dataset, resulting in 332 abnormal datasets. These datasets were then subjected to serial detection, identifying 318 abnormal cases (regardless of anomaly type, only the detection result was considered), achieving a recall rate of 95.78% and an accuracy rate of 100%. (For the normal dataset, other anomalies may exist for a certain anomaly type, so they are not calculated here.) Furthermore, the method according to the present invention can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the above-described method of the present invention.

[0122] Alternatively, the present invention can also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) storing executable code (or computer program, or computer instruction code) thereon, which, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform part or all of the steps of the method described above according to the present invention.

[0123] The present invention has been described in detail above with reference to the accompanying drawings. In the above embodiments, the descriptions of each embodiment have their own emphasis; for parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. Those skilled in the art should also understand that the actions and modules involved in the specification are not necessarily essential to the present invention. Furthermore, it is understood that the steps in the method of the embodiments of the present invention can be adjusted, combined, and deleted according to actual needs, and the modules in the device of the embodiments of the present invention can be combined, divided, and deleted according to actual needs.

[0124] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both.

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0126] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A multi-type image quality anomaly detection system, characterized in that, include: The data acquisition module is used to acquire the image or video stream to be detected and perform preprocessing. The multi-anomaly detection module includes at least two types of image quality anomaly detection sub-modules, wherein the image quality anomaly detection sub-modules are selected from the jitter anomaly detection sub-module, scene change detection sub-module, occlusion anomaly detection sub-module, stuttering anomaly detection sub-module, freeze anomaly detection sub-module, brightness anomaly detection sub-module, strong light anomaly detection sub-module, black and white anomaly detection sub-module, color cast anomaly detection sub-module, stripe anomaly detection sub-module, blur anomaly detection sub-module, and random stripe noise anomaly detection sub-module. The result output module is used to summarize and output the detection results of the multiple anomaly detection modules.

2. The system according to claim 1, characterized in that, The jitter anomaly detection submodule includes a corner point extraction unit, an optical flow tracking unit, a reciprocating motion analysis unit, and a jitter determination unit. The corner point extraction unit is used to extract corner points using the Shi-Tomasi algorithm. The optical flow tracking unit is used to determine the motion trajectory of the corner points using the LK optical flow method. The reciprocating motion analysis unit is used to determine the direction reversal by calculating the angle between the motion vectors of the corner points. The jitter determination unit is used to determine the occurrence of jitter anomalies by the ratio of the number of points with direction reversal to the total number of tracked corner points.

3. The system according to claim 1, characterized in that, The scene change detection submodule is used to determine scene changes based on three indicators: motion consistency of corner points in the image, motion participation, and frequency of direction change.

4. The system according to claim 1, characterized in that, The occlusion anomaly detection submodule is used to determine occlusion based on the temporal features of edge density from Canny edge detection, triggered by scene changes or timed detection.

5. The system according to claim 1, characterized in that, The stuttering anomaly detection submodule is used to extract timestamps using an OCR model, optimize recognition accuracy through bilateral filtering and Laplacian sharpening, and determine stuttering through timestamp difference analysis.

6. The system according to claim 1, characterized in that, The freeze anomaly detection submodule is used to extract timestamps using an OCR model. When the extracted timestamp "second" value remains unchanged for multiple consecutive periods, it is determined that the image is frozen.

7. The system according to claim 1, characterized in that, The brightness anomaly detection submodule is used to calculate the average brightness of the grayscale image. When the average brightness exceeds the overbrightness threshold, it is determined to be an overbrightness anomaly. When the average brightness is lower than the underbrightness threshold, it is determined to be an underbrightness anomaly.

8. The system according to claim 1, characterized in that, The strong light anomaly detection submodule is used to set a dynamic highlight threshold based on the maximum grayscale value of the image; perform binarization and morphological operations on the grayscale image; determine the largest connected highlight region and calculate the proportion of its area to the total image area; when the proportion exceeds the dynamic highlight threshold, it is determined to be a strong light anomaly.

9. The system according to claim 1, characterized in that, The black-and-white anomaly detection submodule is used to detect anomalies by judging the number of channels and analyzing the consistency of pixel values ​​in three channels. A single-channel image is directly determined to be black and white, while a black-and-white anomaly is determined when the absolute difference of the R, G, and B channel values ​​of more than 95% of the pixels in a three-channel image is less than a preset fault tolerance threshold.

10. The system according to claim 1, characterized in that, The color cast detection submodule is used to calculate the ratio of the maximum to the minimum value of the mean values ​​of the R, G, and B channels. When the ratio exceeds a preset threshold, it is determined to be a color cast anomaly.

11. The system according to claim 1, characterized in that, The stripe anomaly detection submodule is used to calculate the absolute difference between two adjacent rows of pixels in an image to obtain an absolute difference matrix; and to calculate the proportion of zero values ​​in each row of the absolute difference matrix and the sum of each row. Similar rows are selected based on the ratio of the sum to zero. If the proportion of similar rows to the total number of rows is less than a set threshold, it is determined that there is no stripe anomaly. For areas of dissimilar rows, blacking is performed, edges are extracted using the Canny operator, and straight lines in the image are detected using the Hough transform. Straight lines with an angle close to 90° are selected. When the number of selected straight lines exceeds a threshold, it is determined that there is a stripe anomaly.

12. The system according to claim 1, characterized in that, The blur anomaly detection submodule is used to convert the image into a grayscale image, expand the image boundary using boundary copying, calculate the image contrast and perform normalization processing, and determine the blur anomaly when the normalized contrast is lower than a preset threshold.

13. The system according to claim 1, characterized in that, The random stripe noise anomaly detection submodule is used to smooth the image using a 3x3 median filter to obtain a smoothed image; the proportion of noise pixels is obtained by calculating the difference between the original image and the smoothed image, and when the proportion exceeds a threshold, it is determined to be a random stripe noise anomaly.

14. A method for detecting multiple types of image quality anomalies, characterized in that, include: Acquire the image or video stream to be detected and perform preprocessing; Perform at least two types of image quality anomaly detection, including jitter anomaly detection, scene change detection, occlusion anomaly detection, stuttering anomaly detection, freeze anomaly detection, brightness anomaly detection, strong light anomaly detection, black and white anomaly detection, color cast anomaly detection, stripe anomaly detection, blur anomaly detection, and random stripe noise anomaly detection. Summarize and output the results of image quality anomaly detection.

Images