Camera imaging quality self-checking method and system, electronic equipment, medium and product

Abstract

The invention provides a camera imaging quality self-checking method and system, electronic equipment, a storage medium and a program product, and aims to solve the problems of leak detection of slow degradation type faults and high false alarm rate of environment change, the method comprises the following steps: acquiring a real-time video stream of a target camera; performing preprocessing to obtain a standardized image sequence; extracting multi-dimensional image quality features of the image sequence; establishing and updating a historical reference feature library of a target camera, wherein the historical reference feature library is constructed by continuously collecting historical image samples shot by the camera in a fixed scene and extracting image quality features of the historical image samples; dynamic difference comparison is carried out on the extracted multi-dimensional image quality features and corresponding historical features, and a degradation index representing the quality degradation degree is calculated; and performing anomaly detection and classification through an analysis model based on the degradation index and the time sequence of the multi-dimensional image quality features. According to the invention, the slow degradation fault can be detected, and false alarms caused by environmental changes are significantly reduced.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a self-inspection method and system for camera imaging quality, electronic equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] In video surveillance networks, the image quality of cameras directly affects the effectiveness of the monitoring system and the accuracy of subsequent intelligent analysis (such as facial recognition and vehicle capture). However, due to the wide variety of camera models and complex installation environments (such as height, orientation, and weather), traditional network management systems can only monitor hardware faults such as "power on" and "network connection," but are powerless to address "soft faults" at the image level. These soft faults include, but are not limited to: lenses being obstructed by foreign objects (leaves, posters), angle shifts after poles are impacted by external forces, aging infrared lights causing a washed-out image at night, abnormal encoding chips causing stripes or color blocks, and even video streams being replaced with still images. These image anomalies do not trigger offline alarms, but they render the recordings worthless for evidence and cause backend intelligent analysis such as facial recognition and vehicle capture to idle or generate false alarms, creating blind spots for daily patrols, event tracing, and command and dispatch.

[0003] Existing technologies for automatic video quality detection mainly fall into the following categories: **Rule-based threshold detection methods:** These methods set fixed threshold ranges for certain low-level features of the video image (such as brightness, contrast, and image entropy). An alarm is triggered when the calculated feature value exceeds the preset range. However, this method has an extremely high false alarm rate because it cannot distinguish between legitimate environmental changes (such as day-night cycles, cloudy days, and backlighting) and genuine equipment malfunctions. For example, low brightness in nighttime scenes may be misjudged as a malfunction; while slow brightness decay (such as infrared lamp aging) may be missed because it does not reach the threshold. **Supervised AI recognition methods:** These methods train a classification model (such as a convolutional neural network, CNN) to identify predefined anomaly categories (such as "occlusion" or "blurring"). While performing well on some known malfunctions, they heavily rely on a large amount of labeled anomaly data for training, meaning the model can only recognize anomaly types seen during training. For new and unknown anomalies that may appear in monitoring scenarios (such as novel screen flickering or unusual stripes), the model will directly fail, requiring re-collection of samples, labeling, and retraining, resulting in high maintenance costs and long cycles.

[0004] For example, patent document CN114626450A, entitled "Method, System, and Computer-Readable Storage Medium for Detecting Abnormal Conditions of a Camera," provides a detection scheme based on the positional relationship of preset markers. Although this method introduces reference objects for comparison, each judgment relies solely on a simple comparison between the current frame and a standard frame, lacking continuous tracking of the historical state of a single camera. This leads to the following inherent defects: Inability to detect slow degradation: It cannot quantify the gradual aging process of a camera "from good to bad," such as slow dust accumulation on the lens or slight focus shift. High false alarm rate: Its judgment is greatly affected by single-time changes in ambient light and instantaneous background changes; Poor scene adaptability: Relying on specific preset markers, it is difficult to deploy in general monitoring scenarios lacking such markers.

[0005] In summary, existing technologies generally suffer from core technical problems such as missed detection of slowly degrading faults, high false alarm rates due to environmental changes, and poor detection capabilities for unknown anomalies. Therefore, there is an urgent need in this field for a video quality self-inspection technology solution that can achieve personalized historical tracking, intelligently distinguish between environment and fault, and predictively maintain performance. Summary of the Invention

[0006] To address at least some of the problems in existing technologies, such as missed detection of slowly degrading faults and high false alarm rates due to environmental changes, this disclosure provides a self-testing method and system for camera imaging quality, an electronic device, a computer-readable storage medium, and a computer program product. By establishing a unique historical baseline feature library for each camera and utilizing dynamic differential and multi-dimensional analysis, it achieves "personalized" and "trend-oriented" self-testing of camera imaging quality. This can accurately detect slowly degrading faults that are easily missed by traditional methods and significantly reduce false alarms caused by environmental changes.

[0007] In a first aspect, this disclosure provides a self-testing method for camera imaging quality, the method comprising:

[0008] Acquire the real-time video stream from the target camera;

[0009] The real-time video stream is preprocessed to obtain a standardized image sequence;

[0010] Extract multi-dimensional image quality features from the standardized image sequence;

[0011] A historical benchmark feature library for the target camera is established and updated. The historical benchmark feature library is constructed by continuously collecting historical image samples taken by the camera in a fixed scene and extracting their image quality features.

[0012] The multi-dimensional image quality features extracted in real time are dynamically compared with the corresponding historical features in the historical benchmark feature library to calculate a degradation index that characterizes the degree of quality degradation.

[0013] Based on the degradation index and time series of multi-dimensional image quality features, anomaly detection and classification are performed through an analysis model.

[0014] Furthermore, the preprocessing of the real-time video stream includes:

[0015] Decode and unify the format of the input video stream;

[0016] Convert the decoded image to the specified color space;

[0017] Scale the image to standard resolution and align it to the timeline;

[0018] Keyframe and time-slice division: Based on the GOP (Group of Pictures) structure or scene segmentation logic, keyframes are extracted and time slices are formed to establish a time reference for subsequent analysis.

[0019] Furthermore, the extraction of multi-dimensional image quality features from the standardized image sequence includes:

[0020] Basic feature extraction: For the preprocessed structured frame sequence, extract basic indicators that can objectively reflect the image quality from a single frame or a short time slice, and output a basic feature sequence bound to time. The basic indicators include brightness and exposure, contrast and detail, sharpness and blur, noise level, motion intensity, and color shift.

[0021] Advanced Feature Analysis: Based on the basic feature extraction results and the original frame sequence, perform more in-depth semantic and temporal analysis on images and video clips. The analysis includes: content complexity, texture features, scene transitions, artifacts and compression traces, text and overlay content, color diversity, and motion consistency.

[0022] Feature fusion and optimization: Features from different sources are uniformly calculated into a comprehensive evaluation index.

[0023] Furthermore, the process of uniformly calculating features from different sources into a comprehensively evaluable index includes:

[0024] Standardize similar features to eliminate differences in dimensions and obtain a uniform score;

[0025] Perform cross-class feature weighted fusion and calculate the comprehensive quality score based on scene category or historical statistics;

[0026] Anomaly suppression is performed by truncating or smoothing outlier features to avoid interference from extreme points.

[0027] Furthermore, the step of dynamically comparing the real-time extracted multi-dimensional image quality features with the corresponding historical features in the historical benchmark feature library includes:

[0028] Spatial registration and timestamp alignment are performed between real-time images and historical benchmark samples;

[0029] Calculate the difference between real-time image features and historical baseline features in the corresponding dimensions;

[0030] The difference values ​​from multiple dimensions are weighted and fused to generate a comprehensive degradation index.

[0031] Furthermore, the anomaly detection and classification through the analysis model includes:

[0032] Based on preset rules, threshold judgments are made on the degradation index and / or multi-dimensional image quality features to quickly identify abnormal frames.

[0033] Based on statistical models, the time series of the multi-dimensional image quality features are subjected to stationarity or abrupt change point detection in order to discover statistically significant anomalies.

[0034] Based on machine learning models, the image sequences and their corresponding multi-dimensional features are classified or clustered to identify abnormal patterns.

[0035] Furthermore, the identified abnormal frames are merged into abnormal events, and the type, confidence level, and duration of each abnormal event are labeled.

[0036] Furthermore, the method also includes:

[0037] Record historical quality data, which includes historical quality score sequences and abnormal event sequences; wherein,

[0038] The historical quality score sequence consists of a comprehensive quality score calculated based on the multi-dimensional image quality features and / or the degradation index, arranged in chronological order.

[0039] The sequence of abnormal events consists of records of abnormal events arranged in chronological order and output after anomaly detection and classification.

[0040] Based on the historical quality score sequence and abnormal event sequence of the target camera, a time series prediction model is used to predict the quality change trend within a specified future time period.

[0041] Based on the prediction results, risk levels are assigned to current anomalies or potential risks.

[0042] Furthermore, the risk level is used to guide maintenance priorities, with high-risk levels triggering immediate maintenance work orders and medium- and low-risk levels being included in planned maintenance processes.

[0043] Secondly, this disclosure provides a camera imaging quality self-testing system, the system comprising:

[0044] The acquisition module is configured to acquire the real-time video stream from the target camera;

[0045] A preprocessing module is configured to preprocess the real-time video stream to obtain a standardized image sequence;

[0046] The extraction module is configured to extract multi-dimensional image quality features from the standardized image sequence;

[0047] The module is configured to establish and update the historical benchmark feature library of the target camera. The historical benchmark feature library is constructed by continuously collecting historical image samples taken by the camera in a fixed scene and extracting their image quality features.

[0048] The calculation module is configured to dynamically compare the multi-dimensional image quality features extracted in real time with the corresponding historical features in the historical benchmark feature library to calculate a degradation index that characterizes the degree of quality degradation.

[0049] The analysis module is configured to perform anomaly detection and classification based on the degradation index and time series of multi-dimensional image quality features through an analysis model.

[0050] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-described camera imaging quality self-test method.

[0051] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described self-checking method for camera imaging quality.

[0052] Fifthly, this disclosure provides a computer program product that includes computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described camera imaging quality self-test method.

[0053] Beneficial effects:

[0054] This disclosure provides a camera imaging quality self-inspection method and system, electronic device, computer-readable storage medium, and computer program product. By introducing a "historical benchmark feature library" and "dynamic differential comparison," a "health record" is established for each device throughout its lifecycle. This "self-comparison" mechanism enables the system to perceive slow degradation (such as lens dust accumulation and infrared lamp attenuation) like never before, quantifying minute, gradual quality degradation into a "degradation index," thus completely eliminating long-term undetected security blind spots due to slow changes. Simultaneously, the introduction of "multi-dimensional image quality features" allows analysis to go beyond a few indicators such as brightness and contrast, providing a comprehensive profile from the spatial, frequency, and temporal domains, offering a rich data foundation for accurately identifying complex anomalies (such as specific types of stripes and color blocks). By combining "historical benchmarks" and "time series analysis," the system can learn and understand the normal fluctuation patterns of the environment in which each camera operates. Distinguishing between "faults" and "scenes," the output "degradation index" and the results classified by the "analysis model" allow anomalies to be graded (e.g., minor, severe). Improve the availability of video networks and the accuracy of intelligent analysis.

[0055] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0056] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0057] Figure 1 This is a flowchart illustrating a camera imaging quality self-testing method provided in Embodiment 1 of this disclosure;

[0058] Figure 2 This is an architecture diagram of a camera imaging quality self-checking system provided in Embodiment 2 of this disclosure;

[0059] Figure 3 This is a data flow diagram of a camera imaging quality self-testing method provided in Embodiment 2 of this disclosure;

[0060] Figure 4 This is a schematic diagram of an unsupervised multimodal algorithm provided in Embodiment 2 of this disclosure;

[0061] Figure 5 This is a schematic diagram of an anomaly detection and analysis result provided in Embodiment 2 of this disclosure;

[0062] Figure 6 This is a schematic diagram of a trend prediction analysis provided in Embodiment 2 of this disclosure;

[0063] Figure 7 This is a block diagram of a camera imaging quality self-checking system provided in Embodiment 3 of this disclosure;

[0064] Figure 8 This is a block diagram of an electronic device provided in Embodiment 4 of this disclosure. Detailed Implementation

[0065] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0066] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0067] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0068] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0069] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein. Those skilled in the art will understand that the specific order of execution of the steps in the methods described above in the specific embodiments should be determined by their function and possible internal logic.

[0070] The camera imaging quality self-test method according to embodiments of this disclosure can be executed by electronic devices such as terminal devices or servers. Terminal devices can be in-vehicle devices, user equipment (UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0071] Example 1

[0072] Figure 1 This is a flowchart illustrating a camera imaging quality self-checking method provided in Embodiment 1 of this disclosure, referring to... Figure 1 The method includes:

[0073] Step S101: Acquire the real-time video stream from the target camera;

[0074] Step S102: Preprocess the real-time video stream to obtain a standardized image sequence;

[0075] Step S103: Extract multi-dimensional image quality features from the standardized image sequence;

[0076] Step S104: Establish and update the historical benchmark feature library of the target camera. The historical benchmark feature library is constructed by continuously collecting historical image samples taken by the camera in a fixed scene and extracting their image quality features.

[0077] Step S105: Perform dynamic difference comparison between the real-time extracted multi-dimensional image quality features and the corresponding historical features in the historical benchmark feature library to calculate a degradation index that characterizes the degree of quality degradation.

[0078] Step S106: Based on the time series of the degradation index and multi-dimensional image quality features, anomaly detection and classification are performed using an analysis model.

[0079] The purpose of this disclosure is to establish an "initial health state" feature library for each camera and continuously collect selfie samples from the same location and image card throughout its lifecycle. A dynamic differential algorithm quantifies progressive degradation into a trackable degradation index, enabling a "self-comparison" and plugging the detection blind spot for slow anomalies. Furthermore, by utilizing a no-reference quality assessment and temporal consistency fusion model, environmental changes (day / night, weather, season) are treated as normal fluctuations, and only pixel-level changes that continuously deviate from the baseline and conform to physical degradation / occlusion / screen distortion patterns are marked as anomalies, significantly reducing the false alarm rate.

[0080] The core of this disclosed technical solution lies in achieving automated, intelligent, and predictive management of camera imaging quality. Its application scenarios are wide-ranging, covering almost all fields requiring reliable and high-quality video surveillance systems. Examples include public safety and urban governance, where it can promptly detect monitoring failures caused by human obstruction (such as posting small advertisements or spraying paint), wind damage, or vehicles hitting poles; intelligent transportation and smart highways, such as real-time detection of image blurring and low contrast issues caused by rain, snow, fog, lens damage, or equipment aging during full-process highway monitoring; urban electronic police and checkpoints, where cameras capture violations and recognize license plates to ensure clear images and proper exposure, avoiding false or missed shots due to image quality issues; smart communities and building parks, automating the inspection of cameras at community perimeters, corridors, elevators, and underground parking garages; industrial production and energy facilities, detecting whether camera lenses are blurred or images distorted due to dust, steam, or high temperatures, ensuring effective monitoring of production processes and safety violations; and any field requiring "ensuring cameras can see clearly, accurately, and precisely." As a universal video quality diagnostic tool, it can upgrade the traditional manual, passive response-based operation and maintenance model to an efficient, proactive, and predictive intelligent management model.

[0081] Specifically, in step S101, acquiring the real-time video stream from the target camera involves multi-source access, such as pulling the stream from the network camera via protocols like RTSP (Real-Time Streaming Protocol), RTMP (Real-Time Messaging Protocol), and HTTP-FLV (HyperText Transfer Protocol - Flash Video), or accessing an NVR (Network Video Recorder) or video cloud platform via an SDK (Software Development Kit). Batch import of local video files is also supported. Decoding and buffering are performed, such as using FFmpeg or OpenCV's VideoCapture module for hardware or software decoding, and a frame buffer is established to handle network jitter and packet loss.

[0082] Then, preprocessing is performed to obtain standardized image sequences through standardization processes such as resolution normalization and time alignment. This unifies video streams or files from different sources into analyzable image sequences. Input conditions: Videos may come from different protocols (e.g., RTP, RTSP), different encoding formats (H.264 / H.265), and different resolutions and frame rates. Output results: Structured frame sequences (uniform format, uniform resolution, time-axis aligned), along with logs and parameters from each batch of processing, which serve as input for subsequent feature extraction.

[0083] Then, for the obtained standardized image sequence, multi-dimensional image quality features are extracted. These multi-dimensional image quality features include at least frequency domain features. For example, video frames are treated as two-dimensional signals, and the image is converted to the frequency domain using Fast Fourier Transform (FFT). The spectral energy distribution, frequency response characteristics, and frequency domain statistical features are analyzed to determine the video quality. Specifically, this includes calculating frequency domain indicators such as power spectral density, spectral flatness, and high-frequency energy ratio, and detecting problems such as blurring, noise, and compression artifacts through changes in frequency domain features.

[0084] To effectively address quality issues in different spatial domains, such as brightness anomalies and color shifts, the multi-dimensional image quality features can also include spatial and temporal features to achieve comprehensive coverage of anomaly analysis. Spatial features include, for example, calculating the mean and standard deviation of the Y component in the YUV space as brightness and contrast; using the Laplacian operator to convolve the grayscale image and calculating its variance as sharpness; calculating the mean of the S (saturation) channel in the HSV space, etc. Frequency domain features include dividing the spectrum into low, medium, and high-frequency annular regions by performing a Fast Fourier Transform (FFT) on the grayscale image and calculating the energy proportion of each region. Temporal features include, for example, using the Farneback dense optical flow method to calculate the average motion vector magnitude between consecutive frames as motion intensity; and calculating the mean of the pixel differences between adjacent frames after motion compensation as temporal noise, etc.

[0085] In step S104, establishing and updating the historical benchmark feature library includes benchmark establishment: after the camera is installed and debugged, during periods of good lighting conditions, video clips containing built-in physical maps or fixed scenes (such as a textured wall) are continuously captured for at least 24 hours to cover a complete daytime cycle. Features are extracted from these samples, and their statistical distributions (such as mean, variance, and quantiles) are calculated to form an initial "healthy" benchmark. Dynamic updates are also included: the system continuously incorporates new, "normal" sample features into the historical benchmark on a daily or weekly basis using a sliding window or exponentially weighted moving average method. This allows the benchmark to slowly adapt to long-term legitimate environmental changes (such as average brightness changes due to seasonal changes) while maintaining sensitivity to rapid degradation.

[0086] Step S105 implements dynamic difference comparison and degradation index calculation. Difference calculation: For each feature, calculate the relative deviation between its current value and the historical baseline mean, for example: (current value - historical mean) / historical standard deviation. Fusion and normalization: After normalizing the deviation values ​​of each feature, perform a weighted sum according to preset weights (e.g., higher weights for sharpness and noise), and finally map it to a degradation index between 0 and 100 using the Sigmoid function. The higher the index, the more severe the degradation relative to its own historical baseline.

[0087] Then, step S106 is performed: anomaly detection and classification based on the analysis model. The model is constructed using a lightweight temporal convolutional network (TCN) or a combination of LSTM and fully connected layers as the analysis model. Input and output: The input to the model is a degradation index sequence and a key image quality feature sequence within a time window. The output is the probability of an anomaly occurring within that time period, and the classification of the anomaly type (e.g., "occlusion", "blurring", "color cast", "screen distortion"). Workflow: The system runs the model in real time. When the anomaly probability output by the model exceeds a threshold, an alarm is triggered. Subsequently, a post-processing module merges consecutive anomaly frames into an anomaly event and records its start time, end time, type, and highest confidence level.

[0088] This disclosure achieves high-precision perception of "slow degradation," solving the industry problem of long-term missed detections. Traditional solutions are insensitive to detecting issues such as slow dust accumulation on lenses and infrared lamp attenuation. This disclosure, by establishing a long-term, personalized historical benchmark, quantifies this minute, gradual quality degradation into a continuously increasing "degradation index." Maintenance personnel can intuitively see the decline curve of the camera's image quality "health," similar to viewing a chart, allowing for intervention before complete image quality failure, transforming reactive maintenance into predictive maintenance. Traditional fixed threshold solutions generate numerous false alarms during day-night cycles and in rainy weather. This disclosure, through multi-dimensional feature and time-series analysis, enables the system to "understand" the normal fluctuation patterns of each camera's environment. The system only alerts to changes that continuously deviate from its historical patterns, filtering out legitimate fluctuations caused by environmental changes. This significantly reduces the number of alarms received by maintenance personnel, while significantly improving the "value" and accuracy of each alarm, greatly enhancing maintenance response efficiency. This solution no longer outputs simple "fault / normal" signals, but rather quantifiable degradation indices and clearly categorized anomalies. This allows for more refined operation and maintenance management. For example, all cameras can be sorted by "degradation index," prioritizing the most severely degraded devices; operation and maintenance resources can also be allocated based on anomaly type (e.g., "obstruction" requires immediate attention, while "slight color cast" can be addressed in a planned manner). This allows limited human and material resources to be invested where they are most needed, maximizing the overall availability of the video surveillance network.

[0089] Furthermore, the preprocessing of the real-time video stream includes:

[0090] Decode and unify the format of the input video stream;

[0091] Convert the decoded image to the specified color space;

[0092] Scale the image to standard resolution and align it to the timeline;

[0093] Keyframe and time slice division: Based on the GOP structure or scene segmentation logic, keyframes are extracted and time slices are formed to establish a time reference for subsequent analysis.

[0094] The specific process of preprocessing the real-time video stream to obtain a standardized image sequence includes:

[0095] Decoding and Format Unification: Using decoding libraries such as FFmpeg, video streams from different protocols (RTP / RTSP / RTMP) and encoding formats (H.264 / H.265) are decoded into original frame images (BGR pixel matrix).

[0096] Color space conversion: Convert the decoded image from native color spaces such as YUV and RGB to the standard working color space specified by the system (such as sRGB or linear RGB), and record the conversion parameters to ensure traceability of the results.

[0097] Resolution normalization: A scaling algorithm that preserves aspect ratio (such as bilinear or bicubic interpolation) is used to scale all frame images to a standard resolution (such as 720p) to provide consistent input for subsequent feature calculations.

[0098] Timeline alignment: Addresses frame rate instability and timestamp drift caused by network jitter, dropped frames, or encoder issues. It outputs a stable, continuous image sequence using frame interpolation or uniform sampling techniques.

[0099] Keyframe and Time Slice Division: Based on GOP Structure: Analyze the Group of Pictures structure of the video stream and prioritize extracting I-frames as keyframes. This is because I-frames contain complete image information, have the highest quality, are unaffected by the cumulative prediction errors of preceding and following frames, and best represent the camera's instantaneous imaging capability.

[0100] Scene-based segmentation: Simultaneously, using color histogram differences or structural similarity indices, abrupt changes in the image content (such as scene transitions) are detected. Continuous and stable segments of content are divided into time slices.

[0101] Establishing a temporal baseline: This step transforms the continuous video stream into semantically meaningful analytical units composed of keyframes and time-slice boundaries. This establishes a reliable temporal baseline for subsequent feature extraction and comparison, ensuring the efficiency and accuracy of the analysis.

[0102] Preprocessing the real-time video stream significantly improves analysis efficiency and ensures system real-time performance. Video data is massive, and performing full analysis on each frame is costly. By intelligently extracting keyframes (I-frames), the system can concentrate computational resources on the most information-rich frames, avoiding numerous redundant P / B frame calculations. This allows the system to perform real-time analysis of more video streams with limited computing power, greatly improving processing efficiency and resource utilization. Compression artifacts and prediction errors in P / B frames are not issues with camera imaging quality. Analyzing them could lead to misclassifying coding noise as camera malfunction. Focusing on I-frame analysis allows for a purer assessment of the true quality of the camera sensor, lens, and image signal processing, eliminating interference from the video encoding stage and resulting in more accurate and robust quality diagnostics. Simply extracting frames at time intervals fragments continuous scenes. By segmenting scenes into time slices, it ensures that the image content and ambient lighting remain relatively stable within a time slice. This makes temporary benchmarks established within the time slice (such as short-term fluctuations in brightness and noise) more valuable, enabling more sensitive detection of subtle quality degradation occurring within the same scene, thus improving detection sensitivity and semantic consistency. A smooth sequence composed of keyframes and time slices, with content abrupt changes removed, is ideal input for temporal models (such as LSTM). This structured data helps the model learn more clearly the continuous evolution pattern of quality degradation, rather than being interrupted by frequent scene changes, thereby further improving the accuracy of trend prediction.

[0103] Furthermore, the extraction of multi-dimensional image quality features from the standardized image sequence includes:

[0104] Basic feature extraction: For the preprocessed structured frame sequence, extract basic indicators that can objectively reflect the image quality from a single frame or a short time slice, and output a basic feature sequence bound to time. The basic indicators include brightness and exposure, contrast and detail, sharpness and blur, noise level, motion intensity, and color shift.

[0105] Advanced Feature Analysis: Based on the basic feature extraction results and the original frame sequence, perform more in-depth semantic and temporal analysis on images and video clips. The analysis includes: content complexity, texture features, scene transitions, artifacts and compression traces, text and overlay content, color diversity, and motion consistency.

[0106] Feature fusion and optimization: Features from different sources are uniformly calculated into a comprehensive evaluation index.

[0107] The extraction of multi-dimensional image quality features from the standardized image sequence employs a hierarchical and progressive architecture, comprehensively characterizing video quality from pixel-level metrics to semantic-level understanding. This includes:

[0108] 1. Basic Feature Extraction

[0109] Input: The preprocessed structured frame sequence (keyframes and time slices).

[0110] Processing logic: Quickly calculate low-level indicators that reflect the physical properties of an image from a single frame or a short time slice (such as 3-5 frames).

[0111] Key basic indicators and calculation methods:

[0112] Brightness and Exposure: Calculate the average value and histogram distribution of the Y (brightness) component in the YUV color space to assess whether the exposure is too dark or too bright.

[0113] Contrast and detail: Calculate the standard deviation of local regions of the image (e.g., divide the image into 8x8 blocks and calculate the average of the standard deviations within each block), or use the RMS contrast algorithm.

[0114] Clarity and blurriness: Apply the Laplacian operator to a grayscale image and calculate the variance of its response value (Laplacian variance method). The lower the variance, the more blurry the image.

[0115] Noise level: Apply a slight Gaussian blur to the image, subtract the blurred image from the original image, and calculate the standard deviation of the residual image as an estimate of spatial noise.

[0116] Motion intensity: The motion vectors of feature points between consecutive frames are calculated using sparse optical flow methods (such as LK optical flow), and the average magnitude of the vectors is statistically analyzed.

[0117] Color shift: In the Lab color space, calculate the mean values ​​of the a and b channels and the degree to which they deviate from the neutral point (0,0) to quantify color shift.

[0118] Output: A sequence of fundamental features (such as a luminance curve and a sharpness curve) that is strictly bound to each frame or each short time slice.

[0119] 2. Advanced Feature Analysis

[0120] Input: Basic feature extraction results + original frame sequence.

[0121] Processing logic: Performs deeper semantic and temporal analysis on images and video clips to provide contextual information for anomaly detection.

[0122] Main analysis content and methods:

[0123] Content complexity: The richness of image detail is statistically determined by calculating the multi-scale gradient histogram of the image or by using the DoG filter response.

[0124] Texture features: By calculating the gray-level co-occurrence matrix, statistical measures such as contrast and homogeneity energy are extracted to evaluate the directionality and regularity of the texture.

[0125] Scene transition: Calculate the Bach distance or structural similarity index of the color histogram between consecutive frames to detect abrupt or gradual scene boundaries.

[0126] Artifacts and Compression Trace: Analyze block boundary discontinuities in the frequency domain (DCT domain) to detect block effects; analyze overshoot and undershoot of ringing artifacts in edge regions.

[0127] Text and Overlay Content: Use lightweight CNN or traditional MSER (Maximally Stable Extremal Regions) algorithms to detect whether there are fixed OSD text or watermarks in the image.

[0128] Color diversity: The entropy value of the hue histogram is statistically analyzed in the HSV (Hue, Saturation, Value) color space. The higher the entropy value, the richer the colors.

[0129] Motion consistency: Analyze the directional consistency of motion vectors in a dense optical flow field to identify abnormal motion patterns caused by camera shake.

[0130] Output: High-level feature vectors, including scene labels, complexity scores, texture consistency metrics, etc., providing rich context for subsequent analysis.

[0131] 3. Feature Fusion and Optimization

[0132] Input: Basic feature sequence + advanced feature vector.

[0133] Processing logic: Unify features from different sources and with different dimensions into a comprehensive evaluation index.

[0134] Output: Quality of Energy (QoE) score sequence and feature contribution description, used for detection and prediction.

[0135] Feature extraction is explicitly divided into three levels, constructing a comprehensive quality perception capability from "pixel" to "semantics." Basic features ensure accurate capture of hard quality issues such as exposure, blur, and noise; advanced features endow the system with the ability to understand image content (such as texture and scene) and identify complex faults (such as compression artifacts and abnormal jitter). This combination enables the system to diagnose common problems such as "unclear image" and accurately determine "why it is unclear," laying the foundation for accurate anomaly classification. Through feature fusion, the objectivity and interpretability of the evaluation results are achieved, as a single image feature (such as low brightness) is prone to misjudgment. By weightedly fusing multiple mutually corroborating or mutually constraining features, the resulting comprehensive quality score is more robust. At the same time, the "feature contribution" function can explain the source of the score (e.g., a low score is mainly due to a significant decrease in sharpness), which not only enhances the credibility of the results but also provides maintenance personnel with clear troubleshooting directions, improving handling efficiency. Through hierarchical processing, the system can prioritize the use of basic features with low computational cost for rapid screening, triggering more complex advanced feature analysis only when anomalies occur in basic features. By combining caching and parallel optimization, this "lightweight-to-heavyweight separation" architecture ensures that the system can achieve real-time, online quality analysis of massive video streams with minimal computational resource consumption, greatly enhancing the feasibility of the solution. It also enhances the system's adaptability to complex scenes. The "anomaly suppression" mechanism in the feature fusion and optimization stages effectively filters out transient, harmless interference (such as headlights sweeping across or small animals intruding), allowing the system to focus on persistent, harmful quality degradation. This enables the system to maintain stable performance under different lighting conditions, weather, and scene content, reducing false alarm rates and demonstrating strong environmental adaptability.

[0136] Furthermore, the process of uniformly calculating features from different sources into a comprehensively evaluable index includes:

[0137] Standardize similar features to eliminate differences in dimensions and obtain a uniform score;

[0138] Perform cross-class feature weighted fusion and calculate the comprehensive quality score based on scene category or historical statistics;

[0139] Anomaly suppression is performed by truncating or smoothing outlier features to avoid interference from extreme points.

[0140] Standardization of similar features involves using Z-score or Min-Max normalization to eliminate the dimensional differences between features, ensuring they fall within the same numerical range. Cross-class feature weighting and fusion calculates a comprehensive quality score by summing the normalized features based on expert knowledge or weights learned through machine learning. For example, sharpness and noise are typically weighted higher than saturation. Anomaly suppression can apply moving median filtering or DBSCAN clustering to the feature time series to smooth out outliers caused by transient interference (such as insects flying across the screen), avoiding false alarms.

[0141] The feature fusion and optimization process also includes caching and reuse: for computationally expensive features (such as frequency domain features), a caching mechanism is established to avoid repeated computation within the same time slice. It also involves parallelism and optimization: multi-threaded / parallel computing technologies such as OpenMP or CUDA (Compute Unified Device Architecture) are used to accelerate multiple feature extraction algorithms simultaneously to meet real-time requirements. This results in a concise, time-bound comprehensive quality score sequence and feature contribution description, serving as the most direct and reliable input to the anomaly detection module.

[0142] Furthermore, the step of dynamically comparing the real-time extracted multi-dimensional image quality features with the corresponding historical features in the historical benchmark feature library includes:

[0143] Spatial registration and timestamp alignment are performed between real-time images and historical benchmark samples;

[0144] Calculate the difference between real-time image features and historical baseline features in the corresponding dimensions;

[0145] The difference values ​​from multiple dimensions are weighted and fused to generate a comprehensive degradation index.

[0146] By using dynamic differential comparison, a degradation index that characterizes the degree of quality degradation can be calculated. This step is the core of achieving "comparing oneself to oneself," quantifying the abstract concept of "image quality deterioration" into concrete numbers through precise comparison.

[0147] The implementation process includes:

[0148] Spatial registration and timestamp alignment include:

[0149] Spatial registration: To ensure fairness in the comparison, the system performs feature point matching (such as SIFT (Scale-Invariant Feature Transform) or ORB (Oriented Fast and Rotated BRIEF)) or phase-correlation-based template matching between the real-time image and the historical image used as a reference, achieving sub-pixel-level alignment. This eliminates comparison errors caused by minor translations or rotations due to wind or slight vibrations. For scenarios using physical image cards, this step accurately locates and crops the image card area for comparison, maximizing the signal-to-noise ratio.

[0150] Timestamp Alignment: Taking into account day-night cycles and seasonal cycles, the system will not directly compare midday footage with midnight benchmarks. Instead, it will select historical feature data from the historical benchmark library within the same or similar time period (e.g., within ±1 hour) based on the timestamp of the current frame (e.g., GPS time or system time) as the comparison benchmark, ensuring that the comparison is conducted under similar lighting and environmental conditions.

[0151] Calculate the multidimensional feature difference:

[0152] After completing the spatiotemporal alignment, for each image quality feature, the difference between its real-time value and the corresponding historical baseline value is calculated.

[0153] The calculation formula uses relative deviations rather than absolute differences to eliminate the influence of the magnitude of different features. For example, for feature i, its difference value D i It can be calculated as:

[0154] D i = (V current -V baseline _ mean ) / V baseline_std

[0155] Among them, V baseline_mean and V baseline_std These are the mean and standard deviation of the feature in the historical benchmark database, respectively. This value intuitively reflects how many standard deviations the current feature has deviated from the historical normal level.

[0156] Weighted fusion generates a degradation index:

[0157] The system maintains a weight vector that defines the importance of each quality feature in contributing to the overall degradation. For example, sharpness and noise typically have much higher weights than saturation.

[0158] The degradation index DI is calculated by weighted summation: DI = Σ(W i ×D i ), where Wi D is the weight of feature i. i It is its difference value.

[0159] Finally, the weighted sum is mapped to a fixed range (e.g., 0-100) using an activation function (such as the sigmoid) to generate the final, easily understood degradation index. The higher the index, the more severe the overall degradation.

[0160] This disclosed embodiment achieves pixel-level precise comparison through dynamic differential comparison, greatly improving detection sensitivity and accuracy. Spatial registration eliminates comparison errors caused by minor changes in the physical position of the camera. Without spatial registration, slight camera offsets can be misjudged as large-scale changes in image content, leading to catastrophic false alarms. Through precise alignment, the system can focus on true quality changes, making it possible to detect slight defocusing and localized stains, achieving pixel-level sensitivity. The timestamp alignment mechanism gives the system "time common sense." It knows that the image should be bright at noon and dim at night, thus avoiding false alarms due to low brightness at night. This comparison logic, synchronized with environmental cycles, simulates the judgment thinking of human maintenance personnel, fundamentally distinguishing between "abnormal" and "legitimate periodic changes," a key innovation in reducing the false alarm rate to a practical level. The "degradation index," generated through relative deviation calculation and weighted fusion, is a dimensionless, standardized scalar. It allows the quality status of different cameras and different models of equipment to be measured and ranked on the same scale. Maintenance personnel can clearly see that "camera A's health is 85% and camera B's health is 45%", thus accurately locating the device with the most serious problem. At the same time, the time series data of this index provides the most direct data foundation for predictive maintenance.

[0161] While generating the comprehensive index, the system can record the differential features that contribute the most. This means that the system can not only report "image quality has deteriorated" (high degradation index), but also clearly point out that "it is mainly caused by a significant decrease in sharpness" or "it is accompanied by color shift." This interpretability greatly facilitates the subsequent work of maintenance personnel, who can quickly locate the root cause of the fault based on these clues (is the lens dirty? Or is there a problem with the color sensor?), achieving a leap from "alarm" to "diagnosis."

[0162] Furthermore, the anomaly detection and classification through the analysis model includes:

[0163] Based on preset rules, threshold judgments are made on the degradation index and / or multi-dimensional image quality features to quickly identify abnormal frames.

[0164] Based on statistical models, the time series of the multi-dimensional image quality features are subjected to stationarity or abrupt change point detection in order to discover statistically significant anomalies.

[0165] Based on machine learning models, the image sequences and their corresponding multi-dimensional features are classified or clustered to identify abnormal patterns.

[0166] Furthermore, the identified abnormal frames are merged into abnormal events, and the type, confidence level, and duration of each abnormal event are labeled.

[0167] Anomaly detection identifies quality anomalies in a video. The input is a fused sequence of quality metrics. The implementation process includes:

[0168] Fast threshold determination based on preset rules:

[0169] Implementation: The system maintains a rule base containing a series of "IF-THEN" logic. For example:

[0170] If the IF degradation index is greater than 70, then a “severe degradation” alert will be triggered.

[0171] If the average brightness is less than 30 and the scene label is not equal to "Night", then trigger the "Too Dark" alert.

[0172] If the frame rate is less than 10 fps, then trigger a "stuttering" alert.

[0173] Function: This module is lightweight and has an extremely fast response time. As the first filter, it is used to quickly capture well-defined and distinctive anomalies and issue immediate alerts.

[0174] Time series analysis based on statistical models:

[0175] Implementation: Apply statistical process control methods to time series data of key features (such as sharpness and noise). For example:

[0176] CUSUM control charts are used to detect small but persistent drifts in feature values. A Bayesian change point detection algorithm is employed to identify points of abrupt change in the time series, typically corresponding to the onset of a fault. This module does not rely on fixed thresholds but instead starts from the statistical regularities of the data itself, enabling it to discover slow trend changes and hidden abrupt changes, effectively complementing rule-based detection.

[0177] Precise classification based on machine learning models:

[0178] Supervised classification: Using a pre-trained deep learning model (such as a lightweight CNN+LSTM hybrid model), a short video clip and its multi-dimensional feature sequence are input, and the anomaly classification results (such as "occlusion", "blur", "striped screen") and confidence scores are directly output.

[0179] Unsupervised clustering: In scenarios where there is insufficient labeled data, high-dimensional feature vectors are clustered (e.g., using K-Means or autoencoders). Normal samples will cluster in dense regions, while outliers will become outliers and thus be identified. This is particularly effective for discovering unknown types of anomalies.

[0180] This module is the system's "intelligent brain," capable of handling complex, non-linear patterns and achieving high-precision anomaly classification and the discovery of new anomalies.

[0181] Event generation and annotation:

[0182] Implementation: A post-processing module scans all frames marked as abnormal.

[0183] Merging: Frames that are consecutive in time (or have very short intervals) and are determined to be of the same type of anomaly are merged into a single "abnormal event". Labeling: The event is assigned the following metadata: Type: The most prevalent anomaly type among all frames within the event; Confidence: The highest confidence level among all frames within the event; Duration: The start and end timestamps of the event are recorded. This transforms scattered "abnormal frames" into "abnormal events" with clear semantics, greatly improving the readability and operability of alarm information and avoiding "alarm storms".

[0184] The anomaly detection and classification embodiments of this disclosure achieve an optimal balance between detection efficiency and accuracy. The rule engine ensures the system's instantaneous response capability to clearly defined faults, meeting real-time requirements. Statistical and machine learning models, based on this, perform in-depth analysis and precise authentication of complex, ambiguous, or progressive faults. This "fast-slow combination" architecture ensures that the system neither delays critical alarms due to comprehensive and complex analysis nor sacrifices accuracy due to simple judgments, achieving intelligent allocation of computing resources. Furthermore, it constructs a complete capability from "detection" to "diagnosis," whereas traditional solutions may only answer "is there a problem?". This disclosure, through the classification capability of the machine learning model, can further answer "what kind of problem is it" (e.g., occlusion, blurriness, screen distortion). This fine-grained fault diagnosis capability provides direct handling guidance for maintenance personnel; for example, "occlusion" requires on-site cleaning, and "screen distortion" may require equipment replacement, thus significantly improving maintenance efficiency. Simultaneously, it significantly enhances the detection and adaptation capabilities to unknown faults. The introduction of unsupervised learning mechanisms means the system no longer relies entirely on preset fault categories. It can proactively discover novel anomalies (such as specific sensor noise) that have never appeared in the training set by identifying outliers or new patterns. This gives the system the ability to continuously evolve, reduces the risk of failure due to changes in the environment, and makes it highly adaptable. The event generation step transforms the underlying technical details (such as thousands of anomaly frames) into structured "anomaly event" records for operation and maintenance management. An alarm containing "Type: Occlusion, Confidence: 96%, Time Period: 14:05-14:35" has far more information and action guidance value than a series of raw messages like "Frame 1001 anomaly, Frame 1002 anomaly..." This completely changes the experience for operation and maintenance personnel, turning alarms from "noise" into "instructions".

[0185] Furthermore, the analysis model is an unsupervised learning model; the anomaly detection and classification using the analysis model specifically includes:

[0186] By fusing video features, network transmission features, and device signaling features, multi-dimensional image quality features are formed.

[0187] The multi-dimensional image quality features are input into a pre-trained unsupervised learning model, and the reconstruction error is calculated.

[0188] When the reconstruction error exceeds the dynamic threshold, it is determined to be abnormal;

[0189] The multi-dimensional image quality features are constructed by fusing multimodal data, specifically including:

[0190] Periodically extract frames from the video stream and extract its high-dimensional visual feature vector (Vf).

[0191] Statistical features are extracted from network transmission protocols to form a network feature vector (Vn).

[0192] Extract status features from the device signaling logs to form a signaling feature vector (Vs);

[0193] The visual feature vector (Vf), network feature vector (Vn), and signaling feature vector (Vs) are aligned and concatenated according to timestamps to form a unified multi-dimensional image quality feature vector.

[0194] The unsupervised learning model is an autoencoder, which is trained through the following steps:

[0195] The multimodal feature vectors generated by the acquisition system under normal operating conditions are used as the training set;

[0196] The autoencoder is trained with the goal of minimizing the reconstruction loss, so that it can accurately reconstruct normal input data.

[0197] The dynamic threshold (T) is determined based on the statistical distribution of the reconstruction error of all normal samples during the training phase. The calculation formula is: T = μ + k·σ, where μ is the mean of the reconstruction error, σ is the standard deviation, and k is an adjustable parameter to control the system's sensitivity to anomalies.

[0198] After determining that an anomaly has occurred, the method further includes:

[0199] The contribution of each dimension of the multi-dimensional image quality feature vector is decomposed to determine whether the main root cause of the anomaly is video content, network transmission or device signaling status.

[0200] The output includes a diagnostic report containing anomaly labels, severity levels, root cause information, and a comprehensive quality score.

[0201] Furthermore, the method also includes:

[0202] Record historical quality data, which includes historical quality score sequences and abnormal event sequences; wherein,

[0203] The historical quality score sequence consists of a comprehensive quality score calculated based on the multi-dimensional image quality features and / or the degradation index, arranged in chronological order.

[0204] The sequence of abnormal events consists of records of abnormal events arranged in chronological order and output after anomaly detection and classification.

[0205] Based on the historical quality score sequence and abnormal event sequence of the target camera, a time series prediction model is used to predict the quality change trend within a specified future time period.

[0206] Based on the prediction results, risk levels are assigned to current anomalies or potential risks.

[0207] This step represents a high level of system intelligence, achieving a leap from "post-event diagnosis" to "pre-event prediction" through the mining and analysis of historical data. Specifically, it includes:

[0208] Historical quality data is recorded, and the data consists of: Historical Quality Score Sequence: The overall quality score (QoE) generated by the feature fusion optimization step is stored in chronological order. Each data point includes a timestamp and a corresponding quality score of 0-100; Anomaly Event Sequence: The structured anomaly event records output by the anomaly detection and classification steps are stored in chronological order. Each record includes metadata such as event type, start time, end time, confidence level, and maximum degradation index.

[0209] Storage and Management: Time-series databases are used for efficient storage, and a data lifecycle management strategy is established to archive expired data regularly.

[0210] Then, based on the recorded historical quality data, quality trend prediction is performed using a time series prediction model. The model selection can use a combination of traditional statistical models (such as ARIMA and Prophet) and machine learning models (such as LSTM and TCN) to construct a hybrid prediction model.

[0211] Input features:

[0212] Main input: A smoothed version of the historical quality score series.

[0213] Auxiliary features: abnormal event sequences (converted into statistical features such as abnormal frequency and abnormal duration), environmental factors (such as season and weather type, if available).

[0214] Predictive output: Predicts the expected quality score and its confidence interval, the expected changes in key quality indicators (such as sharpness and noise), and the probability of anomalies for a specified future time period (such as the next 7 days).

[0215] Then, the output results are categorized by risk level and used for decision support. The categorization logic is as follows: Based on the prediction results and current status, a multi-dimensional risk assessment matrix is ​​established: High Risk (Immediate Action): There is a serious anomaly (e.g., complete obstruction) OR the predicted display quality will drastically degrade to an unusable level within 24 hours. Medium Risk (Planned Maintenance): There is a minor anomaly, but the functionality is basically normal OR the predicted display quality will continue to slowly degrade to the warning line within one week. Low Risk (Observation and Monitoring): There is no current anomaly, but the predicted display shows a slight degradation trend OR there are occasional, self-recovering minor anomalies. Maintenance priority suggestions are generated for each camera, and the system is linked with the maintenance work order system to automatically create maintenance tasks of different urgency levels.

[0216] By enhancing recording, prediction, and risk grading capabilities, this disclosed embodiment achieves a fundamental shift in the operation and maintenance (O&M) model from "passive response" to "proactive prediction." Traditional O&M involves emergency repairs after a fault occurs. This system, through time-series prediction, can identify degradation trends 1-7 days before a camera completely fails, giving the O&M team the opportunity to schedule maintenance before user complaints, eliminating faults in their nascent stage and significantly improving service availability and user satisfaction. Furthermore, it provides data-driven scientific decision-making support, enabling optimal resource allocation. Based on the risk level classification of prediction results, it provides clear priority guidance for limited O&M human and material resources. O&M managers can accurately know which cameras should be addressed immediately, which can be scheduled for incidental handling during next week's inspection, and which only require continued observation. This precise decision support avoids the chaos of "firefighting" O&M and maximizes the return on O&M resource input. Moreover, it helps build a closed loop for the full lifecycle health management of equipment, with complete historical quality data records forming an "electronic health record" for each camera. By analyzing long-term data, we can not only assess the current state of equipment but also its reliability: identify common problems in specific models or batches of equipment; optimize equipment replacement cycles: scientifically formulate equipment upgrade strategies based on performance degradation curves; and verify maintenance effectiveness: quantitatively evaluate the effectiveness of maintenance work by comparing quality data before and after maintenance. This achieves an upgrade from single-fault handling to full lifecycle health management of assets. It also enhances the system's decision-making ability in complex fault scenarios, as a single current state detection may not reflect the urgency of the problem. For example, a "slightly blurry" camera, if its clarity is predicted to deteriorate rapidly, should be classified as medium-high risk; while another "slightly blurry" but stable camera can be classified as low risk. This comprehensive risk assessment, combining current state and future trends, makes decision-making more scientific and forward-looking, effectively preventing small faults from escalating into major problems.

[0217] Furthermore, the risk level is used to guide maintenance priorities, with high-risk levels triggering immediate maintenance work orders and medium- and low-risk levels being included in planned maintenance processes.

[0218] By leveraging risk-driven precision operations and maintenance, all technical analysis results are transformed into clear, actionable, and measurable business instructions.

[0219] High-risk levels trigger immediate repair work orders. The criteria for this are: the existence of an anomaly causing complete or near-complete failure of monitoring functions (e.g., complete obstruction, severe screen distortion, signal loss), or a predictive model indicating that quality will drastically degrade to unusable levels within the next 24-48 hours. The system automatically generates and dispatches an "Immediate Repair Work Order." This work order is immediately delivered to frontline maintenance personnel via high-priority notifications such as SMS and app push notifications, and is mandated to respond and handle within a strict SLA (Service Level Agreement) timeframe (e.g., 4-8 hours). The work order is tracked throughout the process and can only be closed after the problem is resolved and quality has been automatically verified by the system.

[0220] Medium and low risk levels are incorporated into the planned maintenance process. The criteria for determination are as follows: Medium risk: Minor anomalies exist, but monitoring functions are basically usable (e.g., slight blurriness, occasional stripes), or the predictive model indicates that quality will continue to slowly decline to a level requiring intervention over the next 1-2 weeks. Low risk: No current anomalies, but the predictive model identifies an extremely slow degradation trend, or there are occasional, automatically recoverable transient anomalies. Execution logic: The IDs and specific problem descriptions of these cameras are integrated into a "planned maintenance work order." The operations manager can batch package these work orders based on geographical region and problem type, and schedule them for unified processing in the next week's or next month's periodic inspection plan. This achieves a streamlined operations and maintenance model of "one deployment, solving a whole area of ​​problems."

[0221] By implementing risk-level-guided operations and maintenance (O&M), optimal allocation of O&M resources can be achieved, ensuring the fastest response to the most urgent faults, while non-urgent tasks are rationally scheduled to avoid resource congestion. This transforms a large number of maintenance activities from "high-cost emergency deployments" to "efficient planned inspections," significantly reducing random expenses such as travel and overtime, and making spare parts preparation more predictable. By promptly eliminating high-risk faults, the annual availability of key monitoring points is greatly improved, providing a solid guarantee for the continuity of core businesses such as security and transportation.

[0222] This disclosure achieves precise quantification and intelligent diagnosis of imaging quality by establishing a dedicated historical benchmark feature library for each camera and integrating multi-dimensional image quality features with dynamic differential comparison technology. Its core beneficial effect lies in its ability to keenly detect slow degradation faults that are easily missed by traditional methods, significantly distinguish between environmental changes and real anomalies to greatly reduce the false alarm rate, and use time-series prediction models to anticipate quality degradation trends in advance. Ultimately, it upgrades the operation and maintenance mode from passive response to proactive predictive maintenance with precise resource allocation, comprehensively improving the reliability and operation and maintenance efficiency of video surveillance networks.

[0223] Example 2

[0224] This disclosure provides a second embodiment of a camera imaging quality self-testing method, the purpose of which is:

[0225] 1. Establish an "initial health state" feature library for each camera and continuously collect selfie samples from the same location and image card throughout the entire life cycle. Use a dynamic differential algorithm to quantify progressive degradation into a trackable degradation index, achieving "comparison with itself" and plugging the detection blind spot of slow anomalies.

[0226] 2. Utilizing a no-reference quality assessment and time consistency fusion model, environmental changes (day / night, weather, seasons) are treated as normal fluctuations. Only pixel-level changes that continuously deviate from the baseline and conform to physical degradation / occlusion / screen distortion patterns are marked as anomalies, significantly reducing the false alarm rate. Simultaneously, adaptive incremental learning is introduced to automatically expand the feature space for novel anomalies, eliminating the need for manual re-labeling. After an anomaly is detected, a "degradation curve for the next 3-7 days" and "risk level" are provided based on time series predictions, directly guiding maintenance scheduling: minor anomalies enter a "delayed repair pool," while high-level anomalies trigger "immediate work orders," thus allocating limited manpower to the most urgent camera repair tasks, improving the availability of the video network and the accuracy of intelligent analysis.

[0227] The method is implemented based on a camera imaging quality self-inspection system. This system supports video quality assessment with and without reference to weak references, combining texture / frequency / temporal domain features with AI anomaly detection to provide interpretable results and traceable data links. It is compatible with both online real-time monitoring and offline batch processing, supporting horizontal scalability and high availability. This enables automated quality analysis, anomaly detection, and operation and maintenance management of multi-source video streams. Its core workflow is: accessing video data from the bottom layer, performing preprocessing, content and quality analysis layer by layer, and finally using an AI model to determine anomalies and generate operation and maintenance services. The entire architecture relies on a robust data and infrastructure layer, ensuring the system's reliability, efficiency, and scalability.

[0228] System architecture such as Figure 2 As shown, the hierarchical description includes:

[0229] 1. Access and Acquisition Layer: Function: As the system's data entry point, it is responsible for acquiring video data from multiple sources.

[0230] Key modules include:

[0231] Camera video stream access: Real-time access to video streams from network cameras (such as RTSP, RTMP).

[0232] Camera recording access: Processes video files stored locally on the device.

[0233] Historical data access: Import existing historical video data for analysis or training.

[0234] Frame sampling and timestamp synchronization: Intelligent frame extraction is performed on the video stream, and a unified and accurate timestamp is generated for all frames to establish a time reference for subsequent analysis.

[0235] 2. Content Analysis Layer: Function: Performs basic analysis and understanding of video content, providing context for quality analysis.

[0236] Key modules include:

[0237] Scene Analyzer: Detects changes in video scenes (such as switching from indoors to outdoors).

[0238] Timeline analysis layer: Analyzes the event sequence of the video over time.

[0239] Metadata processor: Extracts and manages the metadata information inherent in the video (such as encoding format, resolution, and device information).

[0240] 3. Video Processing and Quality Analysis Layer: This is the core computing layer of the system, responsible for evaluating the objective quality of the video.

[0241] Key modules include:

[0242] Video preprocessor: performs operations such as decoding, color space conversion, and resolution normalization.

[0243] Quality Indicator Calculator: Calculates multiple quality indicators such as brightness, contrast, sharpness, noise, and blockiness.

[0244] Quality scoring system: integrates various indicators and outputs a comprehensive quality score (QoE).

[0245] Performance Manager: Optimizes the processing flow to ensure the real-time performance or efficiency of analysis tasks.

[0246] 4. Data and Infrastructure Layer: Functions: Provides data persistence, system support, and operation and maintenance assurance for upper-layer applications.

[0247] Key modules include:

[0248] Data warehouse: Stores raw videos, feature data, analysis results, and system logs.

[0249] Cache manager: Improves data access speed and reduces database pressure.

[0250] Error tracker & log manager: Records system running status and faults, making it easier to troubleshoot problems.

[0251] Report Generator: Automates the generation of visual diagnostic reports from analysis results.

[0252] 5. Service and Editing Layer: This layer is an application layer for users or operations and maintenance personnel, providing task management and system configuration functions.

[0253] Key modules include:

[0254] Batch Manager: Manages offline analysis tasks for large amounts of historical video.

[0255] Task Editor: Allows users to create, configure, and schedule analytics tasks.

[0256] 6. Anomaly Detector (Core AI Decision Layer): Function: Utilizing AI models and all information provided by the lower layers, it ultimately determines whether the video contains anomalies and their types.

[0257] Key modules include:

[0258] AI Model Anomaly Detector: Utilizes a trained machine learning / deep learning model to identify specific faults such as image occlusion, blurriness, and color cast.

[0259] Degradation detector: By comparing historical quality data, it identifies slow trends in video quality decline and enables predictive maintenance.

[0260] Monitoring Manager: May be responsible for coordinating the scheduling and status monitoring of all anomaly detection tasks.

[0261] The quality indicators are shown in Table 1 below.

[0262] Table 1: Explanation of Quality Indicators

[0263]

[0264] The core processing flow of the method is as follows: Figure 3 As shown, the entire process begins with the client sending an analysis request to the cloud and finally receives a response containing quality diagnostic results.

[0265] Starting point: Request submission

[0266] A user or system submits an analysis task via the API service. The request includes the source video file or video stream address to be analyzed.

[0267] Step 1: Video Preprocessing

[0268] The API service passes video data to the preprocessing module.

[0269] This module is responsible for basic tasks such as decoding, alignment, and frame extraction, which converts the video stream into a standardized sequence of timestamp-aligned image frames that can be analyzed.

[0270] Step 2: Feature Extraction and Fusion

[0271] The preprocessed frame sequence is sent to the feature extraction / fusion module.

[0272] This module first calculates basic / advanced features (such as brightness, contrast, sharpness, texture, frequency domain features, etc.).

[0273] Subsequently, these features from different dimensions are fused from multiple sources to form a comprehensive feature vector that can fully characterize the image quality.

[0274] Step 3: Intelligent Analysis and Decision-Making

[0275] The fused feature vectors are then fed into the anomaly detection / prediction module.

[0276] This is the system's "brain," which uses pre-trained models to identify anomalies in videos (such as occlusion, blurring, and screen tearing) and may predict future trends in their quality.

[0277] Endpoint: Result returned

[0278] After the analysis is completed, the system returns the final analysis results to the client via API, which typically includes a comprehensive quality score and a detailed list of anomalies (type, location, confidence level, etc.).

[0279] Support and assurance process

[0280] To ensure the stability, efficiency, and maintainability of the core processes, the system also incorporates three key horizontal support modules:

[0281] Repository: Responsible for the persistent storage of all data. This includes: result loading / caching, which involves storing analysis results in the database for later retrieval, and potentially using caching to speed up responses to frequent requests.

[0282] Monitoring and Error Handling: This is an operations and maintenance assurance module. It is responsible for collecting logs, metrics, and error information throughout the entire process. By monitoring this data, system performance can be tracked, fault points can be located, and service reliability can be ensured.

[0283] This flowchart illustrates a complete, production-grade AI service:

[0284] User side: Simply submit your video and get intelligent analysis results.

[0285] On the server side: internally, there is a complex processing pipeline that sequentially performs preprocessing -> feature extraction -> fusion -> AI model inference, and is supported by a data storage and monitoring and alarm system.

[0286] Core steps such as Figure 4 As shown, Figure 4This paper demonstrates the complete workflow of an unsupervised, multimodal anomaly detection algorithm for video surveillance systems. It combines video content, network quality, and device signaling status for comprehensive fault diagnosis and root cause analysis.

[0287] Overall Process Overview

[0288] The core idea of ​​this algorithm is to learn the pattern of its "health state" (baseline modeling) when the system is running normally; and to detect anomalies by detecting the degree of deviation between real-time data and the "healthy baseline" (reconstruction error) during runtime. It cleverly utilizes unsupervised learning, avoiding the challenge of requiring a large amount of labeled abnormal data for training.

[0289] The entire process can be divided into three main stages: A) Multimodal data fusion -> B) Unsupervised baseline modeling -> C) Online anomaly detection and diagnosis.

[0290] Detailed analysis in stages

[0291] A) Data preprocessing and multimodal feature fusion

[0292] Multi-source data acquisition:

[0293] Video modality: High-dimensional feature vector V is extracted by periodically sampling frames (e.g., one frame every 5 seconds) and feeding them into a pre-trained CNN (e.g., ResNet). f (For example, 1024 dimensions). This feature contains visual content information of the image.

[0294] Network modality: Extract network statistics (packet loss, jitter, latency, bit rate) from the RTCP / RTP protocol and form a network feature vector V by statistically analyzing time windows (e.g., 30 seconds). n .

[0295] Signaling Modes: Extract device registration, heartbeat, and subscription status from signaling logs such as SIP / G.71181 to form a state feature vector V. s (Usually a Boolean value or an enumeration value).

[0296] Feature alignment and fusion:

[0297] The feature vectors collected at different frequencies are aligned according to the timestamp, and missing values ​​are imputed.

[0298] All features are concatenated into a unified multimodal feature vector V combined = [V f V n V s This step is crucial, as it links image quality, network transmission quality, and the device's own status.

[0299] B) Unsupervised baseline modeling

[0300] Model selection: Baseline modeling is performed using an autoencoder. It consists of an encoder f θ (·) and a decoder g θ (·) Composition.

[0301] Training process:

[0302] Data: V during a period of normal system operation (e.g., the last two weeks). combined The data is used as the training set. "Using only normal data for training" is the core of unsupervised anomaly detection.

[0303] Objective: Train an autoencoder so that it can perfectly reconstruct normal input data. That is, minimize the reconstruction loss L = |x - g|. θ (f θ (x))| |². Through this process, the autoencoder learns the compressed representation (latent space) and distribution pattern of normal data.

[0304] C) Online anomaly detection and diagnosis

[0305] Exception trigger:

[0306] During online runtime, real-time multimodal features x t = V combined(t) Input the trained autoencoder.

[0307] Calculate its reconstruction error E t = |x t - x t '|² (where x t 'This is the reconstructed output.'

[0308] If E t If the error exceeds a dynamic threshold T, it is considered abnormal. The threshold T is the mean μ of the reconstruction error from normal data during the training period. E and standard deviation σ E Decision: T = μ E + k×σ E The parameter k can be used to adjust the sensitivity of the system, balancing false alarms and missed alarms.

[0309] Intelligent diagnosis and output:

[0310] Contribution decomposition: The algorithm can analyze the source of reconstruction errors and determine whether the anomalies are caused by video content (such as occlusion or blurring), network problems (such as high packet loss), or signaling problems (such as device offline). This is the key to achieving root cause localization.

[0311] Overall Output: The final output includes not only the exception label, but also:

[0312] Level: The severity of the anomaly.

[0313] Root cause: points to the most important module that is experiencing an error.

[0314] Quality score Q: A normalized score between 0 and 1 that directly reflects the current quality status.

[0315] The core algorithm deeply integrates video visual features, network transmission indicators, and device signaling status, providing a panoramic view for fault diagnosis. Employing unsupervised learning, it eliminates the need for time-consuming and laborious anomaly data annotation, enabling it to adapt to different scenarios and discover unknown types of anomalies.

[0316] Business value:

[0317] Comprehensive detection: It can detect network or signaling problems that cannot be detected by purely visual analysis.

[0318] The root cause is clear: it can directly tell maintenance personnel "whether the camera is broken, the network is down, or the device is offline", which greatly improves maintenance efficiency.

[0319] Highly adaptive: By adjusting parameter k and continuously updating the baseline, it can adapt to the unique environment of different monitoring points.

[0320] Specifically, the image preprocessing process involves unifying video streams or files from different sources into an analyzable image sequence.

[0321] Input conditions: The video may come from different protocols (such as RTP, RTSP), different encoding formats (H.264 / H.265), different resolutions and frame rates.

[0322] Main processing steps:

[0323] 1. Decoding and Format Unification: Use decoding libraries (such as FFmpeg, OpenCV) to decode the compressed video stream into frame images and unify them into a specified pixel format (such as BGR).

[0324] 2. Color Space Conversion: Different cameras may output YUV or RGB. This step converts them to a fixed working color space and records the conversion parameters to ensure traceability of the results.

[0325] 3. Resolution normalization: A scaling algorithm that maintains the aspect ratio is used to unify the frame images to a standard resolution (such as 720p or 1080p) to provide consistent input for subsequent feature calculations.

[0326] 4. Time alignment: Handles unstable frame rates, dropped frames, and timestamp drift by using interpolation or sampling methods to ensure that the output frames are evenly arranged on the timeline.

[0327] 5. Noise Reduction and Enhancement: Perform light noise reduction and sharpening on the frame image to ensure detail clarity, but avoid over-processing.

[0328] Keyframe and time slice division: Based on the GOP structure or scene segmentation logic, keyframes are extracted and time slices are formed to establish a time reference for subsequent analysis.

[0329] Output: Structured frame sequences (uniform format, uniform resolution, time-axis aligned), and logs and parameters for each batch, which serve as input for subsequent feature extraction.

[0330] The basic feature extraction process involves extracting fundamental indicators that objectively reflect image quality from a single frame or short time slice.

[0331] Input conditions: Preprocessed frame sequence.

[0332] Main feature categories:

[0333] 1. Brightness and Exposure: Statistically calculate the mean and variance of image brightness, determine whether the pixel distribution is within a reasonable range, and assess whether the exposure is excessive or insufficient.

[0334] 2. Contrast and Detail: Measure the depth of an image by pixel differences and local variance.

[0335] 3. Sharpness and blurriness: The sharpness of an image is evaluated using edge operators and frequency domain energy.

[0336] 4. Noise Level: Different types of noise are distinguished and quantified through filtering comparison and residual calculation.

[0337] 5. Motion intensity: Calculates inter-frame motion vectors to distinguish between camera shake and scene changes.

[0338] 6. Color Shift: Analyze the color space distribution to determine if there is a color cast.

[0339] Output: Time-bound basic feature sequences (such as brightness curves and sharpness curves) for subsequent advanced feature analysis and anomaly detection.

[0340] Advanced feature analysis processes enable deeper semantic and temporal analysis of images and video clips.

[0341] Input conditions: basic feature results and original frame sequence.

[0342] Main analysis content:

[0343] 1. Content complexity: Analyze texture direction and detail energy to determine whether the image loses detail due to out-of-focus or blurry conditions.

[0344] 2. Texture features: The complexity of the image is assessed by statistically analyzing texture consistency through directional filtering.

[0345] 3. Scene transition: Detect abrupt or gradual scene changes based on color distribution and structural similarity.

[0346] 4. Artifacts and Compression Trace: Detects block artifacts and edge artifacts, and identifies the quality degradation caused by compression.

[0347] 5. Text and Overlay Content: Identify the presence of text in the image to help determine image quality interference.

[0348] 6. Color diversity: Measures the richness of color distribution to judge the realism of the image.

[0349] 7. Motion Consistency: Analyze whether the motion direction is coordinated and identify abnormal jitter or ghosting.

[0350] Output: High-level feature vectors (including scene labels, complexity scores, etc.) that provide contextual information for anomaly detection.

[0351] The feature fusion and optimization process involves uniformly calculating features from different sources into a comprehensive evaluation index.

[0352] Input conditions: results of basic features and advanced features.

[0353] Main processing logic:

[0354] 1. Standardization of similar features: Eliminate differences in dimensions to obtain a uniform score.

[0355] 2. Cross-class feature weighted fusion: Calculate the comprehensive quality score based on scene category or historical statistics.

[0356] 3. Anomaly suppression: Truncate or smooth out outlier features to avoid interference from extreme points.

[0357] 4. Caching and reuse: Avoid redundant calculations and improve efficiency.

[0358] 5. Parallelism and Optimization: Combining batch processing and multi-core parallelism improves real-time performance.

[0359] Output: Quality of Energy (QoE) score sequence and feature contribution description, used for detection and prediction.

[0360] Anomaly detection: Anomalies were detected in the video.

[0361] Input condition: The fused quality index sequence.

[0362] Detection logic:

[0363] 1. Rule detection: Set thresholds to quickly identify problems such as excessively dark brightness, overexposure, and abnormal frame rate.

[0364] 2. Statistical model testing: Abrupt changes are detected by testing the stationarity of time series data.

[0365] 3. Machine learning model detection: Use clustering or classification models to identify abnormal frames or segments.

[0366] 4. Event Generation: Merge consecutive abnormal frames into an event and label it with type, confidence level and time period.

[0367] Output: A list of abnormal events, including type, occurrence time, duration, and reference characteristics. For example... Figure 5 As shown. This chart consists of two parts: upper and lower.

[0368] The upper part (Feature Contributions) is a stacked bar chart showing the weight of each image quality feature on the overall score at different time points.

[0369] The lower half (Quality Score Trend & Anomaly Score Trend): A time series line chart with dual Y-axis plots showing the trends in overall quality score and anomaly score.

[0370] The horizontal axis represents time points (0, 2, A, 6, B, 8...), indicating different timestamps or analysis windows.

[0371] Feature list: The system monitors 8 core image quality features: brightness, contrast, sharpness, noise, complexity, texture, compression artifacts, and consistency (temporal stability).

[0372] Blue curve (Quality Score): Represents the overall quality score; the higher the value, the better (0-1 or 0-100 scale).

[0373] The red curve (Anomaly Score) represents the anomaly score; the higher the value, the more abnormal the anomaly.

[0374] Trend prediction: Predict future quality changes based on historical data to identify potential problems in advance.

[0375] Input conditions: historical quality score sequence and abnormal event sequence.

[0376] Prediction logic:

[0377] 1. Statistical trend forecasting: Analyze periodicity and trends to predict the possible quality of future intervals.

[0378] 2. Regression model prediction: Combine environmental factors (day and night, scene type) to predict short-term changes.

[0379] 3. Risk Level Classification: Based on the prediction results, the risk is divided into three levels: high, medium, and low.

[0380] Output results: Quality predictions and risk levels for a future period, providing early warnings and maintenance recommendations. For example... Figure 6 As shown.

[0381] The key algorithm program is as follows, including:

[0382] Texture feature analysis

[0383] def calculate_texture_features(frame: np.ndarray) -> Dict[str,float]:

[0384] """

[0385] Calculate texture features

[0386] Implementation principle:

[0387] 1. Calculate the gray-level co-occurrence matrix.

[0388] 2. Extract multiple statistical features

[0389] 3. Analyze the directionality and regularity of the texture.

[0390] Args:

[0391] frame: BGR format frame data

[0392] Returns:

[0393] Dict: Contains multiple texture feature metrics

[0394] """

[0395] gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

[0396] # Calculate GLCM

[0397] def compute_glcm(img: np.ndarray) -> np.ndarray:

[0398] levels = 16

[0399] img = ((img / 256) * levels).astype(np.uint8)

[0400] h, w = img.shape

[0401] glcm = np.zeros((levels, levels))

[0402] for i in range(h-1):

[0403] for j in range(w-1):

[0404] i_val = img[i,j]

[0405] j_val = img[i+1,j+1]

[0406] glcm[i_val,j_val] += 1

[0407] # Normalization

[0408] if glcm.sum() > 0:

[0409] glcm = glcm / glcm.sum()

[0410] return glcm

[0411] glcm = compute_glcm(gray)

[0412] # Calculate texture features

[0413] i, j = np.ogrid[:glcm.shape[0], :glcm.shape[1]]

[0414] # Contrast

[0415] contrast = np.sum(glcm * ((ij)**2))

[0416] # Homogeneity

[0417] homogeneity = np.sum(glcm / (1 + (ij)**2))

[0418] # Energy

[0419] energy = np.sum(glcm**2)

[0420] # Correlation

[0421] mean_i = np.sum(i * np.sum(glcm, axis=1))

[0422] mean_j = np.sum(j * np.sum(glcm, axis=0))

[0423] std_i = np.sqrt(np.sum((i - mean_i)**2 * np.sum(glcm, axis=1)))

[0424] std_j = np.sqrt(np.sum((j - mean_j)**2 * np.sum(glcm, axis=0)))

[0425] correlation = np.sum(glcm * (i - mean_i) * (j - mean_j)) / (std_i* std_j)

[0426] return {

[0427] 'contrast': float(contrast),

[0428] [[ID=�]])'homogeneity': float(homogeneity),

[0429] 'energy': float(energy),

[0430] 'correlation': float(correlation)

[0431] }

[0432] Frequency domain feature analysis

[0433] def analyze_frequency_domain(frame: np.ndarray) -> Dict[str, float]:

[0434] """

[0435] Analyze frequency domain features

[0436] Implementation principle:

[0437] 1. Perform a two-dimensional Fourier transform

[0438] 2. Analyze the spectral energy distribution

[0439] 3. Extract frequency band features

[0440] Args:

[0441] frame: BGR format frame data

[0442] Returns:

[0443] Dict: Frequency domain characteristic index

[0444] """

[0445] gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

[0446] # Perform FFT

[0447] dft = cv2.dft(np.float32(gray), flags=cv2.DFT_COMPLEX_OUTPUT)

[0448] dft_shift = np.fft.fftshift(dft)

[0449] magnitude_spectrum = 20 * np.log(

[0450] cv2.magnitude(dft_shift[:,:,0], dft_shift[:,:,1]) )

[0452] # Analyze frequency band energy

[0453] h, w = magnitude_spectrum.shape

[0454] center_y, center_x = h / / 2, w / / 2

[0455] # Low-frequency energy

[0456] low_freq = magnitude_spectrum[

[0457] center_y-h / / 8:center_y+h / / 8,

[0458] center_x-w / / 8:center_x+w / / 8 ]

[0460] # High-frequency energy

[0461] high_freq = magnitude_spectrum[

[0462] 0:h / / 4, 0:w / / 4 ]

[0464] # Mid-frequency energy

[0465] mid_freq = magnitude_spectrum[

[0466] h / / 4:3*h / / 4,

[0467] w / / 4:3*w / / 4 ]

[0469] return {

[0470] 'low_freq_energy': float(np.sum(low_freq)),

[0471] 'high_freq_energy': float(np.sum(high_freq)),

[0472] 'mid_freq_energy': float(np.sum(mid_freq)),

[0473] 'freq_ratio': float(np.sum(low_freq) / np.sum(high_freq))

[0474] }

[0475] This algorithm aims to quantify the texture characteristics of an image. Its core principle is to reveal the microstructure of the image by analyzing the spatial relationships between pixels and their neighboring pixels. It mainly consists of three steps:

[0476] First, the algorithm converts the color image to a grayscale image and captures the frequency of pixel pairs occurring in a specific direction (the bottom right diagonal in the code) by calculating the Gray-Level Co-occurrence Matrices (GLCM). This matrix essentially reflects the coarseness and directional patterns of the image texture.

[0477] The algorithm then extracts four key statistical features from the matrix:

[0478] Contrast: Measures the intensity of local changes in an image. A high value indicates deep texture grooves and sharp edges.

[0479] Homogeneity: Reflects the uniformity of local texture in an image; a high value indicates fine and regular texture.

[0480] Energy: Represents the regularity of image texture. A high value indicates that pixel pairs are concentrated and the texture pattern is highly repetitive.

[0481] Correlation: Indicates the linearity of image texture; a high value means that the texture has a clear linear structure in a certain direction.

[0482] Ultimately, the algorithm outputs a set of these four metrics, providing quantitative data support for determining whether an image has lost details due to blur, smudges, or defocus.

[0483] Frequency Domain Feature Analysis Algorithm Description

[0484] This algorithm analyzes images from a frequency perspective. Its core idea is to transform the image from the spatial domain to the frequency domain to analyze its energy distribution across different frequency components. This helps to discover degradation patterns that are difficult for the human eye to perceive directly in the spatial domain.

[0485] The algorithm execution process is as follows:

[0486] First, the image is converted to grayscale and then subjected to a two-dimensional Fast Fourier Transform (FFT) to obtain the image's spectrum. The position in the spectrum represents a frequency component, while the brightness represents the intensity of that frequency's energy.

[0487] Next, the algorithm divides the spectrogram into three core regions and calculates the sum of their energies:

[0488] Low-frequency energy: mainly concentrated in the center of the spectrum, representing the macroscopic contours and smooth areas of the image.

[0489] High-frequency energy: distributed at the edges of the spectrum, corresponding to fine details, sharp edges, and noise in an image.

[0490] Mid-frequency energy: Located between the two, it carries most of the structural information of the image.

[0491] In addition, the algorithm also calculates the low-to-high frequency energy ratio, which is a very effective indicator. If the image becomes blurry, its high-frequency energy will decrease significantly, causing this ratio to increase; conversely, if the image has artifacts such as ringing, there may be an abnormal increase in energy in a specific mid-frequency region.

[0492] By analyzing the distribution and proportion of energy in these frequency bands, the algorithm can effectively detect image quality degradation caused by lens focusing errors, compression damage (such as blockiness), or motion blur.

[0493] Example 3

[0494] Embodiment 3 of this disclosure provides a camera imaging quality self-checking system, such as Figure 7 As shown, the system includes:

[0495] Acquisition module 11 is configured to acquire the real-time video stream from the target camera;

[0496] Preprocessing module 12 is configured to preprocess the real-time video stream to obtain a standardized image sequence;

[0497] Extraction module 13 is configured to extract multi-dimensional image quality features from the standardized image sequence;

[0498] Module 14 is configured to establish and update the historical benchmark feature library of the target camera. The historical benchmark feature library is constructed by continuously collecting historical image samples taken by the camera in a fixed scene and extracting their image quality features.

[0499] The calculation module 15 is configured to perform dynamic difference comparison between the real-time extracted multi-dimensional image quality features and the corresponding historical features in the historical benchmark feature library to calculate a degradation index that characterizes the degree of quality degradation.

[0500] The analysis module 16 is configured to perform anomaly detection and classification based on the time series of the degradation index and multi-dimensional image quality features through an analysis model.

[0501] Furthermore, the preprocessing module 12 is specifically configured as follows:

[0502] Decode and unify the format of the input video stream;

[0503] Convert the decoded image to the specified color space;

[0504] Scale the image to standard resolution and align it to the timeline;

[0505] Keyframe and time slice division: Based on the GOP structure or scene segmentation logic, keyframes are extracted and time slices are formed to establish a time reference for subsequent analysis.

[0506] Furthermore, the extraction module 13 is specifically configured as follows:

[0507] Basic feature extraction: For the preprocessed structured frame sequence, extract basic indicators that can objectively reflect the image quality from a single frame or a short time slice, and output a basic feature sequence bound to time. The basic indicators include brightness and exposure, contrast and detail, sharpness and blur, noise level, motion intensity, and color shift.

[0508] Advanced Feature Analysis: Based on the basic feature extraction results and the original frame sequence, perform more in-depth semantic and temporal analysis on images and video clips. The analysis includes: content complexity, texture features, scene transitions, artifacts and compression traces, text and overlay content, color diversity, and motion consistency.

[0509] Feature fusion and optimization: Features from different sources are uniformly calculated into a comprehensive evaluation index.

[0510] Furthermore, the extraction module 13 is specifically configured as follows:

[0511] Standardize similar features to eliminate differences in dimensions and obtain a uniform score;

[0512] Perform cross-class feature weighted fusion and calculate the comprehensive quality score based on scene category or historical statistics;

[0513] Anomaly suppression is performed by truncating or smoothing outlier features to avoid interference from extreme points.

[0514] Furthermore, the calculation module 15 is specifically configured as follows:

[0515] Spatial registration and timestamp alignment are performed between real-time images and historical benchmark samples;

[0516] Calculate the difference between real-time image features and historical baseline features in the corresponding dimensions;

[0517] The difference values ​​from multiple dimensions are weighted and fused to generate a comprehensive degradation index.

[0518] Furthermore, the calculation module 15 is specifically configured as follows:

[0519] Based on preset rules, threshold judgments are made on the degradation index and / or multi-dimensional image quality features to quickly identify abnormal frames.

[0520] Based on statistical models, the time series of the multi-dimensional image quality features are subjected to stationarity or abrupt change point detection in order to discover statistically significant anomalies.

[0521] Based on machine learning models, the image sequences and their corresponding multi-dimensional features are classified or clustered to identify abnormal patterns.

[0522] Furthermore, the identified abnormal frames are merged into abnormal events, and the type, confidence level, and duration of each abnormal event are labeled.

[0523] Furthermore, the system also includes a prediction module 17;

[0524] The prediction module 17 is configured to record historical quality data, which includes historical quality score sequences and abnormal event sequences; wherein...

[0525] The historical quality score sequence consists of a comprehensive quality score calculated based on the multi-dimensional image quality features and / or the degradation index, arranged in chronological order.

[0526] The sequence of abnormal events consists of records of abnormal events arranged in chronological order and output after anomaly detection and classification.

[0527] Based on the historical quality score sequence and abnormal event sequence of the target camera, a time series prediction model is used to predict the quality change trend within a specified future time period; and...

[0528] Based on the prediction results, risk levels are assigned to current anomalies or potential risks.

[0529] Furthermore, the risk level is used to guide maintenance priorities, with high-risk levels triggering immediate maintenance work orders and medium- and low-risk levels being included in planned maintenance processes.

[0530] The camera imaging quality self-testing system of this disclosure is used to implement the camera imaging quality self-testing method in method embodiments one and two, so the description is relatively simple. For details, please refer to the relevant descriptions in the previous method embodiments, which will not be repeated here.

[0531] Figure 8 This is a block diagram of an electronic device provided in Embodiment 4 of this disclosure.

[0532] Reference Figure 8 This disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702; and one or more I / O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by at least one processor 701, and the one or more computer programs are executed by at least one processor 701 to enable at least one processor 701 to perform the above-described camera imaging quality self-test method.

[0533] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the aforementioned camera imaging quality self-checking method. The computer-readable storage medium may be volatile or non-volatile.

[0534] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described camera imaging quality self-test method.

[0535] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0536] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0537] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0538] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0539] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0540] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0541] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0542] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0543] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0544] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A camera imaging quality self-checking method, characterized in that, The method comprises: acquiring a real-time video stream of a target camera; preprocessing the real-time video stream to obtain a standardized image sequence; extracting multi-dimensional image quality features of the standardized image sequence; establishing and updating a historical reference feature library of the target camera, the historical reference feature library being constructed by continuously collecting historical image samples taken by the camera under a fixed scene and extracting image quality features thereof; dynamically comparing the real-time extracted multi-dimensional image quality features with corresponding historical features in the historical reference feature library, and calculating a degradation index representing a quality degradation degree; based on the degradation index and the time sequence of the multi-dimensional image quality features, performing abnormality detection and classification through an analysis model.

2. The method of claim 1, wherein, The preprocessing of the real-time video stream comprises: decoding and unifying the format of the input video stream; converting the decoded image to a specified color space; scaling the image to a standard resolution and performing time axis alignment; key frame and time slice division: extracting key frames and forming time slices according to the GOP structure or scene division logic, and establishing a time reference for subsequent analysis.

3. The method of claim 2, wherein, The extraction of the multi-dimensional image quality features of the standardized image sequence comprises: basic feature extraction: extracting basic indicators that can objectively reflect image quality from single frames or short time slices based on the structured frame sequence obtained after preprocessing, outputting a time-bound basic feature sequence, and the basic indicators include brightness and exposure, contrast and detail, definition and blurriness, noise level, motion intensity, color shift; advanced feature analysis: performing more in-depth semantic and time sequence analysis on images and video segments based on the basic feature extraction results and the original frame sequence, and the analysis content includes content complexity, texture features, scene switching, artifacts and compression traces, text and superimposed content, color diversity, motion consistency; feature fusion and optimization: uniformly calculating different source features into comprehensive evaluation indicators.

4. The method of claim 3, wherein, The uniformly calculating of different source features into comprehensive evaluation indicators comprises: standardizing similar features to eliminate dimensional differences and obtaining uniform scores; performing cross-class feature weighting fusion to calculate a comprehensive quality score based on scene categories or historical statistics; performing abnormality suppression to clip or smooth outlier feature values to avoid extreme point interference.

5. The method of claim 1, wherein, The dynamic differential comparison of the real-time extracted multi-dimensional image quality features with the corresponding historical features in the historical reference feature library comprises: spatial registration and time stamp alignment of real-time images and historical reference samples; calculating the differential values of real-time image features and historical reference features in corresponding dimensions; weighting and fusing the differential values of multiple dimensions to generate a comprehensive degradation index.

6. The method of claim 5, wherein, The abnormality detection and classification through the analysis model comprises: based on a preset rule, performing threshold judgment on the degradation index and / or multi-dimensional image quality features to quickly identify abnormal frames; based on a statistical model, performing stationarity or mutation point detection on the time sequence of the multi-dimensional image quality features to find statistical anomalies. based on the machine learning model, the image sequence and the corresponding multi-dimensional features are classified or clustered to identify abnormal patterns; and the identified abnormal frames are merged into an abnormal event, and the abnormal event is labeled with type, confidence and duration.

7. The method of claim 6, wherein, The method further comprises: record historical quality data, including a historical quality score sequence and an abnormal event sequence; wherein, the historical quality score sequence is composed of comprehensive quality scores calculated based on the multi-dimensional image quality features and / or the degradation index, arranged in chronological order; the abnormal event sequence is composed of abnormal event records output by abnormal detection and classification, arranged in chronological order; based on the historical quality score sequence and the abnormal event sequence of the target camera, a time series prediction model is used to predict the quality change trend in a specified future period; according to the prediction result, the current abnormality or potential risk is classified into a risk level.

8. The method of claim 7, wherein, The risk level is used to guide the operation and maintenance priority, wherein a high risk level triggers an immediate repair work order, and medium and low risk levels are included in the planned maintenance process.

9. A camera imaging quality self-checking system, characterized in that, The system comprises: an acquisition module configured to acquire a real-time video stream of a target camera; a preprocessing module configured to preprocess the real-time video stream to obtain a standardized image sequence; an extraction module configured to extract multi-dimensional image quality features of the standardized image sequence; a construction module configured to establish and update a historical reference feature library of the target camera, the historical reference feature library being constructed by continuously collecting historical image samples taken by the camera in a fixed scene and extracting image quality features thereof; a calculation module configured to dynamically compare the real-time extracted multi-dimensional image quality features with corresponding historical features in the historical reference feature library, and calculate a degradation index representing the degree of quality degradation; an analysis module configured to perform abnormal detection and classification through an analysis model based on the degradation index and the time series of multi-dimensional image quality features.

10. An electronic device, comprising: comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the camera imaging quality self-checking method of any one of claims 1-8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the camera imaging quality self-checking method of any one of claims 1-8.

12. A computer program product, characterised in that, comprises computer readable code, or a non-volatile computer readable storage medium carrying computer readable code, when the computer readable code is run in the processor of an electronic device, the processor in the electronic device performs the camera imaging quality self-checking method of any one of claims 1-8.

Images