Camera anomaly handling methods, devices, equipment and readable storage media

By extracting feature information from camera video streams and calculating comprehensive scores, a multi-dimensional judgment system is constructed, which solves the problems of high false alarm rate, high data cost and complex operation and maintenance of existing camera monitoring technologies, and realizes low-cost and easy-to-maintain camera anomaly monitoring.

CN123123559APending Publication Date: 2026-11-17CHINA MOBILE (JIANGXI) VIRTUAL REALITY TECH CO LTD +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611438855.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-16
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing camera anomaly detection technologies suffer from high false alarm rates, high data collection and labeling costs, poor environmental adaptability, stringent computing power requirements, and high maintenance costs, leading to camera monitoring failures and the loss of critical information.

Method used

By extracting feature information from N frames of images after decoding the target video stream, the comprehensive score is calculated using the analytic hierarchy process (AHP) to construct a multi-dimensional fusion judgment system. This enables accurate identification and automatic response to camera anomalies, reduces data dependence and operational barriers, and adapts to complex environmental changes.

Benefits of technology

It achieves high accuracy, low cost, and easy operation and maintenance for camera anomaly detection. It can monitor and respond quickly in real time in complex environments, reduce false alarm rate, adapt to edge deployment requirements, and reduce operation and maintenance costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123559A_ABST
    Figure CN123123559A_ABST
Patent Text Reader

Abstract

This application discloses a method, apparatus, device, and readable storage medium for handling camera anomalies. The method includes: extracting feature information from N frames of decoded images of a target video stream; calculating a comprehensive score for the N frames based on the feature information; and processing the camera anomaly based on the comprehensive score. By decoding the target video stream captured by the target camera and extracting N consecutive frames, and calculating a comprehensive score for the N frames based on their features, camera anomalies can be processed, thereby improving the accuracy of camera anomaly monitoring.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of camera anomaly monitoring technology, specifically relating to a camera anomaly handling method, device, equipment, and readable storage medium. Background Technology

[0002] Fixed cameras have been widely used in fields such as safety monitoring and security, but current monitoring systems generally lack automatic inspection capabilities. The phenomenon of people blocking or rotating cameras to avoid being filmed is frequent. Manual inspection is a huge workload and has a slow response time, which may lead to the loss of key information and the failure of camera monitoring functions.

[0003] Currently, camera anomaly detection technology typically employs a deep learning feature extraction plus threshold judgment / statistical modeling approach. This approach has high data collection and annotation costs, and the trained model is difficult to adapt to complex environments such as lighting changes and scene switching, resulting in a high false alarm rate. Summary of the Invention

[0004] This application provides a method, apparatus, device, and readable storage medium for handling camera anomalies, which solves the problem of high false alarm rate in existing monitoring schemes.

[0005] In a first aspect, embodiments of this application provide a method for handling camera anomalies, the method comprising:

[0006] Feature information is extracted from N frames of images after decoding the target video stream. The target video stream is acquired based on the target camera. The N frames are N consecutive frames from all frames of images after decoding the target video stream. The feature information includes at least one of the following: effective matching point count, geometric distance mean, feature point distribution uniformity, and inter-frame matching stability. The effective matching point count is used to determine the matching quality. The geometric distance mean is used to reflect the spatial consistency of the matching points. The feature point distribution uniformity is used to determine whether there is local occlusion. The inter-frame matching stability is used to identify progressive anomalies. N is an integer greater than 1.

[0007] Based on the feature information of the N frames of images, calculate the comprehensive score of the N frames of images;

[0008] Based on the comprehensive score, abnormal situations of the camera are handled.

[0009] Optionally, calculating the comprehensive score of the N frames of images based on their feature information includes:

[0010] The weight values ​​of effective matching point information, mean geometric distance information, feature point distribution uniformity information, and inter-frame matching stability information of the N frames of images are determined by using the hierarchical analysis method.

[0011] Based on the effective matching point information, geometric distance mean information, feature point distribution uniformity information, and inter-frame matching stability information of the N frames of images, as well as the weight values ​​of the effective matching point information, the geometric distance mean information, the feature point distribution uniformity information, and the inter-frame matching stability information, the comprehensive score of the N frames of images is calculated.

[0012] Optionally, the process of handling abnormal situations of the camera based on the comprehensive score includes:

[0013] Based on the comprehensive score, an abnormal situation of the camera is determined;

[0014] Response processing is based on the judgment results;

[0015] The step of determining the abnormal situation of the camera based on the comprehensive score includes:

[0016] If the overall score is greater than or equal to the first preset benchmark value, the camera's determination result is normal.

[0017] or

[0018] If the overall score is greater than or equal to the second preset benchmark value and less than the first preset benchmark value, the camera's determination result is a slight anomaly, which includes a small offset and / or a slight local occlusion.

[0019] or

[0020] If the overall score is less than the second preset benchmark value, the camera is determined to be severely abnormal. The severe abnormality includes severe displacement and / or complete occlusion and / or progressive abnormality.

[0021] Wherein, the first preset benchmark value is greater than the second preset benchmark value.

[0022] Optionally, the response processing based on the determination result includes:

[0023] If the determination result is normal, a first response process is performed, which includes recording the log information of the target video stream and maintaining the current sampling frequency.

[0024] or

[0025] If the determination result is slightly abnormal, a second response process is performed. The second response process includes an audible and visual warning, increasing the sampling frequency to a first preset sampling frequency, and sending the warning information corresponding to the determination result to the operation and maintenance terminal.

[0026] or

[0027] In the event that the determination result is a serious anomaly, a third response process is performed, which includes sending the warning information to the operation and maintenance terminal and the application APP on the mobile device of the person in charge, and increasing the sampling frequency to a second preset sampling frequency.

[0028] Wherein, the first preset sampling frequency is less than the second preset sampling frequency, the response time of the first response processing is less than the response time of the second response processing, and the response time of the second response processing is less than the response time of the third response processing.

[0029] Optionally, before extracting the feature information of the N frames of images after decoding the target video stream, the method further includes:

[0030] Based on the texture complexity and noise level of the N frames of images, scene classification is performed on the N frames of images;

[0031] Based on the scene classification results and the texture complexity, the feature point density of the N frames of images is adjusted.

[0032] Optionally, before extracting the feature information of the N frames of images after decoding the target video stream, the method further includes:

[0033] The N frames of images are enhanced based on an illumination enhancement algorithm. The enhancement process includes at least one of highlight suppression, overexposure correction, adaptive fusion, dark area compensation, and color gamut constraint.

[0034] Optionally, the method further includes:

[0035] If the camera's determination result is normal, the benchmark feature library is updated. The benchmark feature library includes the first preset benchmark value and the second preset benchmark value.

[0036] Secondly, embodiments of this application provide a camera anomaly handling device, the device comprising:

[0037] An extraction module is used to extract feature information from N frames of images after decoding the target video stream. The target video stream is acquired based on images captured by a target camera. The N frames are N consecutive frames from all frames of images after decoding the target video stream. The feature information includes at least one of the following: effective matching point count information, geometric distance mean information, feature point distribution uniformity information, and inter-frame matching stability information. The effective matching point count information is used to determine the matching quality. The geometric distance mean information is used to reflect the spatial consistency of the matching points. The feature point distribution uniformity information is used to determine whether there is local occlusion. The inter-frame matching stability information is used to identify progressive anomalies. N is an integer greater than 1.

[0038] The calculation module is used to calculate the comprehensive score of the N frames of images based on the feature information of the N frames of images;

[0039] The determination module is used to handle abnormal situations of the camera based on the comprehensive score.

[0040] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the method described in the first aspect.

[0041] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0042] In this embodiment, feature information is extracted from N frames of decoded target video stream images; a comprehensive score is calculated based on the feature information of the N frames; and anomalies of the camera are processed based on the comprehensive score. By decoding the target video stream captured by the target camera and extracting N consecutive frames, and calculating a comprehensive score based on the features of the N frames, anomalies of the camera can be processed, thereby improving the accuracy of camera anomaly monitoring. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating the camera anomaly handling method provided in the embodiments of this application. Figure 1 ;

[0044] Figure 2 This is a flowchart illustrating the camera anomaly handling method provided in the embodiments of this application. Figure 2 ;

[0045] Figure 3This is a schematic diagram of the camera anomaly handling device provided in the embodiments of this application;

[0046] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and are not used to describe a specified order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, a first object can be one or more. Furthermore, in the specification and claims, "and" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0049] It should be understood that the camera malfunctions described in this application specifically refer to the conditions affecting the monitoring function of fixed cameras, such as image obstruction or unexpected displacement caused by human operation or environmental factors. Fixed cameras are widely used in fields such as security monitoring and protection, but current monitoring systems generally lack automatic inspection capabilities: phenomena such as human obstruction and camera rotation to avoid being filmed are frequent, while manual inspection is labor-intensive and has a slow response time, resulting in the loss of key information and the failure of camera monitoring functions.

[0050] The core flaw of the related technology:

[0051] 1. Extremely data-dependent: Deep learning models require the collection of tens of thousands of labeled data covering multiple scenarios, resulting in high data collection and labeling costs (labeling costs for a single scenario exceed 10,000 yuan).

[0052] 2. Poor environmental adaptability: After training, the model is difficult to adapt to complex environments such as changes in lighting and scene switching, with a false alarm rate as high as 15%-30%, and the retraining and adaptation cycle is as long as 1-3 months.

[0053] 3. Stringent computing power requirements: Full feature extraction requires GPU hardware (≥8GB of video memory), and the hardware cost for deploying a single camera exceeds 500 yuan. The computing power cost for large-scale deployment (thousands of cameras) increases dramatically, making it unsuitable for edge deployment needs.

[0054] 4. High operation and maintenance costs: The model needs to be updated regularly to adapt to new scenarios, and the operation and maintenance personnel need to have professional deep learning knowledge, which has a high threshold.

[0055] The relevant technologies have led to four core solutions to this problem, as follows:

[0056] 1. Classification of related technologies and core principles

[0057] Temporal dynamic feature fusion detection method: By analyzing the temporal correlation of consecutive frame images, it captures the dynamic evolution of stains and occlusions; it uses CNN networks such as ResNet-18 to extract multi-dimensional features such as edge, texture, and brightness, and combines sliding window filtering to suppress noise. Finally, it outputs the abnormal probability through a fully connected layer and a Sigmoid function, which is good at detecting progressive dirt and intermittent occlusion.

[0058] Feature vector dot product comparison method: Based on a specifically trained network, feature vectors of the entire image are extracted. Anomalies are determined by calculating the similarity of features between consecutive frames and setting a threshold. It is suitable for the rapid detection of sudden changes in camera content in fixed scenes.

[0059] Quality Quantification Scoring Method: Construct a database of 20,000+ surveillance images covering multiple types of cameras and complex environments. Use the DB-CNN deep learning model to quantify and score video quality (threshold 40 points). It can identify eight major types of faults such as blur, color cast, and jitter, forming a complete closed loop.

[0060] Unsupervised learning detection method: By introducing an unsupervised learning framework, through feature vector histogram statistics and anomaly score calculation, abnormal devices with distribution deviations from the norm can be automatically identified without labeled data. It is suitable for unlabeled scenarios or novel fault modes.

[0061] In summary, existing solutions all revolve around "deep learning feature extraction + threshold determination / statistical modeling," achieving anomaly recognition through the learning and comparison of all image features, covering multiple technical dimensions such as time series analysis, feature comparison, quality scoring, and unsupervised detection.

[0062] Compared to the aforementioned related technologies, this application differs significantly in its technical approach, core dependencies, and deployment characteristics. These differences are specifically reflected in the following core dimensions, which are also the core content of this application:

[0063] 1. Different core technologies: The related technologies all rely on deep learning models as the core driving force and depend on the deep feature learning ability of the network to achieve anomaly recognition; this application takes "optimization of traditional feature detection algorithm family + dynamic closed-loop architecture" as the core, and through intelligent algorithm scheduling, feature hierarchical screening and multi-dimensional fusion judgment, it gets rid of the dependence on deep learning models and achieves higher scene adaptation flexibility.

[0064] 2. Different data dependence: Related technologies (including unsupervised learning methods) all require large-scale image data support. Supervised learning requires tens of thousands of labeled samples, while unsupervised learning, although it does not require labeling, still requires massive amounts of unlabeled data to construct feature distribution models. This application achieves adaptive optimization through a dynamic benchmark update mechanism of "incremental iteration + historical backtracking", with data costs close to zero, solving the pain point of high data collection and labeling costs in related technologies.

[0065] 3. Different environmental adaptation logic: Related technologies adopt the adaptation mode of "model pre-training + scene fine-tuning". When faced with changes in lighting and slow scene evolution (such as leaf growth and wall fading), the model needs to be retrained or fine-tuned, and the adaptation cycle can be as long as 1-3 months. This application, through dynamic benchmark iteration, algorithm family adaptive scheduling (real-time optimization based on texture complexity and noise level) and adaptive lighting enhancement module, can adapt to ±90% lighting fluctuation and long-term environmental changes in real time without manual intervention, and the false anomaly false alarm rate is reduced to near zero.

[0066] 4. Difference in computing power and deployment cost: Due to the computing power requirements of deep learning models, related technologies generally rely on GPU hardware (≥8GB of video memory), with a single-path deployment cost exceeding 500 yuan. The computing power cost of thousand-path deployments skyrockets, making it unsuitable for the low-cost requirements of edge computing. This application optimizes through lightweight algorithms (such as SIFT feature descriptor dimensionality reduction and hierarchical feature selection), requiring only CPU / MCU to run (computing power ≤0.5TOPS), supporting low-cost chips such as ARM Cortex-A53, with a single-path deployment cost ≤100 yuan, fully adapting to large-scale edge deployment scenarios.

[0067] 5. Different judgment and response mechanisms: Most related technologies are passive modes of "single-dimensional judgment + output results only", and the judgment logic is limited to matching score or quality score, and there is no linkage response capability. This application constructs an active closed loop of "four-dimensional fusion judgment + hierarchical feedback + sampling adaptation". The judgment dimensions cover matching quality, geometric consistency, feature distribution and inter-frame stability. At the same time, it links hardware sampling frequency adjustment and operation and maintenance early warning, which shortens the abnormal response time from 2 hours of manual inspection relied by existing technologies to within 30 seconds, and significantly improves the practicality of implementation.

[0068] 6. Different operation and maintenance thresholds: Related technologies require professional personnel to update models, optimize parameters, and troubleshoot faults, resulting in a high operation and maintenance threshold; the closed-loop architecture of this application realizes "automatic calibration + simple parameter tuning", which can be operated by ordinary operation and maintenance personnel without having deep learning knowledge, greatly reducing operation and maintenance costs and thresholds.

[0069] In summary, this application is not an improvement on existing deep learning technologies, but rather constructs a completely new technical system with lightweight, low dependency, and high closed-loop characteristics as its core, which precisely solves the core bottlenecks faced by related technologies in large-scale deployment, such as high cost, difficulty in adaptation, and complex operation and maintenance.

[0070] Addressing the four core bottlenecks of existing technologies in the field of fixed camera occlusion and displacement anomaly monitoring—namely, strong data dependence, poor environmental adaptability, high computing power costs, and high maintenance barriers—this application, guided by the core principles of "accuracy, efficiency, low cost, and ease of implementation," proposes a systematic technical objective covering all dimensions of technical performance, scenario adaptability, and deployment application. The aim is to construct a real-time monitoring method with low data requirements, strong environmental adaptability, low computing power requirements, and controllable deployment and maintenance costs, thereby completely resolving the obstacles to large-scale application of existing solutions. Specific objectives are as follows:

[0071] 1. Optimize data requirements to the extreme

[0072] By eliminating reliance on large-scale labeled or unlabeled data, it achieves a lightweight deployment model of "starting with just one initial baseline map + dynamic iterative optimization," eliminating the need for a professional team for data collection and labeling. The data preparation cost per scenario is reduced to less than 1% of traditional deep learning solutions, addressing the industry pain point of high data costs. Simultaneously, through a dynamic baseline update mechanism, it ensures that the model can adapt to slow environmental changes in long-term monitoring scenarios without requiring additional data, bringing data maintenance costs close to zero.

[0073] 2. Environmental adaptability enhancement target

[0074] It builds robust monitoring capabilities across all scenarios, stably adapting to extreme light fluctuations ranging from 5 lux (moonlight night) to 100,000 lux (strong light at noon), with the false alarm rate caused by light intensity controlled within 2.5%. For complex environments such as low texture (e.g., white-walled classrooms), high noise (e.g., rainy / foggy days, infrared monitoring), and slowly evolving scenes (e.g., leaf growth, wall fading), the anomaly detection accuracy remains above 95%, eliminating the need for scene-customized training. The environment adaptation cycle is shortened from 1-3 months in traditional solutions to within 12 hours (including automatic baseline calibration).

[0075] 3. The goal of balancing computing power and real-time performance

[0076] Achieving lightweight computing power adaptation, the core algorithm can run stably on mid-to-low-end CPU / MCU hardware such as ARM Cortex-A53 and Intel Core i3 without GPU support. The computing power requirement of a single camera is controlled within 0.5 TOPS, which reduces computing power consumption by more than 95% compared with existing deep learning solutions. At the same time, it ensures real-time monitoring performance, with single-frame anomaly detection time ≤45ms, meeting the dynamic sampling requirements of 15-30fps, and anomaly response latency controlled within 30 seconds, far exceeding the efficiency of traditional manual inspection.

[0077] 4. Improve the accuracy of anomaly detection

[0078] A multi-dimensional integrated judgment system is constructed to achieve accurate identification of anomalies of different levels. For Level 1 serious anomalies such as complete camera occlusion and malicious displacement, the judgment accuracy reaches 100% with no missed judgments. For Level 2 anomalies such as partial slight occlusion and small displacement, the accuracy is ≥96% and the false alarm rate is ≤4%. For scenes without anomalies, the false judgment rate is ≤2%. The overall judgment accuracy comprehensively surpasses the existing technology of single-dimensional judgment.

[0079] 5. Deployment and maintenance cost control targets

[0080] Lowering the barrier to large-scale deployment, the hardware deployment cost of a single camera is controlled within 100 yuan, which is more than 80% lower than the existing solution; simplifying the operation and maintenance process, through the "automatic benchmark update + hierarchical early warning feedback" mechanism, ordinary operation and maintenance personnel can complete equipment debugging and fault handling without having deep learning professional knowledge, effectively reducing camera operation and maintenance costs and achieving cost control throughout the entire process.

[0081] In summary, the technical objective of this application is not only to break through a single technical indicator, but also to build a complete technical solution that is "low-dependency, highly robust, low-cost, and easy to maintain" and adaptable to large-scale fixed camera deployment scenarios such as security monitoring, security, and park management. Ultimately, it aims to solve the core bottleneck that existing technologies cannot be scaled up and promote the upgrade of fixed camera anomaly monitoring from "manual assistance" to "fully automated intelligent inspection".

[0082] Please see Figure 1 , Figure 1 A flowchart illustrating the camera anomaly handling method provided in this application embodiment is shown in the figure. The method includes:

[0083] Step 101: Extract feature information from N frames of the target video stream after decoding.

[0084] The target video stream is acquired based on images captured by the target camera. The N frames are N consecutive frames from all decoded frames of the target video stream to avoid the risk of misjudgment of a single frame. Here, N is an integer greater than 1. In an optional embodiment, the N frames may be 10 frames selected to ensure no delay in real-time image output.

[0085] The feature information includes at least one of the following: effective matching point count information, mean geometric distance information, feature point distribution uniformity information, and inter-frame matching stability information.

[0086] The number of valid matches is used to determine the quality of the match. In an optional embodiment, valid matches can be filtered using Lowe's ratio test, the core of which is to eliminate duplicate and false matches.

[0087] Its specific process includes:

[0088] 1. For each feature point, find its two nearest neighbor matching points in the baseline map;

[0089] 2. Calculate the ratio of the distances between two nearest neighbors (nearest distance / second nearest distance);

[0090] 3. When the ratio is less than or equal to the threshold, it is determined to be a valid match point; otherwise, it is a false match point.

[0091] The threshold can be set to 0.75, which is the optimal value obtained from testing multiple sets of samples, so as to achieve a balance between matching rate and accuracy.

[0092] In an optional embodiment, the number of valid matches is calculated after Lowe's ratio test. The judgment rules can be set as follows: (Sufficient matching, matching rate ≥ 80%) (Medium match, matching rate 60%-80%) (Insufficient matches, match rate <60%).

[0093] The mean geometric distance information is used to reflect the spatial consistency of the matching points. In an optional embodiment, the mean Euclidean distance of valid matching point pairs can be calculated. Among them, for the matching point pairs The distance is The average distance of all valid matching point pairs is .

[0094] The feature point distribution uniformity information is used to determine whether local occlusion exists. In an optional embodiment, a grid coverage method can be used for quantitative evaluation. Specific steps include:

[0095] 1. Divide the image into Uniform grid (per grid in 1080P image) (pixels), grid size tested to effectively identify over 20% of local occlusion;

[0096] 2. Count the number of valid matching points in each grid. When there is at least one valid matching point in a grid, it is determined to be a "grid with matching points".

[0097] 3. Count the number of grid points with matching points. ;

[0098] 4. Uniformity score, i.e., the percentage of grid points with matching points (out of 100):

[0099]

[0100] Judgment rules can be set: (Uniformly distributed, with no local occlusion) (Localized areas, suspected to be partially obstructed, with an obstruction area of ​​approximately 20%-50%) (Severe concentration, confirmed local occlusion, occlusion area greater than 50%).

[0101] The inter-frame matching stability information is used to identify progressive anomalies, such as slow camera movement or gradual dust accumulation. In an optional embodiment, the average matching score of five consecutive frames can be used for evaluation.

[0102] Specific calculations:

[0103] 1. Select the initial normal frame as the reference frame, and its matching point count is: ;

[0104] 2. For 5 consecutive frames ( to ), calculate the matching score for each frame:

[0105]

[0106] in, For the first The number of valid matching points in a frame. For the first The average distance between frames, Pixels (the actual measured maximum effective distance);

[0107] 3. Stability score (i.e., the average score of 5 frames matching):

[0108]

[0109] Judgment rules can be set: (Stable, no progressive anomalies, inter-frame change rate <5%) (Slight fluctuations, suspected to be a gradual shift, with an inter-frame change rate of 5%-15%) (Significant decrease, confirming progressive occlusion / shift, inter-frame change rate > 15%).

[0110] In this embodiment, feature information of N frames of images is extracted to analyze the N frames of images.

[0111] Step 102: Calculate the overall score of the N frames of images based on the feature information of the N frames of images.

[0112] In an optional embodiment, the weight of each feature information can be determined using the analytic hierarchy process (AHP), and a comprehensive score for N frames of images can be calculated based on the weight of each feature information.

[0113] In an optional embodiment, the scientific nature of the weight allocation can be ensured by steps such as constructing a judgment matrix, calculating a weight vector, and performing consistency checks.

[0114] In an optional embodiment, the judgment matrix can be constructed based on scores from multiple computer vision experts, with a consistency check metric set. This satisfies the consistency requirement. Thus, the final weight allocation is: number of effective matching points (0.3), mean geometric distance (0.3), uniformity of feature point distribution (0.2), and inter-frame matching stability (0.2).

[0115] A weighted comprehensive score is used to determine the overall score. This achieves precise classification of anomaly levels. The specific calculation logic and scoring rules are as follows:

[0116]

[0117] Matching points score ( →100 points →60 points →20 points);

[0118] : Distance mean score (D≤BEST_DISTANCE→100 points, BEST_DISTANCE<D≤GOOD_DISTANCE→60 points, D>GOOD_DISTANCE→20 points);

[0119] Uniformity score (taken directly) value);

[0120] Stability score ( →100 points →60 points →20 points).

[0121] Step 103: Based on the comprehensive score, handle any abnormal situations of the camera.

[0122] After calculating the comprehensive score in step 102, the abnormal situation of the camera can be rated. Based on different abnormality levels, different judgment results are generated, and different processing methods are used to process different judgment results.

[0123] In this embodiment, feature information is extracted from N frames of decoded target video stream images; a comprehensive score is calculated based on the feature information of the N frames; and anomalies of the camera are processed based on the comprehensive score. By decoding the target video stream captured by the target camera and extracting N consecutive frames, and calculating a comprehensive score based on the features of the N frames, anomalies of the camera can be processed, thereby improving the accuracy of camera anomaly monitoring.

[0124] Optionally, calculating the comprehensive score of the N frames of images based on their feature information includes:

[0125] The weight values ​​of effective matching point information, mean geometric distance information, feature point distribution uniformity information, and inter-frame matching stability information of the N frames of images are determined by using the hierarchical analysis method.

[0126] Based on the effective matching point information, geometric distance mean information, feature point distribution uniformity information, and inter-frame matching stability information of the N frames of images, as well as the weight values ​​of the effective matching point information, the geometric distance mean information, the feature point distribution uniformity information, and the inter-frame matching stability information, the comprehensive score of the N frames of images is calculated.

[0127] In this embodiment, the weight value of each feature information is determined by using the analytic hierarchy process. Then, based on each feature information and its corresponding weight value, the comprehensive score of N frames of images is calculated, which can improve the accuracy of obtaining the comprehensive score and thus improve the efficiency of handling abnormal situations of the camera.

[0128] Optionally, the process of handling abnormal situations of the camera based on the comprehensive score includes:

[0129] Based on the comprehensive score, an abnormal situation of the camera is determined;

[0130] Response processing is carried out based on the judgment results.

[0131] In this embodiment, after obtaining the comprehensive score, the abnormal situation of the camera is judged based on the comprehensive score. Then, corresponding response processing is performed based on different judgment results, which can improve the processing efficiency of camera abnormal situations.

[0132] In an optional embodiment, determining the abnormality of the camera based on the comprehensive score includes:

[0133] If the overall score is greater than or equal to the first preset benchmark value, the camera's determination result is normal.

[0134] or

[0135] If the overall score is greater than or equal to the second preset benchmark value and less than the first preset benchmark value, the camera's determination result is a slight anomaly, which includes a small offset and / or a slight local occlusion.

[0136] or

[0137] If the overall score is less than the second preset benchmark value, the camera is determined to be severely abnormal. The severe abnormality includes severe displacement and / or complete occlusion and / or progressive abnormality.

[0138] Wherein, the first preset benchmark value is greater than the second preset benchmark value.

[0139] In this embodiment, the camera's abnormal situation is determined by comparing the comprehensive score with a first preset benchmark value and a second preset benchmark value, and the specific abnormal situation of the camera is obtained based on the determination result.

[0140] In an optional embodiment, a first preset benchmark value of 80 points and a second preset benchmark value of 50 points can be set. When the overall score is greater than or equal to 80 points, the camera's determination result is normal (no abnormality), indicating that the current camera is displaying a normal monitoring image without obstruction or displacement. When the overall score is greater than or equal to 50 points but less than 80 points, the camera's determination result is slightly abnormal (small shift and / or partial slight obstruction), indicating that the current camera may have slight camera displacement causing image edge shift, or tree branches slightly obstructing part of the lens. When the overall score is less than 50 points, the camera's determination result is severely abnormal (severe shift and / or complete obstruction and / or progressive abnormality), indicating that the current camera may have been maliciously rotated, completely obstructed by debris, or have blurred images due to long-term dust accumulation.

[0141] In another optional embodiment, the abnormality level of the camera can also be classified based on the comprehensive score. For example: if the comprehensive score is greater than or equal to a first preset benchmark value, the abnormality of the camera is classified as level 3; if the comprehensive score is greater than or equal to a second preset benchmark value and less than the first preset benchmark value, the abnormality of the camera is classified as level 2; if the comprehensive score is less than the second preset benchmark value, the abnormality of the camera is classified as level 1.

[0142] In this way, by classifying the abnormal situations of the camera based on the comprehensive score, the corresponding abnormal situation response can be carried out based on the abnormality level.

[0143] Optionally, the response processing based on the determination result includes:

[0144] If the determination result is normal, a first response process is performed, which includes recording the log information of the target video stream and maintaining the current sampling frequency.

[0145] or

[0146] If the determination result is slightly abnormal, a second response process is performed. The second response process includes an audible and visual warning, increasing the sampling frequency to a first preset sampling frequency, and sending the warning information corresponding to the determination result to the operation and maintenance terminal.

[0147] or

[0148] In the event that the determination result is a serious anomaly, a third response process is performed, which includes sending the warning information to the operation and maintenance terminal and the application APP on the mobile device of the person in charge, and increasing the sampling frequency to a second preset sampling frequency.

[0149] Wherein, the first preset sampling frequency is less than the second preset sampling frequency, the response time of the first response processing is less than the response time of the second response processing, and the response time of the second response processing is less than the response time of the third response processing.

[0150] In this embodiment, the camera is processed accordingly based on different judgment results, which can improve the efficiency of handling abnormal camera situations.

[0151] In an optional embodiment, the current sampling frequency can be set to 15fps, the first preset sampling frequency to 25fps, and the second preset sampling frequency to 30fps. When the determination result is normal, the camera's operation status can be logged, and the sampling frequency of 15fps can be maintained. The log entries may include frame numbers, timestamps, and scores for each dimension. When the determination result is slightly abnormal, a local audio-visual warning can be issued, the sampling frequency is increased from 15fps to 25fps, and the warning information can be sent to the maintenance terminal. When the determination result is severely abnormal, an emergency alarm can be pushed to the maintenance platform and the responsible person's mobile device APP, the sampling frequency can be further increased to 30fps, and an on-site alarm can be triggered. The emergency alarm information may include screenshots of five consecutive abnormal frames, timestamps, etc.

[0152] In an optional embodiment, the first response process, the second response process, and the third response process may each have a corresponding response time limit. This allows for responses based on the severity of camera anomalies, thereby improving the efficiency of handling such anomalies.

[0153] In an optional embodiment, the response time of the first response processing may be set to be shorter than that of the second response processing, and the response time of the second response processing may be set to be shorter than that of the third response processing.

[0154] In another optional embodiment, the response can also be based on the severity of the camera malfunction. For example, when performing a first response to the camera, the response content can be set to retain log records for 90 days; when performing a second response to the camera, the response content can be set to require maintenance personnel to confirm the current situation within 5 minutes before taking action; when performing a third response to the camera, the response content can be set to require maintenance personnel to arrive on-site and handle the situation directly within 30 minutes.

[0155] Optionally, before extracting the feature information of the N frames of images after decoding the target video stream, the method further includes:

[0156] Based on the texture complexity and noise level of the N frames of images, scene classification is performed on the N frames of images;

[0157] Based on the scene classification results and the texture complexity, the feature algorithm for the N frames of images is adjusted.

[0158] It should be understood that texture complexity corresponds to the gray-level entropy H of the image, while the noise level corresponds to the standard deviation σ. In an optional embodiment, texture complexity H is achieved by calculating the gray-level histogram entropy value of the image. The specific calculation steps are: 1. Convert the image to an 8-bit grayscale image and count the number of pixels for each grayscale value (0-255); 2. Calculate the probability of each grayscale value. =Number of pixels with this grayscale value / Total number of pixels in the image; 3. Substitute into the formula calculate, H represents the probability of grayscale value i. The larger the H value, the richer the texture.

[0159] noise level Through calculation The mean variance of neighboring pixels is obtained through the following steps: 1. Divide the image into non-overlapping pixels. 1. Calculate the pixel variance of each neighborhood; 2. The average of the variances of all neighborhoods is the mean. , This indicates that the image is significantly noisy.

[0160] In this way, the scene corresponding to the current N frames of images is judged based on texture complexity and noise level, and different algorithms are selected to improve the accuracy of obtaining feature information of N frames of images.

[0161] For example, when the entropy value H is greater than or equal to 10, the texture complexity is high. In this case, the scene corresponding to the N frames of images can be a busy street, a machine shop, etc., and the SIFT algorithm can be selected accordingly.

[0162] When the entropy value H is greater than or equal to 5 and less than 10, the texture complexity is moderate. In this case, the scene corresponding to N frames of images can be an office building corridor, a shopping mall lobby, etc., and the SURF algorithm can be selected accordingly.

[0163] When the entropy value H is less than 10, the texture complexity can be determined to be low. In this case, the scene corresponding to the N frames of images can be a white-walled classroom, an empty warehouse, etc., and the ORB algorithm can be selected accordingly.

[0164] at standard deviation When the value is greater than or equal to 20, it is in a high-noise scene. In this case, the scene corresponding to the N frames of images can be rainy or foggy days, nighttime infrared monitoring, etc. The Harris+Shi-Tomasi algorithm can be selected accordingly.

[0165] Optionally, before extracting the feature information of the N frames of images after decoding the target video stream, the method further includes:

[0166] The N frames of images are enhanced based on an illumination enhancement algorithm. The enhancement process includes at least one of highlight suppression, overexposure correction, adaptive fusion, dark area compensation, and color gamut constraint.

[0167] In this embodiment, the principle of highlight suppression is based on the characteristic of logarithmic transformation that "compresses highlights and stretches shadows," and is calculated using the following formula:

[0168]

[0169] use This lower base value results in a smoother compression effect on highlight areas, better preserving highlight details (compared to...). (Highlight detail retention improved by 15%). Additionally, in the formula... This is the maximum pixel value of the image, used for normalization to avoid inconsistent enhancement effects for images with different brightness levels. For example, when... In a scene with strong light, the highlight area after logarithmic transformation ( The grayscale range of the instrument was compressed from 55 to 20, effectively suppressing highlight saturation.

[0170] in, Input image pixel values ​​(0-255). The maximum pixel value of the image is used to reduce the contrast of the highlight areas through logarithmic transformation, thus avoiding saturation of strong light.

[0171] The principle of overexposure correction is based on the characteristic of exponential functions that "attenuate quickly at high intensity and at low intensity slowly." After normalizing X to [0,1], overexposure areas are corrected through exponential decay. Specifically, it can be calculated using the following formula:

[0172]

[0173] In an alternative embodiment, high-intensity pixels The attenuation rate was increased from the original fixed parameter of 0.3 to 0.6, which is a 2-fold increase, and the texture clarity of overexposed areas was improved by 40%; low-intensity pixels The attenuation amplitude is ≤0.1, basically maintaining the original brightness. It should be noted that after normalizing X to [0,1], exponential attenuation is performed on high-intensity pixels. The attenuation rate is increased by 2 times, while preserving the texture of the highlight areas.

[0174] Adaptive fusion is then performed. Dynamic calculation, in this implementation, can break through the original fixed... The design limitations of the value (usually 0.3). Based on the percentage of pixels in the dark area of ​​the image ( Adaptive calculation achieves an intelligent fusion effect of "preserving more dark areas and blending less bright areas". The specific calculation logic and parameter meanings are as follows:

[0175]

[0176]

[0177] in, The pixel is defined as a dark area pixel (corresponding to an area with a brightness of less than 20%). The calculation is performed by traversing all pixels in the image, and the time taken is ≤5ms.

[0178]

[0179] It should be noted that the higher the percentage of dark areas (such as at night), the better. The larger the value (maximum 0.5), the greater the weight of preserving details in dark areas during fusion, thus avoiding the image becoming too dark at night.

[0180] For example: Nighttime scenes hour, 38. During fusion, the weight of preserving details in dark areas is increased, and the blackish tone of the image is eliminated; in midday strong light scenes. hour, Avoid making bright areas too bright.

[0181] Dark area compensation, also known as local compensation, in an optional embodiment, introduces dark channel prior theory to achieve local compensation of dark areas based on the original hyperbolic secant distribution, thus solving the problem of "brightening dark areas but overexposing bright areas" caused by traditional global enhancement. The specific implementation steps and parameter settings are as follows:

[0182] Dark channel calculation: using Neighborhood (neighborhood size tested) Calculate the image dark channel (optimizing the balance between dark area localization accuracy and computational efficiency). Where 'c' represents the RGB channel, and the dark channel is used to locate dark areas. ).

[0183] Local compensation coefficient determination: The compensation coefficient is set based on the dark channel value. The smaller the value, the larger the compensation coefficient (0.3 is the empirically optimal value, ensuring brightening effect without noise).

[0184] For dark areas ( The formula is revised as follows:

[0185]

[0186] Effect verification: In nighttime monitoring scenarios, the clarity of details in dark areas such as pedestrian faces is improved by 30% (facial features can be clearly identified), the brightness of dark areas is improved by 15%-20%, while the brightness of bright areas (such as street light areas) does not change significantly, and there is no overexposure phenomenon.

[0187] Color gamut constraint stretches the image grayscale to a target value to ensure appropriate image contrast. In an optional embodiment, the target value can be set to... At this point, the calculation can be performed using the following formula:

[0188]

[0189] In this embodiment, by using threshold clipping (pixels less than 0 are set to 0, and pixels greater than 255 are set to 255) on the normalized pixel values, image distortion (such as color cast or whitening) caused by exceeding the display color gamut is avoided. The measured color gamut constraint improves the color reproduction of the image by 20%.

[0190] Optionally, the method further includes:

[0191] If the camera's determination result is normal, the benchmark feature library is updated. The benchmark feature library includes the first preset benchmark value and the second preset benchmark value.

[0192] In this embodiment, the benchmark feature library is updated based on an incremental iteration and historical backtracking strategy to address false anomalies caused by slow environmental changes in the original fixed benchmark. Examples include leaf growth, wall fading, and seasonal changes in sunlight.

[0193] For example, when the matching result of 5 consecutive frames (each frame is 200ms apart, covering a duration of 1 second) is level 3 (no anomalies), the benchmark feature library is updated. The new benchmark is a weighted fusion of the current frame features and the original benchmark features (weight ratio 7:3). At the same time, the historical benchmarks of the last 3 versions are retained for retrospective comparison, which is suitable for long-term (more than 6 months) monitoring scenarios.

[0194] In an optional embodiment, the sampling frequency can also be updated. For example, the video stream sampling frequency can be dynamically adjusted based on the anomaly level. Under normal conditions, sampling at 15fps (approximately 1.5MB of data per frame) can reduce computing power consumption by 60%. Under level 2 anomalies, the sampling frequency can be increased to 25fps, and under level 1 anomalies, it can be increased to 30fps to ensure the integrity of detail capture under abnormal conditions.

[0195] To better understand the camera abnormality handling method provided in the embodiments of this application, the solution proposed in this application will be described in detail below.

[0196] It should be noted that this embodiment adopts a parallel design of "mainline real-time monitoring + branchline intelligent diagnosis," which is compatible with hardware platforms of different computing power levels, such as ARM Cortex-A53 and Intel Core i3. Figure 2 As shown, the core functions and data flow rules of each module in this embodiment are explained as follows: After receiving the video stream, the controller synchronously distributes it to the decoding module. The decoded data is stored in two buffers. The monitoring buffer ensures that the real-time image output is delayed, and the diagnostic buffer caches nearly 10 frames of images for time-series feature analysis to avoid misjudgment of a single frame. After the diagnostic module outputs the results, it drives the dynamic benchmark update module to realize benchmark iteration on the one hand, and the controller is linked by the hierarchical feedback module to adjust the sampling frequency on the other hand, forming a complete closed loop.

[0197] The beneficial effects of this embodiment are as follows:

[0198] 1. Dynamic Benchmark Update Mechanism: Employing an "incremental iteration + historical backtracking" strategy, this mechanism addresses false alarms caused by slow environmental changes (such as leaf growth, wall fading, and seasonal changes in sunlight) in the original fixed benchmark. When the matching results for 5 consecutive frames (each frame spaced 200ms apart, covering a 1-second duration) are all at level 3 (no anomalies), the benchmark feature library is updated. The new benchmark is a weighted fusion of the current frame features and the original benchmark features (weight ratio 7:3). Simultaneously, the last 3 versions of historical benchmarks are retained for backtracking comparison, adapting to long-term (over 6 months) monitoring scenarios.

[0199] 2. Adaptive sampling frequency: The sampling frequency of the video stream is dynamically adjusted based on the level of abnormality. Under normal conditions, the sampling frequency is 15fps (the data volume of a single frame is about 1.5MB), which can reduce the computing power consumption by 60%. Under level 2 abnormality, the sampling frequency is increased to 25fps, and under level 1 abnormality, the sampling frequency is increased to 30fps to ensure the integrity of detail capture under abnormal conditions. This adjustment is achieved by directly controlling the frame rate parameters of the camera RTSP stream through the controller, without the need for additional data processing.

[0200] 3. Closed-Loop Feedback Link: A closed-loop "detection-judgment-response-adjustment" system is constructed. The anomaly level output by the diagnostic module directly drives two feedback branches: first, the hardware layer links with the controller to adjust the sampling frequency; second, the application layer reaches the operations and maintenance end through the hierarchical feedback module. This closed loop reduces the anomaly response time from the original 2 hours of manual inspection to within 30 seconds, significantly improving its feasibility.

[0201] The feature point detection module includes:

[0202] 1. Algorithm dynamic scheduling mechanism

[0203] Overcoming the limitation of "a single algorithm adapting to all scenarios leading to an imbalance between accuracy and efficiency," this invention constructs a dynamic scheduling logic of "scene feature quantification - intelligent algorithm selection - real-time effect evaluation," covering four major categories of traditional algorithms: SIFT, ORB, SURF, and Harris+Shi-Tomasi. It can switch algorithms within 10ms based on real-time image features. The core of this scheduling mechanism is to achieve scene classification through two quantified indicators: texture complexity (grayscale entropy H) and noise level (standard deviation σ). Specific scheduling rules are shown in Table 1 below, which also includes data on the computational power consumption and applicable hardware levels for each algorithm.

[0204]

[0205] Table 1

[0206] It should be noted that the texture complexity H is achieved by calculating the entropy value of the image's grayscale histogram. The specific calculation steps are: 1) Convert the image to an 8-bit grayscale image and count the number of pixels for each grayscale value (0-255); 2) Calculate the probability of each grayscale value. = Number of pixels with this grayscale value / Total number of pixels in the image; 3) Substitute into the formula calculate, H represents the probability of grayscale value i; a larger H value indicates richer texture. Noise level. Through calculation The mean variance of neighboring pixels is obtained by the following steps: 1) Divide the image into non-overlapping pixels. 1) Calculate the pixel variance of each neighborhood; 2) Calculate the average of the variances of all neighborhoods. , This indicates that the image is significantly noisy.

[0207] 2. Feature point hierarchical filtering mechanism

[0208] To address the issue of low matching efficiency and high false matching rate caused by the extraction of redundant feature points (such as weak contrast points and noise points in solid color areas) in traditional algorithms, a three-stage progressive screening process of "coarse screening - fine screening - uniform screening" is designed. This process can complete the screening of 1000+ feature points within 20ms, with a final redundancy rate of ≤5% for effective feature points. The specific process details are as follows:

[0209] First-level filtering: Contrast threshold filtering. The core logic is to retain feature points with significant grayscale changes. Specifically, for the initially extracted feature points, select those with significant grayscale changes. The neighborhood is calculated by subtracting the difference between the maximum and minimum grayscale values ​​within the neighborhood and removing weak contrast points with a difference ≤ 5. This step is designed based on the characteristic that "effective feature points must have obvious grayscale abrupt changes," and in practice, it can reduce redundant points by 30% while maintaining an effective feature point retention rate of ≥ 95%.

[0210] Secondary screening: RANSAC outlier removal. To address the mismatch problem in feature point matching, the Random Sample Consensus (RANSAC) algorithm is used to fit the fundamental matrix. Specific parameters include: 1000 iterations, an inlier threshold of 2 pixels, and a confidence level of 99%. By removing outliers that deviate from the fundamental matrix model using this algorithm, the mismatch rate can be reduced from the original 25%-30% to 10%-12%.

[0211] Three-level screening: Uniformity of distribution constraint. To avoid feature points concentrating in local areas such as image edges and corners, which could lead to occlusion detection failure, the image is divided into... Grid (1080P image per grid) (pixels), each grid retains ≤5 feature points. If the number of feature points in a grid exceeds 5, the top 5 with the highest contrast are retained. This step can increase the full image coverage of feature points from the original 60% to 90%, and improve the occlusion detection accuracy by 25%, which is especially suitable for partial occlusion scenarios (such as when the camera is partially occluded).

[0212] 3. SIFT Algorithm Optimization

[0213] Based on the original four-step SIFT workflow (scale space extremum detection, keypoint localization, orientation assignment, and feature descriptor generation), a new module for "dynamic adjustment of feature point density" is added, and the feature descriptors are optimized for dimensionality reduction, forming an enhanced SIFT algorithm with "adaptive density + efficient description". The overall performance of the optimized algorithm is improved by 45%. The specific optimization details are as follows:

[0214] Dynamic adjustment of feature point density: hierarchical control based on texture complexity H, suitable for high-texture scenes ( For scenes like busy streets or machine workshops, retain ≥80 feature points, typically between 80-100 in practical applications, to ensure matching accuracy (matching rate ≥90%); for medium-texture scenes (… Retain 50-60 feature points;

[0215] Low-texture scene adaptation: Low-texture scenes ( For example, in low-texture scenes such as white walls and empty warehouses, ≥30 feature points are forcibly retained. This is achieved by reducing the contrast threshold (from 0.03 to 0.01) to avoid matching failure due to insufficient feature points. In actual tests, the feature point extraction success rate in low-texture scenes increased from 60% to 90%.

[0216] Feature descriptor dimensionality optimization: PCA (Principal Component Analysis) dimensionality reduction technique is used to compress the original 128-dimensional feature vector to 64 dimensions. Dimensionality reduction process: 1) Collect 10,000 sets of SIFT feature vectors from different scenarios as a training set; 2) Calculate the covariance matrix and solve for eigenvalues ​​and eigenvectors; 3) Select the eigenvectors corresponding to the top 64 largest eigenvalues ​​to form a projection matrix; 4) Project the 128-dimensional vector to a 64-dimensional space. After optimization, the computational load is reduced by 50% (feature matching time reduced from 8ms to 4ms), and the matching accuracy loss is ≤3% (accuracy loss ranges from 1.5% to 3% in different scenarios).

[0217] The illumination enhancement algorithm module includes:

[0218] Core Improvement: Enhanced Framework for Dynamic Parameterization

[0219] Regarding the traditional solution "fixed" To address the issue of overexposure in strong light and underexposure in weak light caused by adapting to full-light scenes, an adaptive parameter calculation logic based on image brightness distribution was designed. This logic constructs a five-stage enhancement process: highlight suppression, overexposure correction, adaptive fusion, shadow compensation, and color gamut constraint. This process can adjust parameters within 30ms based on image brightness characteristics. The basis and physical meaning of the relevant key parameters are as follows:

[0220] Phase 1: Logarithmic scaling (suppressing highlights)

[0221]

[0222] Principle: Logarithmic transformation has the property of "compressing highlights and stretching shadows," and employs... The lower resolution is because it provides a smoother compression effect on highlight areas, better preserving highlight details (compared to...). (Highlight detail retention improved by 15%). This is the maximum pixel value of the image, used for normalization to avoid inconsistent enhancement effects for images with different brightness levels. For example, when... In a scene with strong light, the highlight area after logarithmic transformation ( The grayscale range of the instrument was compressed from 55 to 20, effectively suppressing highlight saturation.

[0223] Note: Input image pixel values ​​(0-255). The maximum pixel value of the image is used to reduce the contrast of the highlight areas through logarithmic transformation, thus avoiding saturation of strong light.

[0224] Phase 2: Exponential Decline (Correction of Overexposure)

[0225]

[0226] Principle: The exponential function has the characteristic of "fast decay at high intensity and slow decay at low intensity". After normalizing X to [0,1], overexposed areas are corrected through exponential decay. Actual measurements show that high-intensity pixels... The attenuation rate was increased from the original fixed parameter of 0.3 to 0.6, which is a 2-fold increase, and the texture clarity of overexposed areas was improved by 40%; low-intensity pixels The attenuation is ≤0.1, basically maintaining the original brightness.

[0227] Note: X is normalized to [0,1] and then exponential decay is performed for high-intensity pixels. The attenuation rate is increased by 2 times, while preserving the texture of the highlight areas.

[0228] Phase 3: Adaptive LIP Fusion (Innovation Points:) (Dynamic calculation)

[0229] Breakthrough the original fixed The design limitations of the value (usually 0.3). Based on the percentage of pixels in the dark area of ​​the image ( Adaptive calculation achieves an intelligent fusion effect of "preserving more dark areas and blending less bright areas". The specific calculation logic and parameter meanings are as follows:

[0230]

[0231]

[0232] in, The pixel is defined as a dark area pixel (corresponding to an area with a brightness of less than 20%). The calculation is performed by traversing all pixels in the image, and the time taken is ≤5ms.

[0233]

[0234] Example: Nighttime scene hour, 38. During fusion, the weight of preserving details in dark areas is increased, and the blackish tone of the image is eliminated; in midday strong light scenes. hour, Avoid making bright areas too bright.

[0235] Logic: The higher the percentage of dark areas (e.g., at night), the better. The larger the value (maximum 0.5), the greater the weight of preserving details in dark areas during fusion, thus avoiding the image becoming too dark at night.

[0236] Phase 4: Improved Brightness Enhancement (with Added Local Compensation)

[0237] Based on the original hyperbolic secant distribution, dark channel prior theory is introduced to achieve local compensation in dark areas, solving the problem of "brightening dark areas but overexposing bright areas" caused by traditional global enhancement. The specific implementation steps and parameter settings are as follows:

[0238] Dark channel calculation: using Neighborhood (neighborhood size tested) Calculate the image dark channel (optimizing the balance between dark area localization accuracy and computational efficiency). Where 'c' represents the RGB channel, and the dark channel is used to locate dark areas. ); 2. Determination of local compensation coefficient: Set the compensation coefficient based on the dark channel value, The smaller the value, the larger the compensation coefficient (0.3 is the empirically optimal value, ensuring brightening effect without noise).

[0239] For dark areas ( The formula is revised as follows:

[0240]

[0241] Effect verification: In nighttime monitoring scenarios, the clarity of details in dark areas such as pedestrian faces is improved by 30% (facial features can be clearly identified), the brightness of dark areas is improved by 15%-20%, while the brightness of bright areas (such as street light areas) does not change significantly, and there is no overexposure phenomenon.

[0242] Effect: Brightness in dark areas is increased by 15%-20% without overexposure (e.g., the clarity of facial details of pedestrians is improved by 30% in nighttime surveillance).

[0243] Phase 5: Normalization and Gamut Constraints

[0244]

[0245] Normalization principle: Stretch the grayscale range of the enhanced image to... This ensures appropriate image contrast. A new color gamut constraint has been added: For normalized pixel values, a threshold cropping method is used (pixels less than 0 are set to 0, and pixels greater than 255 are set to 255) to avoid image distortion (such as color cast or whitening) caused by exceeding the monitor's color gamut. Actual measurements show that color gamut constraint improves image color reproduction by 20%.

[0246] New constraint: Forcefully clip normalized pixel values ​​to... To avoid image distortion caused by exceeding the monitor's color gamut.

[0247] Enhancement effect quantification indicators

[0248] To verify the robustness of the illumination enhancement algorithm, a test platform with adjustable illumination intensity and controllable scenes was built. Test conditions included illumination intensity ranging from 5 lux (moonlight night) to 100,000 lux (strong sunlight at noon), with a variation range of ±90%. The test samples consisted of five typical monitoring scenes (street, corridor, parking lot, warehouse, and classroom), with 1000 frames per scene. The quantitative indicators obtained from the actual measurements are as follows, along with supplementary comparative data with the original fixed-parameter scheme:

[0249] Image detail retention rate: ≥92%, with 90% detail retention rate in highlight areas and 95% detail retention rate in shadow areas;

[0250] Feature point extraction success rate: ≥96%, with a 90% success rate in low-light scenes (5-10 lux);

[0251] False alarm rate due to light: ≤2.5%, with the main false alarm scenario being the superposition of heavy rain and strong light, and the false alarm rate in other scenarios being ≤1%.

[0252] The comprehensive judgment module is a four-dimensional fusion judgment model:

[0253] Breaking away from the traditional approach's reliance on a single judgment logic that "only relies on the number of matching points and geometric distance, leading to occlusion and missed detection of progressive anomalies," this system adds two new dimensions: "feature point distribution uniformity" and "inter-frame matching stability." It constructs a four-dimensional judgment system comprising "basic indicators + optimized indicators + innovative indicators," with each dimension complementing the others to cover various anomaly scenarios, including sudden anomalies (such as sudden occlusion), progressive anomalies (such as slow displacement), and local anomalies (such as local occlusion). The specific judgment system design is as follows:

[0254] Dimension 1: Number of valid match points (basic indicator)

[0255] The number of valid matches is a fundamental indicator of match quality. The Lowe's ratio test is used to filter valid matches, the core of which is to eliminate duplicate and false matches. The specific process is as follows: 1) For each feature point, find its two nearest neighbors in the baseline image; 2) Calculate the distance ratio between the two nearest neighbors (nearest distance / second nearest distance); 3) When the ratio is ≤0.75, it is considered a valid match; otherwise, it is a false match. A threshold of 0.75 is the optimal value after testing with 10,000 samples, achieving a balance between matching rate and accuracy. The number of valid matches after the Lowe's ratio test is then calculated. ;

[0256] Judgment rules: (Sufficient matching, matching rate ≥ 80%) (Medium match, matching rate 60%-80%) (Insufficient matches, match rate <60%).

[0257] Dimension 2: Mean Geometric Distance (Optimization Metric)

[0258] The mean geometric distance reflects the spatial consistency of matching points. The mean Euclidean distance of valid matching point pairs is calculated. Euclidean distance calculation: for matching point pairs The distance is The average distance of all valid matching point pairs is ;

[0259] The original fixed threshold scheme had a high misclassification rate in different texture scenes, so it was optimized to a "scene-adaptive threshold". The threshold setting is based on the matching accuracy test results of different texture scenes, as follows:

[0260] High-texture scenes: BEST_DISTANCE=5 pixels, GOOD_DISTANCE=9 pixels;

[0261] Low-texture scenes: BEST_DISTANCE=7 pixels, GOOD_DISTANCE=11 pixels.

[0262] Dimension 3: Uniformity of Feature Point Distribution (Innovation Indicator)

[0263] Feature point distribution uniformity is used to determine whether local occlusion exists, and a grid covering method is used for quantitative evaluation. Specific steps: 1) Divide the image into... Uniform grid (per grid in 1080P image) 1) The grid size (pixels) has been tested and can effectively identify more than 20% of local occlusion; 2) The number of valid matching points in each grid is counted. When there is at least one valid matching point in a grid, it is determined to be a "grid with matching points"; 3) The number of grids with matching points is counted. ;

[0264] Uniformity score, which is the percentage of grid points with matching points (out of 100):

[0265]

[0266] Judgment rules: (Uniformly distributed, with no local occlusion) (Localized areas, suspected to be partially obstructed, with an obstruction area of ​​approximately 20%-50%) (Severe concentration, confirmed local occlusion, occlusion area greater than 50%).

[0267] Dimension 4: Inter-frame matching stability (innovative metric)

[0268] Inter-frame matching stability is used to identify progressive anomalies (such as slow camera movement or gradual dust accumulation), and is evaluated using the average matching score of 5 consecutive frames. Specific calculation: 1) Select the initial normal frame as the reference frame, and its matching points are... ;2) For 5 consecutive frames ( to ), calculate the matching score for each frame:

[0269]

[0270] in For the first The number of valid matching points in a frame. For the first The average distance between frames, Pixels (the maximum effective distance measured); 3) Stability score (i.e., the average of the matching scores for 5 frames):

[0271]

[0272] Judgment rules: (Stable, no progressive anomalies, inter-frame change rate <5%) (Slight fluctuations, suspected to be a gradual shift, with an inter-frame change rate of 5%-15%) (Significant decrease, confirming progressive occlusion / shift, inter-frame change rate > 15%).

[0273] Multi-dimensional weighted judgment algorithm

[0274] The Analytic Hierarchy Process (AHP) was used to determine the weights of each dimension. The scientific nature of the weight allocation was ensured through steps such as constructing a judgment matrix, calculating the weight vector, and performing consistency checks. The judgment matrix was constructed based on scores from 30 computer vision experts, and the consistency check index... The consistency requirement was met. The final weight allocation was: number of effective matching points (0.3), mean geometric distance (0.3), uniformity of feature point distribution (0.2), and inter-frame matching stability (0.2). A comprehensive judgment score was calculated through weighted calculation. This achieves precise classification of anomaly levels. The specific calculation logic and scoring rules are as follows:

[0275]

[0276] Matching points score ( →100 points →60 points →20 points);

[0277] : Distance mean score (D≤BEST_DISTANCE→100 points, BEST_DISTANCE<D≤GOOD_DISTANCE→60 points, D>GOOD_DISTANCE→20 points);

[0278] Uniformity score (taken directly) value);

[0279] Stability score ( →100 points →60 points →20 points).

[0280] The final anomaly level determination is shown in Table 2:

[0281]

[0282] Table 2

[0283] Technical effectiveness verification: In 500 frames of testing in three typical scenarios (indoor corridor, outdoor street, and nighttime parking lot), the overall accuracy rate is as follows:

[0284] Serious anomalies (Level 1): 100% accuracy (no missed cases);

[0285] Minor anomalies (Level 2): ​​Accuracy 96.8% (false alarm rate 3.2%);

[0286] No anomalies (Level 3): Accuracy 98.2% (false alarm rate 1.8%);

[0287] Overall judgment time: ≤45ms per frame, meeting the requirements for real-time monitoring.

[0288] In summary, this application has the following execution modules:

[0289] 1. Scene Adaptive Algorithm Scheduling Module

[0290] Abandoning the inefficient traditional model of "single algorithm adapting to all scenarios," a fully adaptive end-to-end system of "scene quantification - algorithm selection - feature purification" is constructed. Real-time scene classification is achieved by calculating texture complexity (H) using grayscale entropy and noise level (σ) using the mean of neighborhood variance. Based on the classification results, the system switches between SIFT / SURF / ORB / Harris+Shi-Tomasi algorithms within 10ms. Then, a three-stage process of "coarse screening - fine screening - uniform screening" is used to purify feature points, reducing redundancy to below 5% and increasing the success rate of feature extraction for low-texture scenes from 60% to 90%.

[0291] In this embodiment of the application, the calculation logic of the scene quantification index is used, including texture complexity. Steps for calculating the entropy value of the grayscale histogram, and the noise level σ. The specific implementation of the neighborhood variance mean solution method, algorithm scheduling rules, and three-level feature point screening process solves the problem of "improvement and efficiency imbalance" in traditional algorithms.

[0292] 2. The dynamic parameterized lighting enhancement module is characterized by:

[0293] Overcoming the "overexposure in strong light and underexposure in weak light" defects caused by traditional fixed-parameter enhancement, a five-stage dynamic enhancement framework is constructed. The core mechanism calculates the proportion of dark areas by traversing pixels. ( Pixel ratio), dynamically generated fusion parameters This achieves a differentiated effect of "preserving more dark areas and blending less bright areas"; combined with logarithmic scaling to suppress highlights, exponential decay to correct overexposure, and dark channel compensation to brighten details, the feature point extraction success rate reaches 96% and the lighting false alarm rate is ≤2.5% in the 5-100000 lux illumination range.

[0294] In this embodiment of the application, a generation mechanism for the dynamic parameter λ is used, including... The computational logic; the five-stage enhancement algorithm formula, covering the derivation and application of highlight suppression, overexposure correction, and adaptive fusion; the dark channel prior dark area compensation method, namely the dynamic allocation rule of 5×5 neighborhood dark channel calculation and compensation coefficient, solves the problem of "poor scene adaptability" of traditional enhancement.

[0295] 3. The dynamic benchmark and sampling coordination module is characterized by:

[0296] Breaking away from the limitations of traditional fixed benchmarks that are prone to failure, a collaborative mechanism of "dynamic benchmark iteration + sampling frequency linkage" is established. When five consecutive frames (with a 200ms interval) are determined to be level 3 without anomalies, a benchmark update is triggered. The new benchmark is a 7:3 weighted fusion of the current frame and the original benchmark, and the last three historical benchmarks are retained for backtracking. The sampling frequency is adjusted based on the anomaly level (15fps for level 3 / 25fps for level 2 / 30fps for level 1). Under normal conditions, the computing power consumption is reduced by 60%, and no detail is missed when anomalies occur.

[0297] In this embodiment, a dynamic benchmark update mechanism is used, including a trigger condition of no anomalies for 5 consecutive frames, a calculation logic of 7:3 weighted fusion of the current frame and the original benchmark, and a method for storing and backtracking comparison of the last 3 versions of historical benchmarks; adaptive sampling frequency control, i.e., the implementation method of adjusting the camera RTSP stream frame rate through the controller, and frame rate switching rules corresponding to level 3 / 2 / 1 anomalies; and a dual buffer design, with a monitoring buffer to ensure real-time output and a diagnostic buffer to cache 10 frames for time-series analysis data flow logic, solving the problem of "environmental adaptation lag" in traditional benchmarks.

[0298] 4. The four-dimensional fusion judgment module lies in:

[0299] Overcoming the shortcomings of traditional "single-dimensional judgment of missed and false positives," a four-dimensional model of "matching quality, geometric consistency, distribution characteristics, and temporal stability" is constructed. Lowe's ratio test is used to screen valid matching points (ratio ≤ 0.75), and the mean geometric distance (dynamic threshold for different scenes), 10×10 grid distribution uniformity, and 5-frame matching stability score are calculated. Weights are allocated using the AHP method (0.3:0.3:0.2:0.2) to obtain a comprehensive score, achieving 100% accuracy for Level 1 anomalies and ≥96.8% accuracy for Level 2 anomalies.

[0300] In this embodiment, a four-dimensional index calculation method is used, including the Lowe's ratio test procedure for the number of effective matching points M, the Euclidean distance solution for the geometric distance mean D, the grid cover statistical method for the distribution uniformity U, and the stability score. The calculation of the average value of 5 frames; the weight allocation logic based on AHP, including judgment matrix construction, consistency check and final weight value; the calculation rules of the comprehensive score S and the corresponding anomaly level classification standard, solve the problem of "insufficient accuracy" in traditional judgment.

[0301] 5. The lightweight closed-loop monitoring architecture is characterized by:

[0302] Abandoning the traditional "GPU-dependent redeployment model," a parallel architecture of "mainline monitoring + branchline diagnostics" was designed. Adapting to low-cost hardware such as ARM Cortex-A53, dimensionality reduction (128-dimensional → 64-dimensional) using SIFT feature descriptor PCA and algorithm lightweight optimization achieves a single-frame decision time of ≤45ms and a computing power requirement of ≤0.5 TOPS. A closed-loop "detection-decision-response-adjustment" system is constructed, reducing anomaly response time from 2 hours to within 30 seconds, with a single-path deployment cost of ≤100 yuan.

[0303] In this embodiment, a lightweight edge-end adaptation scheme is used, including the PCA dimensionality reduction step of the SIFT algorithm (training with 100,000 samples, solving the covariance matrix, and constructing a 64-dimensional projection matrix), and the computing power optimization strategy for each module; a closed-loop feedback link, namely the linkage logic of diagnostic results driving benchmark updates, sampling frequency adjustment, and operation and maintenance early warning; and a low-cost hardware adaptation method to achieve algorithm operation optimization technology without GPU support, solving the problems of "high deployment cost and slow response" in traditional solutions.

[0304] Please see Figure 3 , Figure 3 This is a schematic diagram of the camera anomaly handling device provided in an embodiment of this application. The device 300 includes:

[0305] Extraction module 301 is used to extract feature information of N frames of images after decoding the target video stream. The target video stream is acquired based on the target camera. The N frames are N consecutive frames of images among all frames of images after decoding the target video stream. The feature information includes at least one of the following: effective matching point information, geometric distance mean information, feature point distribution uniformity information, and inter-frame matching stability information. The effective matching point information is used to determine the matching quality. The geometric distance mean information is used to reflect the spatial consistency of the matching points. The feature point distribution uniformity information is used to determine whether there is local occlusion. The inter-frame matching stability information is used to identify progressive anomalies. N is an integer greater than 1.

[0306] The calculation module 302 is used to calculate the comprehensive score of the N frames of images based on the feature information of the N frames of images;

[0307] The determination module 303 is used to process abnormal situations of the camera based on the comprehensive score.

[0308] Optionally, the computing module 302 includes:

[0309] The determining unit is used to determine the weight values ​​of the effective matching point number information, geometric distance mean information, feature point distribution uniformity information, and inter-frame matching stability information of the N frames of images using the hierarchical analysis method.

[0310] The calculation unit is used to calculate the comprehensive score of the N frames of images based on the effective matching point information, geometric distance mean information, feature point distribution uniformity information, and inter-frame matching stability information of the N frames of images, as well as the weight values ​​of the effective matching point information, the geometric distance mean information, the feature point distribution uniformity information, and the inter-frame matching stability information.

[0311] Optionally, the determination module 303 includes:

[0312] The determination unit is used to determine the abnormal situation of the camera based on the comprehensive score;

[0313] A processing unit is used to perform response processing based on the judgment result;

[0314] Specifically, the determination unit is used for:

[0315] If the overall score is greater than or equal to the first preset benchmark value, the camera's determination result is normal.

[0316] or

[0317] If the overall score is greater than or equal to the second preset benchmark value and less than the first preset benchmark value, the camera's determination result is a slight anomaly, which includes a small offset and / or a slight local occlusion.

[0318] or

[0319] If the overall score is less than the second preset benchmark value, the camera is determined to be severely abnormal. The severe abnormality includes severe displacement and / or complete occlusion and / or progressive abnormality.

[0320] Wherein, the first preset benchmark value is greater than the second preset benchmark value.

[0321] Optionally, the processing unit is specifically used for:

[0322] If the determination result is normal, a first response process is performed, which includes recording the log information of the target video stream and maintaining the current sampling frequency.

[0323] or

[0324] If the determination result is slightly abnormal, a second response process is performed. The second response process includes an audible and visual warning, increasing the sampling frequency to a first preset sampling frequency, and sending the warning information corresponding to the determination result to the operation and maintenance terminal.

[0325] or

[0326] In the event that the determination result is a serious anomaly, a third response process is performed, which includes sending the warning information to the operation and maintenance terminal and the application APP on the mobile device of the person in charge, and increasing the sampling frequency to a second preset sampling frequency.

[0327] Wherein, the first preset sampling frequency is less than the second preset sampling frequency, the response time of the first response processing is less than the response time of the second response processing, and the response time of the second response processing is less than the response time of the third response processing.

[0328] Optionally, the device further includes:

[0329] A classification module is used to classify the N frames of images into scenes based on the texture complexity and noise level of the N frames of images;

[0330] The adjustment module is used to adjust the feature point density of the N frames of images based on the scene classification results and the texture complexity.

[0331] Optionally, the device further includes:

[0332] An enhancement module is used to enhance the N frames of images based on an illumination enhancement algorithm. The enhancement process includes at least one of highlight suppression, overexposure correction, adaptive fusion, dark area compensation, and color gamut constraint.

[0333] Optionally, the device further includes:

[0334] An update module is used to update the benchmark feature library when the camera's determination result is normal. The benchmark feature library includes the first preset benchmark value and the second preset benchmark value.

[0335] It should be noted that the camera anomaly handling device provided in this application embodiment is a device capable of executing the above-described camera anomaly handling method. Therefore, all implementation methods in the above-described camera anomaly handling method embodiments are applicable to this device and can achieve the same or similar beneficial effects. To avoid repetition, this embodiment will not elaborate further.

[0336] See Figure 4 As shown in the figure, this application embodiment also provides an electronic device, including a bus 401, a transceiver 402, an antenna 403, a bus interface 404, a processor 405, and a memory 406.

[0337] The processor 405 is used for:

[0338] Feature information is extracted from N frames of images after decoding the target video stream. The target video stream is acquired based on the target camera. The N frames are N consecutive frames from all frames of images after decoding the target video stream. The feature information includes at least one of the following: effective matching point count, geometric distance mean, feature point distribution uniformity, and inter-frame matching stability. The effective matching point count is used to determine the matching quality. The geometric distance mean is used to reflect the spatial consistency of the matching points. The feature point distribution uniformity is used to determine whether there is local occlusion. The inter-frame matching stability is used to identify progressive anomalies. N is an integer greater than 1.

[0339] Based on the feature information of the N frames of images, calculate the comprehensive score of the N frames of images;

[0340] Based on the comprehensive score, abnormal situations of the camera are handled.

[0341] exist Figure 4 In this context, a bus architecture (represented by bus 401) is used. Bus 401 can include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 405 and memory represented by memory 406. Bus 401 can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 404 provides an interface between bus 401 and transceiver 402. Transceiver 402 can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 405 is transmitted over a wireless medium via antenna 403, which further receives data and transmits data to processor 405.

[0342] Processor 405 is responsible for managing bus 401 and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Memory 406 can be used to store data used by processor 405 during operation.

[0343] Alternatively, the processor 405 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0344] Optionally, there are no more than two spatial units between any two central spatial units in the N coverage areas;

[0345] The processor 405 is specifically used for:

[0346] The weight values ​​of effective matching point information, mean geometric distance information, feature point distribution uniformity information, and inter-frame matching stability information of the N frames of images are determined by using the hierarchical analysis method.

[0347] Based on the effective matching point information, geometric distance mean information, feature point distribution uniformity information, and inter-frame matching stability information of the N frames of images, as well as the weight values ​​of the effective matching point information, the geometric distance mean information, the feature point distribution uniformity information, and the inter-frame matching stability information, the comprehensive score of the N frames of images is calculated.

[0348] Optionally, the processor 405 is further configured to:

[0349] Based on the comprehensive score, an abnormal situation of the camera is determined;

[0350] Response processing is based on the judgment results;

[0351] The step of determining the abnormal situation of the camera based on the comprehensive score includes:

[0352] If the overall score is greater than or equal to the first preset benchmark value, the camera's determination result is normal.

[0353] or

[0354] If the overall score is greater than or equal to the second preset benchmark value and less than the first preset benchmark value, the camera's determination result is a slight anomaly, which includes a small offset and / or a slight local occlusion.

[0355] or

[0356] If the overall score is less than the second preset benchmark value, the camera is determined to be severely abnormal. The severe abnormality includes severe displacement and / or complete occlusion and / or progressive abnormality.

[0357] Wherein, the first preset benchmark value is greater than the second preset benchmark value.

[0358] Optionally, the processor 405 is further configured to:

[0359] If the determination result is normal, a first response process is performed, which includes recording the log information of the target video stream and maintaining the current sampling frequency.

[0360] or

[0361] If the determination result is slightly abnormal, a second response process is performed. The second response process includes an audible and visual warning, increasing the sampling frequency to a first preset sampling frequency, and sending the warning information corresponding to the determination result to the operation and maintenance terminal.

[0362] or

[0363] In the event that the determination result is a serious anomaly, a third response process is performed, which includes sending the warning information to the operation and maintenance terminal and the application APP on the mobile device of the person in charge, and increasing the sampling frequency to a second preset sampling frequency.

[0364] Wherein, the first preset sampling frequency is less than the second preset sampling frequency, the response time of the first response processing is less than the response time of the second response processing, and the response time of the second response processing is less than the response time of the third response processing.

[0365] Optionally, the processor 405 is further configured to:

[0366] Based on the texture complexity and noise level of the N frames of images, scene classification is performed on the N frames of images;

[0367] Based on the scene classification results and the texture complexity, the feature point density of the N frames of images is adjusted.

[0368] Optionally, the processor 405 is further configured to:

[0369] The N frames of images are enhanced based on an illumination enhancement algorithm. The enhancement process includes at least one of highlight suppression, overexposure correction, adaptive fusion, dark area compensation, and color gamut constraint.

[0370] Optionally, the processor 405 is further configured to:

[0371] If the camera's determination result is normal, the benchmark feature library is updated. The benchmark feature library includes the first preset benchmark value and the second preset benchmark value.

[0372] It should be noted that the electronic device provided in this application embodiment is a device capable of executing the above-described camera anomaly handling method. Therefore, all implementation methods in the above-described camera anomaly handling method embodiments are applicable to this electronic device and can achieve the same or similar beneficial effects. To avoid repetition, this embodiment will not elaborate further.

[0373] This invention also provides an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the various processes of the above-described camera anomaly handling method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0374] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described camera anomaly handling method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0375] This application also provides a computer program product, including computer instructions. When these computer instructions are executed by a processor, they implement the various processes of the above-described camera anomaly handling method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0376] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0377] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0378] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A camera abnormality processing method, characterized by, The method includes: Feature information is extracted from N frames of images after decoding the target video stream. The target video stream is acquired based on the target camera. The N frames are N consecutive frames from all frames of images after decoding the target video stream. The feature information includes at least one of the following: effective matching point count, geometric distance mean, feature point distribution uniformity, and inter-frame matching stability. The effective matching point count is used to determine the matching quality. The geometric distance mean is used to reflect the spatial consistency of the matching points. The feature point distribution uniformity is used to determine whether there is local occlusion. The inter-frame matching stability is used to identify progressive anomalies. N is an integer greater than 1. Based on the feature information of the N frames of images, calculate the comprehensive score of the N frames of images; Based on the comprehensive score, abnormal situations of the camera are handled.

2. The method of claim 1, wherein, The step of calculating the comprehensive score of the N frames of images based on their feature information includes: The weight values ​​of the effective matching point count, mean geometric distance, feature point distribution uniformity, and inter-frame matching stability information of the N frames of images are determined by using the hierarchical analysis method. Based on the effective matching point information, geometric distance mean information, feature point distribution uniformity information, and inter-frame matching stability information of the N frames of images, as well as the weight values ​​of the effective matching point information, the geometric distance mean information, the feature point distribution uniformity information, and the inter-frame matching stability information, the comprehensive score of the N frames of images is calculated.

3. The method of claim 2, wherein, The process of handling abnormal situations of the camera based on the comprehensive score includes: Based on the comprehensive score, an abnormal situation of the camera is determined; Response processing is based on the judgment results; The step of determining the abnormal situation of the camera based on the comprehensive score includes: If the overall score is greater than or equal to the first preset benchmark value, the camera's determination result is normal. or If the overall score is greater than or equal to the second preset benchmark value and less than the first preset benchmark value, the camera's determination result is a slight anomaly, which includes a small offset and / or a slight local occlusion. or If the overall score is less than the second preset benchmark value, the camera is determined to be severely abnormal. The severe abnormality includes severe displacement and / or complete occlusion and / or progressive abnormality. Wherein, the first preset benchmark value is greater than the second preset benchmark value.

4. The method of claim 3, wherein, The response processing based on the determination result includes: If the determination result is normal, a first response process is performed, which includes recording the log information of the target video stream and maintaining the current sampling frequency. or If the determination result is slightly abnormal, a second response process is performed. The second response process includes an audible and visual warning, increasing the sampling frequency to a first preset sampling frequency, and sending the warning information corresponding to the determination result to the operation and maintenance terminal. or In the event that the determination result is a serious anomaly, a third response process is performed, which includes sending the warning information to the operation and maintenance terminal and the application APP on the mobile device of the person in charge, and increasing the sampling frequency to a second preset sampling frequency. Wherein, the first preset sampling frequency is less than the second preset sampling frequency, the response time of the first response processing is less than the response time of the second response processing, and the response time of the second response processing is less than the response time of the third response processing.

5. The method of claim 1, wherein, Before extracting the feature information of the N frames of images after decoding the target video stream, the method further includes: Based on the texture complexity and noise level of the N frames of images, scene classification is performed on the N frames of images; Based on the scene classification results and the texture complexity, the feature point density of the N frames of images is adjusted.

6. The method of claim 1, wherein, Before extracting the feature information of the N frames of images after decoding the target video stream, the method further includes: The N frames of images are enhanced based on an illumination enhancement algorithm. The enhancement process includes at least one of highlight suppression, overexposure correction, adaptive fusion, dark area compensation, and color gamut constraint.

7. The method of claim 3, wherein, The method further includes: If the camera's determination result is normal, the benchmark feature library is updated. The benchmark feature library includes the first preset benchmark value and the second preset benchmark value.

8. A camera anomaly handling device, characterized in that, The device includes: An extraction module is used to extract feature information from N frames of images after decoding the target video stream. The target video stream is acquired based on images captured by a target camera. The N frames are N consecutive frames from all frames of images after decoding the target video stream. The feature information includes at least one of the following: effective matching point count information, geometric distance mean information, feature point distribution uniformity information, and inter-frame matching stability information. The effective matching point count information is used to determine the matching quality. The geometric distance mean information is used to reflect the spatial consistency of the matching points. The feature point distribution uniformity information is used to determine whether there is local occlusion. The inter-frame matching stability information is used to identify progressive anomalies. N is an integer greater than 1. The calculation module is used to calculate the comprehensive score of the N frames of images based on the feature information of the N frames of images; The determination module is used to handle abnormal situations of the camera based on the comprehensive score.

9. An electronic device, comprising: include: The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the camera anomaly handling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the camera anomaly handling method as described in any one of claims 1 to 7.