Image recognition-based video monitoring transmission delay detection method and system

Abstract

The application discloses the technical field of video monitoring and image recognition, and particularly relates to a video monitoring transmission time delay detection method and system based on image recognition. First, stable texture features of an image are extracted by SIFT to generate a 128-dimensional descriptor, then a Lucas-Kanade algorithm is used to capture inter-frame motion trends to generate time features, and finally, attention mechanism dynamic weighting fusion is used to generate a space-time joint feature vector. Then, a dynamic template containing space-time features and a local clock is generated at a sending end, and a FLANN algorithm is used for cross-end matching at a receiving end. Through a double-calibration mechanism of basic time delay calculation, clock offset calibration and jitter compensation, the influence of network jitter and clock deviation is eliminated, and finally, the calibrated time delay is output. Based on this, the application has the advantages of avoiding the interference of network jitter on time delay detection accuracy and reducing the false alarm situation caused by instantaneous deviation through double-calibration time delay calculation and cross-end feature accurate matching design.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of video monitoring and image recognition, in particular to a video monitoring transmission delay detection method and system based on image recognition. BACKGROUND

[0002] With the development of smart cities and intelligent industries, video monitoring systems have been widely used in traffic intersection red light running snapshot, factory production line equipment monitoring, community security intrusion detection and other scenarios. Such scenarios have very high requirements for video transmission delay: for example, traffic intersections need to complete the "vehicle red light running-video capture-transmission-back end judgment" process within 100ms, otherwise it is easy to miss the judgment; industrial production lines need to transmit abnormal equipment pictures within 50ms to avoid expanding faults.

[0003] 1. The current video monitoring transmission delay detection technology has the following core problems, which seriously restricts the reliability of the system:

[0004] Dependence on special hardware, high cost

[0005] The existing detection scheme needs to deploy high-precision timers (such as GPS synchronous clock modules) at the camera end and the back-end server, and the cost of a single monitoring device increases by 300-800 yuan. For example: a certain industrial park needs to deploy 200 production line monitoring, and the hardware cost of the existing scheme increases by 6-16 million yuan, which is difficult for small and medium-sized enterprises to bear.

[0006] 2. Detection accuracy is disturbed by network jitter, with high false alarm rate

[0007] The existing scheme calculates the delay by "time stamping at the camera end-receiving time stamping at the back end". However, network jitter (such as sudden bandwidth occupation) will cause time stamp transmission deviation. For example: a certain city traffic intersection monitoring, network jitter, single frame delay detection value from 80ms jumps to 250ms, false alarm "delay exceeds standard", leading to invalid troubleshooting of operation and maintenance personnel.

[0008] 3. Unable to adapt to scene differences, low detection efficiency

[0009] Different monitoring scenarios have different requirements for delay detection frequency and accuracy: for example, industrial high-speed production lines need to detect 10 times per second at high frequency, while community security only needs to detect 2 times per second at low frequency. The existing scheme uses fixed detection parameters, for example: a certain community uses industrial high-frequency detection mode, leading to CPU occupancy rate of back-end server from 30% to 75%, causing lag, which affects the normal operation of monitoring.

[0010] 4. Image distortion leads to detection failure

[0011] Rainy days and night backlight in monitoring scenes will cause image distortion. The existing scheme does not pre-process the image, resulting in inter-frame matching failure. For example, in rainy day traffic monitoring, raindrops block license plates, and the existing system cannot identify inter-frame correlation features, directly determining that "time delay detection fails", interrupting the control process. SUMMARY

[0012] To solve the technical problems of relying on special hardware, high cost, detection accuracy affected by network jitter, high false positive rate, inability to adapt to scene differences, low detection efficiency, and image distortion leading to detection failure, the present application provides the following technical solutions:

[0013] A video monitoring transmission time delay detection method based on image recognition, which includes the following specific steps:

[0014] S1: Multiple types of cameras are connected and data formats are adapted, and interference types are identified by CNN for targeted noise reduction. Different resolution images are then normalized;

[0015] S2: First, 128-dimensional descriptors are generated by extracting stable texture features of images using SIFT. Then, time features are generated by capturing inter-frame motion trends using the Lucas-Kanade algorithm. Finally, spatio-temporal joint feature vectors are generated by dynamically weighting fusion with attention mechanism;

[0016] S3: The algorithm performance is monitored in real time by a three-dimensional evaluation model. When the indicators do not meet the standards, the optimal solution is called from the multi-algorithm candidate library;

[0017] S4: A dynamic template containing spatio-temporal features and local clock is generated at the sending end, and the FLANN algorithm is used for cross-end matching at the receiving end. Then, through the double calibration mechanism of basic time delay calculation + clock offset calibration + jitter compensation, the influence of network jitter and clock deviation is eliminated, and the calibrated time delay is finally output;

[0018] S5: According to the scene priority and real-time computing power of edge devices, the computing task is dynamically allocated. At the same time, by deploying lightweight algorithms and dynamically allocating bandwidth, the cloud computing power consumption and bandwidth occupation are reduced, supporting stable operation of thousands of road systems;

[0019] S6: The scene type is identified by the CNN model, and the configuration parameters are output accordingly. At the same time, the server resources are monitored in real time. If the CPU exceeds 80%, the detection frequency of non-critical scenes is temporarily reduced, and the high-priority scenes are prioritized. The CPU occupancy rate of the server is controlled within 60%, avoiding resource overload and causing lag;

[0020] S7: First, the time delay is divided into three levels according to the preset threshold of the scene, and multi-channel alarms are triggered accordingly. Then, the 6-month time delay data is stored using AES-256 encryption, and a trend report is generated every week;

[0021] S8: Collect multi-source data of delay data, device data, environment data, and network data, and establish the association between delay exceeding and data through a Bayesian network model to accurately distinguish the root cause, generate a traceability report containing evidence and repair suggestions, and feedback to the scene module to adjust parameters.

[0022] An image recognition-based video monitoring transmission delay detection system, comprising:

[0023] An image acquisition and preprocessing module for supporting multiple types of camera access and adapting data formats, and for reducing noise through CNN recognition of interference types, and for normalizing different resolution images;

[0024] A spatio-temporal joint feature extraction module for first extracting stable texture features of images through SIFT to generate 128-dimensional descriptors, then capturing inter-frame motion trends through Lucas-Kanade algorithm to generate time features, and finally dynamically weighting and fusing with attention mechanism to generate spatio-temporal joint feature vectors;

[0025] An algorithm dynamic iterative optimization module for monitoring algorithm performance in real time through a three-dimensional evaluation model, and for calling optimal solutions from a multi-algorithm candidate library when indicators are not up to standard;

[0026] A dynamic feature matching and delay calculation module for generating a dynamic template containing spatio-temporal features and local clock at the sending end, and for matching across ends using FLANN algorithm at the receiving end, and for eliminating the effects of network jitter and clock deviation through a double-calibration mechanism of basic delay calculation + clock offset calibration + jitter compensation, and for finally outputting calibrated delay;

[0027] An edge-cloud collaborative computing module for dynamically allocating computing tasks according to scene priority and real-time computing power of edge devices, and for reducing cloud computing power consumption and bandwidth occupancy by deploying lightweight algorithms and dynamic bandwidth allocation, and for supporting stable operation of a thousand-way system;

[0028] A scene adaptive adjustment module for identifying scene types through a CNN model and outputting configuration parameters accordingly, and for monitoring server resources in real time, and for temporarily reducing non-critical scene detection frequency if CPU exceeds 80% to prioritize high-priority scenes and control server CPU occupancy rate within 60% to avoid resource overload causing lag;

[0029] An alarm and data management module for dividing delay into three levels according to preset thresholds for each scene, triggering multi-channel alarms accordingly, and storing 6-month delay data using AES-256 encryption, and generating trend reports every week;

[0030] A multi-dimensional fault tracing module is used to collect multi-source data of time delay data, equipment data, environment data and network data, and establish the association between time delay exceeding and data through a Bayesian network model, so as to accurately distinguish the root cause, generate a tracing report containing evidence and repair suggestions, and feedback to the scene module to adjust parameters.

[0031] As a preferred scheme of the video monitoring transmission time delay detection system based on image recognition, the image acquisition and preprocessing module comprises:

[0032] A multi-source image acquisition unit is used to support access to visible light cameras, infrared cameras and industrial high-speed cameras to realize the acquisition of multi-source monitoring images, automatically identify the camera type and adapt the data format without manual configuration;

[0033] An adaptive noise reduction unit is used to identify the image interference type through CNN and perform targeted noise reduction according to the identified interference type;

[0034] A resolution normalization unit is used to scale the image to a uniform resolution for different camera resolutions to avoid feature matching deviation caused by resolution differences.

[0035] As a preferred scheme of the video monitoring transmission time delay detection system based on image recognition, the spatio-temporal joint feature extraction module comprises:

[0036] A spatial texture feature extraction unit is used to extract stable texture points in the image using the SIFT algorithm, and generate a 128-dimensional descriptor for each feature point to ensure the uniqueness of the spatial feature;

[0037] A time frame change feature extraction unit is used to calculate the optical flow field of three consecutive frames of images using the Lucas-Kanade algorithm to capture the target motion trend and generate a time feature of motion vector + inter-frame difference;

[0038] A feature fusion unit is used to weight and fuse the spatial texture feature and the time frame feature through an attention mechanism to generate a spatio-temporal joint feature vector and improve the feature recognition degree.

[0039] As a preferred scheme of the video monitoring transmission time delay detection system based on image recognition, the algorithm dynamic iteration optimization module comprises:

[0040] An algorithm effect evaluation unit is used to collect the feature recognition degree of the spatio-temporal joint feature extraction module, the matching success rate and time delay calculation error of the dynamic feature matching and time delay calculation module in real time to establish a three-dimensional evaluation model of feature quality-matching effect-error value, and trigger the algorithm optimization process when a certain dimension index is lower than the threshold value for 5 consecutive frames;

[0041] A multi-algorithm candidate library unit is configured to internally store multiple sets of algorithm schemes suitable for different scenes and store them in a classified manner.

[0042] A real-time algorithm switching unit is configured to call an optimal algorithm from the candidate library according to the result of the algorithm effect evaluation unit and the scene type of the scene adaptive adjustment module.

[0043] As a preferred scheme of the video monitoring transmission time delay detection system based on image recognition, the dynamic feature matching and time delay calculation module comprises:

[0044] A dynamic template generation unit is configured to package the space-time joint feature vector and the local clock at the sending time into a feature template at the sending end.

[0045] A cross-end feature matching unit is configured to match the feature template received at the receiving end with the feature vector extracted in real time locally by using the FLANN algorithm, and the matching threshold is adaptively adjusted.

[0046] A double-calibration time delay calculation unit is configured to first perform basic time delay calculation to obtain the original time delay through the local clock at the receiving time and the clock at the sending time; then perform clock offset calibration, compare the clock difference between the sending end and the receiving end during the daily network idle period, generate a calibration coefficient, and correct the original time delay; then perform jitter compensation, calculate the average time delay by using a sliding window, eliminate the instantaneous deviation caused by network jitter, and finally output the calibrated actual time delay.

[0047] As a preferred scheme of the video monitoring transmission time delay detection system based on image recognition, the edge-cloud collaborative calculation module comprises:

[0048] A collaborative strategy decision unit is configured to receive the scene parameters of the scene adaptive adjustment module and the real-time computing power of the edge device, and dynamically allocate the computing task.

[0049] An edge computing acceleration unit is configured to deploy a lightweight algorithm on the edge device.

[0050] A bandwidth dynamic allocation unit is configured to monitor the edge-cloud transmission bandwidth in real time, and when the bandwidth is insufficient, preferentially guarantee the data transmission of high-priority scenes, and use the inter-frame compression + priority queuing strategy for low-priority scene data to ensure that critical time delay data is not lost.

[0051] As a preferred scheme of the video monitoring transmission time delay detection system based on image recognition, the scene adaptive adjustment module comprises:

[0052] A scene recognition unit is configured to identify the monitoring scene type by using a pre-trained CNN model.

[0053] The detection parameter dynamic configuration unit is configured to output configuration parameters according to scene types.

[0054] The resource scheduling unit is configured to monitor the CPU and memory occupancy rates of the backend server in real time; if the CPU occupancy rate exceeds 80%, temporarily reduce the detection frequency of non-critical scenes, and give priority to guaranteeing the detection accuracy of traffic and industrial scenes to avoid server lag.

[0055] As a preferred scheme of the video monitoring transmission time delay detection system based on image recognition, the alarm and data management module comprises:

[0056] The time delay level determination unit is configured to preset time delay threshold values for each scene, divide the calibrated time delay into three levels of normal, mild over-standard and serious over-standard, trigger a pre-warning when the time delay is mildly over-standard, and trigger an emergency alarm when the time delay is seriously over-standard.

[0057] The multi-channel alarm unit is configured to implement emergency alarms and pre-warnings.

[0058] The encrypted storage and analysis unit is configured to first store the time delay data by using AES-256 encryption, retain historical data for 6 months, then generate a time delay trend report every week, identify high-frequency over-standard cameras, and assist in operation and maintenance decisions.

[0059] As a preferred scheme of the video monitoring transmission time delay detection system based on image recognition, the multi-dimensional fault tracing module comprises:

[0060] The multi-source data acquisition unit is configured to acquire time delay data, device state data, environmental data and network data.

[0061] The fault correlation analysis unit is configured to use a Bayesian network model to establish the correlation between time delay over-standard and multi-source data.

[0062] The traceability report and repair suggestion unit is configured to generate a traceability report containing root cause types, associated data evidence and impact ranges, and push targeted repair suggestions.

[0063] Compared with the prior art, the video monitoring transmission time delay detection system based on image recognition has the following advantages:

[0064] The adaptive noise reduction and resolution normalization of the image preprocessing module eliminate the influence of image distortion, provide high-quality standardized basic data for the space-time combined feature extraction module, and real-time switch the optimal algorithm of the scene in combination with the algorithm dynamic iterative optimization module, cooperate with the cross-end precise matching and double-calibration time delay calculation mechanism of the dynamic feature matching and time delay calculation module, effectively avoid the interference of network jitter on detection accuracy; at the same time, relying on the scene recognition and parameter dynamic configuration of the scene adaptive adjustment module, the resource reasonable allocation of the edge-cloud collaborative computing module, realizing the adaptive adaptation of different monitoring scenes, without relying on special hardware throughout the process, only through the coordinated linkage of each module, not only reduces the system deployment cost, but also reduces the false alarm situation of time delay detection, avoids the interruption or failure of the detection process, at the same time, improves the overall detection efficiency, realizes the comprehensive optimization of precision, efficiency and cost. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 It is a whole framework schematic diagram of the application;

[0066] Figure 2 It is a framework schematic diagram of the image acquisition and preprocessing module of the application;

[0067] Figure 3 It is a framework schematic diagram of the space-time combined feature extraction module of the application;

[0068] Figure 4 It is a framework schematic diagram of the algorithm dynamic iterative optimization module of the application;

[0069] Figure 5 It is a framework schematic diagram of the dynamic feature matching and time delay calculation module of the application;

[0070] Figure 6 It is a framework schematic diagram of the edge-cloud collaborative computing module of the application;

[0071] Figure 7 It is a framework schematic diagram of the scene adaptive adjustment module of the application;

[0072] Figure 8 It is a framework schematic diagram of the alarm and data management module of the application;

[0073] Figure 9 It is a framework schematic diagram of the multi-dimensional fault tracing module of the application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical scheme and advantages of the application clearer, the embodiments of the application will be further described in detail below with reference to the drawings.

[0075] The application provides a video monitoring transmission time delay detection method based on image recognition, as shown in Figure 1 The specific steps are as follows:

[0076] S1: Multi-type camera access and data format adaptation are performed, and the CNN is used to identify the interference type for targeted noise reduction, and different resolution images are normalized;

[0077] S2: First, the SIFT is used to extract the stable texture features of the image to generate a 128-dimensional descriptor, then the Lucas-Kanade algorithm is used to capture the inter-frame motion trend to generate a time feature, and finally the attention mechanism is used to dynamically weight the fusion to generate a spatio-temporal joint feature vector;

[0078] S3: The three-dimensional evaluation model is used to monitor the algorithm performance in real time, and when the indicators do not meet the standards, the optimal solution is called from the multi-algorithm candidate library;

[0079] S4: A dynamic template containing spatio-temporal features and a local clock is generated at the sending end, and the FLANN algorithm is used for cross-end matching at the receiving end, and then through the double calibration mechanism of basic time delay calculation + clock offset calibration + jitter compensation, the influence of network jitter and clock deviation is eliminated, and finally the calibrated time delay is output;

[0080] S5: According to the scene priority and the real-time computing power of the edge device, the computing task is dynamically allocated; at the same time, by deploying lightweight algorithms and dynamically allocating bandwidth, the cloud computing power consumption and bandwidth occupation are reduced, and the stable operation of the system is supported;

[0081] S6: The CNN model is used to identify the scene type and output the configuration parameters accordingly, and the server resources are monitored in real time. If the CPU exceeds 80%, the non-critical scene detection frequency is temporarily reduced, the high-priority scene is prioritized, and the CPU occupancy rate of the server is controlled within 60%, avoiding resource overload and causing lag;

[0082] S7: First, the time delay is divided into three levels according to the preset threshold of the scene, and multi-channel alarms are triggered accordingly, and the AES-256 encryption is used to store the time delay data for 6 months, and a trend report is generated every week;

[0083] S8: The multi-source data of time delay data, device data, environment data, and network data are collected, and the Bayesian network model is used to establish the association between time delay exceeding and data, to accurately distinguish the root cause, generate a traceability report containing evidence and repair suggestions, and feedback to the scene module to adjust the parameters.

[0084] A video monitoring transmission time delay detection system based on image recognition, comprising:

[0085] An image acquisition and preprocessing module is used to support multi-type camera access and data format adaptation, and the CNN is used to identify the interference type for targeted noise reduction, and different resolution images are normalized;

[0086] The spatio-temporal joint feature extraction module is configured to first extract stable texture features of an image by SIFT to generate 128-dimensional descriptors, then capture inter-frame motion trends by Lucas-Kanade algorithm to generate time features, and finally dynamically weight and fuse by attention mechanism to generate a spatio-temporal joint feature vector;

[0087] The algorithm dynamic iterative optimization module is configured to monitor algorithm performance in real time by a three-dimensional evaluation model, and when the index is not up to standard, call the optimal scheme from a multi-algorithm candidate library;

[0088] The dynamic feature matching and time delay calculation module is configured to generate a dynamic template containing spatio-temporal features and a local clock at the sending end, and match across ends by FLANN algorithm at the receiving end, and then through a double calibration mechanism of basic time delay calculation + clock offset calibration + jitter compensation, eliminate the influence of network jitter and clock deviation, and finally output the calibrated time delay;

[0089] The edge-cloud collaborative computing module is configured to dynamically allocate computing tasks according to scene priority and real-time computing power of edge devices; at the same time, by deploying lightweight algorithms and dynamically allocating bandwidth, reduce the consumption of cloud computing power and bandwidth occupation, and support stable operation of thousands of road systems;

[0090] The scene adaptive adjustment module is configured to identify scene types by a CNN model, and output configuration parameters accordingly, and at the same time, monitor server resources in real time, if CPU is over 80%, temporarily reduce the detection frequency of non-critical scenes, preferentially guarantee high-priority scenes, and control the CPU occupancy rate of the server within 60%, to avoid resource overload and cause lag;

[0091] The alarm and data management module is configured to first divide the time delay into three levels according to the preset threshold of the scene, and trigger multi-channel alarms accordingly, and then store the time delay data for six months by AES-256 encryption, and generate a trend report every week;

[0092] The multi-dimensional fault tracing module is configured to collect multi-source data of time delay data, device data, environment data and network data, and establish the association between time delay exceeding and data by a Bayesian network model, to accurately distinguish the root cause, and at the same time, generate a tracing report containing evidence and repair suggestions, and feedback to the scene module to adjust parameters.

[0093] As shown in Figure 2 , the image acquisition and preprocessing module comprises:

[0094] The multi-source image acquisition unit is configured to support access to visible light cameras, infrared cameras (night scenes), and industrial high-speed cameras (production line scenes), to realize the acquisition of multi-source monitoring images, and automatically identify the camera type and adapt the data format (such as H.264, MPEG-4) without manual configuration;

[0095] Adaptive noise reduction unit, for identifying image interference types by CNN (convolutional neural network) and conducting targeted noise reduction according to the identified interference types; if it is raindrop interference in rainy days, Gaussian filtering + morphological opening operation is adopted, and if it is night backlight interference, histogram equalization + adaptive threshold segmentation is adopted, so as to improve the image signal-to-noise ratio to more than 40 dB;

[0096] Resolution normalization unit, for scaling images to a unified resolution (default 1080P, configurable) according to different camera resolutions (such as 720P, 1080P, 4K), so as to avoid feature matching deviation caused by resolution difference.

[0097] As shown in Figure 3 , the spatio-temporal joint feature extraction module comprises:

[0098] Spatial texture feature extraction unit, for extracting stable texture points (such as traffic signal lamp edges, production line equipment screw holes) in images by using SIFT (scale invariant feature transform) algorithm, generating 128-dimensional descriptors for each feature point, and ensuring spatial feature uniqueness;

[0099] Time frame change feature extraction unit, for calculating the optical flow field of consecutive 3 frames of images by using Lucas-Kanade algorithm, capturing target motion trend (such as vehicle moving direction, equipment rotation angle), and generating time features of motion vector + inter-frame difference;

[0100] Feature fusion unit, for weighted fusion (dynamically adjusting weights: motion scene time feature weight 60%, static scene spatial feature weight 70%) of spatial texture features and time frame features by attention mechanism, generating spatio-temporal joint feature vectors, and improving feature recognition degree.

[0101] As shown in Figure 4 , the algorithm dynamic iteration optimization module comprises:

[0102] Algorithm effect evaluation unit, for real-time collecting feature recognition degree (such as feature point repetition rate, descriptor uniqueness) of the spatio-temporal joint feature extraction module, matching success rate and time delay calculation error of the dynamic feature matching and time delay calculation module, so as to establish a three-dimensional evaluation model of feature quality-matching effect-error value, and trigger algorithm optimization process when a certain dimension index is lower than a threshold value (such as matching success rate < 90%) for 5 consecutive frames;

[0103] Multi-algorithm candidate library unit, for internally storing multiple sets of algorithm schemes adapted to different scenes and classified storage;

[0104] Feature extraction type: SIFT (high texture scene), ORB (low power scene), SURF (high-speed motion scene);

[0105] Matching algorithm class: FLANN (large data scenario), brute force matching (low latency scenario), KD tree matching (high precision scenario);

[0106] Algorithm parameter preset: For rainy days, night, high-speed moving scenes, pre-configure the optimal parameters of the algorithm (such as the threshold value of the number of feature points of ORB algorithm in rainy days is set to 500);

[0107] Real-time algorithm switching unit, for adjusting the scene type of the scene adaptive adjustment module according to the results of the algorithm effect evaluation unit (such as high-speed traffic scene), calling the optimal algorithm from the candidate library;

[0108] Example: In the rainy traffic scene, the success rate of SIFT algorithm matching drops to 85%, and automatically switches to ORB algorithm + rainy day parameters, and the matching success rate rises to more than 95%;

[0109] Switching process is not perceived, through caching the first 3 frames of feature data, avoiding time delay interruption, switching time ≤10ms.

[0110] As shown in Figure 5 , the dynamic feature matching and time delay calculation module comprises:

[0111] Dynamic template generation unit, for packing the space-time joint feature vector + local clock at sending time into a feature template at the sending end (camera end), generating 1 template every 20ms (which can be dynamically adjusted by the subsequent module), avoiding the invalidation of fixed templates due to scene changes;

[0112] Cross-end feature matching unit, for receiving the feature template at the receiving end (back-end server), using the FLANN (fast nearest neighbor search library) algorithm to match with the feature vector extracted in real time, and adaptively adjusting the matching threshold (such as 0.85 for industrial scene and 0.75 for community scene), the matching success rate is improved to more than 98%;

[0113] Double-calibration time delay calculation unit, for first performing basic time delay calculation, obtaining the original time delay through the local clock at receiving time-the local clock at sending time; then performing clock offset calibration, comparing the clock difference between the sending end and the receiving end during the daily network idle period (such as 3 am), generating a calibration coefficient to correct the original time delay; then performing jitter compensation, using a sliding window (window size 5-10 frames, adaptively adjusting) to calculate the average time delay, eliminating the instantaneous deviation caused by network jitter, and finally outputting the calibrated actual time delay.

[0114] As shown in Figure 6 , the edge-cloud collaborative calculation module comprises:

[0115] The collaborative strategy decision-making unit receives scene parameters (such as detection frequency and accuracy requirements) from the scene adaptive adjustment module and the real-time computing power (CPU utilization and memory usage) of the edge device, and dynamically allocates computing tasks. In high-priority scenarios (such as industrial production lines), the edge only performs preliminary feature screening (removing invalid feature points), and the cloud completes fine matching and latency calculation. In low-priority scenarios (such as community security), the edge directly completes feature matching and basic latency calculation, and only uploads data exceeding the standard to the cloud, reducing bandwidth consumption by more than 60%.

[0116] Edge computing acceleration unit for deploying lightweight algorithms on edge devices;

[0117] Feature compression: PCA (Principal Component Analysis) was used to reduce the 128-dimensional feature vector to 64-dimensional, reducing the data volume by 50%.

[0118] Local caching: It caches feature templates from nearly 100 frames. When the network is briefly interrupted, the edge device can perform short-term latency calculations based on the cache to avoid data loss.

[0119] The bandwidth dynamic allocation unit is used to monitor the edge-to-cloud transmission bandwidth in real time. When the bandwidth is insufficient (such as when the bandwidth utilization rate is >90% during peak periods), it prioritizes the data transmission of high-priority scenarios and adopts an inter-frame compression + priority queuing strategy for low-priority scenario data to ensure that critical latency data is not lost.

[0120] like Figure 7 As shown, the scene adaptive adjustment module includes:

[0121] The scene recognition unit is used to identify the type of monitored scene through a pre-trained CNN model. First, a pre-processed image is input, and then the classification results of traffic scene / industrial scene / residential scene are output (the training dataset contains more than 100,000 images of various scenes, with a recognition accuracy of more than 95%).

[0122] The detection parameter dynamic configuration unit is used to output configuration parameters according to the scene type;

[0123] Traffic scenario: Feature template generation frequency 10 times / second, matching threshold 0.85, jitter compensation window 5 frames;

[0124] Industrial scenario: Feature template generation frequency 15 times / second, matching threshold 0.9, jitter compensation window 3 frames;

[0125] Cellular scenario: Feature template generation frequency 2 times / second, matching threshold 0.75, jitter compensation window 10 frames;

[0126] Resource scheduling unit, for real-time monitoring of backend server CPU, memory occupancy rate; if CPU occupancy rate is more than 80%, temporarily reduce the detection frequency of non-critical scene (such as small area), preferentially guarantee the detection accuracy of traffic and industrial scene, avoid server jam.

[0127] As shown in Figure 8 The alarm and data management module comprises:

[0128] Delay level determination unit, for presetting delay threshold of each scene (such as traffic scene ≤100 ms, industrial scene ≤50 ms), dividing calibrated delay into three levels of normal, slight over-standard and serious over-standard, triggering early warning for slight over-standard, and triggering emergency alarm for serious over-standard;

[0129] Multi-channel alarm unit, for realizing emergency alarm and early warning; the emergency alarm is for synchronously sending short message to operation and maintenance personnel, triggering sound and light alarm of monitoring center, and pushing APP pop-up window; the early warning is for pushing APP notification and system log only, avoiding interference to operation and maintenance; wherein the alarm content contains "over-standard delay value, involved camera ID and scene type", facilitating rapid positioning;

[0130] Encrypted storage and analysis unit, for first adopting AES-256 to encrypt and store delay data (containing original delay, calibration coefficient and alarm record), and retaining 6 months of historical data; then generating delay trend report every week, identifying high-frequency over-standard camera (such as a road intersection camera with continuously over-standard delay due to line aging), and assisting operation and maintenance decision.

[0131] As shown in Figure 9 The multi-dimensional fault tracing module comprises:

[0132] Multi-source data acquisition unit, for acquiring delay data, device state data, environment data and network data; the device state data: camera frame rate, CPU temperature and network port packet loss rate; the environment data: illumination intensity (night / day), humidity (rainy day / sunny day) and shelter condition (leaves / dust); the network data: bandwidth utilization rate, routing jump number and transmission protocol (TCP / UDP) state;

[0133] Fault correlation analysis unit, for adopting Bayesian network model to establish the correlation between delay over-standard and multi-source data;

[0134] Example 1: delay over-standard + camera frame rate from 30 fps to 10 fps + CPU temperature > 80℃ → determined as "camera hardware failure";

[0135] Example 2: delay over-standard + network packet loss rate > 15% + bandwidth utilization rate > 95% → determined as "network congestion caused transmission delay";

[0136] Example 3: Time delay exceeds the standard + humidity > 90% + feature matching success rate < 80% -> determine as "rainy day environment interference causes detection error";

[0137] A traceability report and repair suggestion unit is configured to generate a traceability report containing root cause type, associated data evidence, and impact range, and push targeted repair suggestions.

[0138] Hardware failure -> suggest replacing the camera;

[0139] Network congestion -> suggest expanding bandwidth or optimizing routing;

[0140] Environmental interference -> suggest cleaning the camera lens or installing a rain cover;

[0141] At the same time, the traceability result is fed back to the scene self-adaptive adjustment module, such as temporarily reducing the detection frequency in the hardware failure scene to avoid invalid alarm.

[0142] Although the present application has been described with reference to the embodiments above, various modifications can be made thereto and equivalents can be substituted therefor without departing from the scope of the present application. In particular, features of the disclosed embodiments can be combined together in any manner, provided that there is no structural conflict. The combinations of features are not exhaustively described in the specification only for the purpose of omitting the length and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A video surveillance transmission delay detection method based on image recognition, characterized in that, The specific steps are as follows: S1: Connect multiple types of cameras and adapt the data formats, identify the types of interference through CNN and perform targeted noise reduction, and then normalize images of different resolutions. S2: First, SIFT is used to extract stable texture features of the image to generate a 128-dimensional descriptor. Then, the Lucas-Kanade algorithm is used to capture the motion trend between frames to generate temporal features. Finally, attention mechanism is used to dynamically weight and fuse the features to generate a spatiotemporal joint feature vector. S3: Monitor algorithm performance in real time through a three-dimensional evaluation model. When the indicators fail to meet the standards, the optimal solution is called from a multi-algorithm candidate library. S4: At the transmitting end, a dynamic template containing spatiotemporal features and a local clock is generated, and at the receiving end, the FLANN algorithm is used for cross-end matching. Then, through a dual calibration mechanism of basic delay calculation + clock offset calibration + jitter compensation, the effects of network jitter and clock deviation are eliminated, and finally the calibrated delay is output. S5: Dynamically allocates computing tasks based on scenario priority and real-time computing power of edge devices; at the same time, it reduces cloud computing power consumption and bandwidth occupation by deploying lightweight algorithms and dynamic bandwidth allocation, supporting the stable operation of thousands of systems; S6: Identify scene types through a CNN model and output configuration parameters accordingly. At the same time, monitor server resources in real time. If the CPU usage exceeds 80%, temporarily reduce the detection frequency of non-critical scenes and prioritize high-priority scenes to keep the server CPU usage below 60% and avoid resource overload that could cause lag. S7: First, divide the latency into three levels according to the preset threshold of the scenario, and trigger multi-channel alarms accordingly. Then, use AES-256 to encrypt and store 6 months of latency data, and generate a trend report every week. S8: Collects multi-source data including latency data, device data, environmental data, and network data, and establishes the correlation between latency exceedance and data through a Bayesian network model to accurately identify the root cause. At the same time, it generates a source tracing report containing evidence and remediation suggestions and feeds it back to the scenario module to adjust parameters.

2. A video surveillance transmission delay detection system based on image recognition, characterized in that, include: The image acquisition and preprocessing module supports access from multiple types of cameras and adapts to data formats. It also uses CNN to identify interference types and perform targeted noise reduction, and then normalizes images of different resolutions. The spatiotemporal joint feature extraction module is used to first extract stable texture features of the image through SIFT to generate a 128-dimensional descriptor, then use the Lucas-Kanade algorithm to capture the motion trend between frames to generate temporal features, and finally use an attention mechanism to dynamically weight and fuse them to generate a spatiotemporal joint feature vector. The algorithm dynamic iterative optimization module is used to monitor the algorithm performance in real time through a three-dimensional evaluation model. When the indicators are not met, the optimal solution is called from a multi-algorithm candidate library. The dynamic feature matching and delay calculation module is used to generate a dynamic template containing spatiotemporal features and local clock at the transmitting end, and to perform cross-end matching using the FLANN algorithm at the receiving end. Then, through a dual calibration mechanism of basic delay calculation + clock offset calibration + jitter compensation, the effects of network jitter and clock deviation are eliminated, and finally the calibrated delay is output. The edge-cloud collaborative computing module is used to dynamically allocate computing tasks based on scenario priority and the real-time computing power of edge devices; at the same time, by deploying lightweight algorithms and dynamically allocating bandwidth, it reduces cloud computing power consumption and bandwidth occupation, and supports the stable operation of thousands of systems. The scene adaptive adjustment module is used to identify scene types through CNN models and output configuration parameters accordingly. At the same time, it monitors server resources in real time. If the CPU usage exceeds 80%, it temporarily reduces the detection frequency of non-critical scenes and prioritizes high-priority scenes, keeping the server CPU usage below 60% to avoid resource overload and lag. The alarm and data management module is used to first classify latency into three levels according to the preset threshold of the scenario, and trigger alarms from multiple channels accordingly. Then, it uses AES-256 to encrypt and store 6 months of latency data and generates a trend report every week. The multi-dimensional fault tracing module is used to collect multi-source data such as latency data, equipment data, environmental data, and network data. It establishes the correlation between latency exceedance and data through a Bayesian network model to accurately identify the root cause. At the same time, it generates a tracing report containing evidence and repair suggestions and feeds it back to the scenario module to adjust parameters.

3. The video surveillance transmission delay detection system based on image recognition according to claim 2, characterized in that, The image acquisition and preprocessing module includes: The multi-source image acquisition unit supports the access of visible light cameras, infrared cameras, and industrial high-speed cameras to achieve the acquisition of multi-source monitoring images. It can also automatically identify the camera type and adapt the data format without manual configuration. An adaptive noise reduction unit is used to identify the type of image interference through CNN and perform targeted noise reduction based on the identified interference type. The resolution normalization unit is used to scale images to a uniform resolution for different camera resolutions, avoiding feature matching deviations caused by resolution differences.

4. The video surveillance transmission delay detection system based on image recognition according to claim 2, characterized in that, The spatiotemporal joint feature extraction module includes: The spatial texture feature extraction unit is used to extract stable texture points in the image using the SIFT algorithm. Each feature point generates a 128-dimensional descriptor to ensure the uniqueness of spatial features. The temporal frame change feature extraction unit is used to calculate the optical flow field of three consecutive frames of images using the Lucas-Kanade algorithm, capture the target motion trend, and generate temporal features of motion vector + frame difference. The feature fusion unit is used to weightedly fuse spatial texture features and temporal frame features through an attention mechanism to generate a spatiotemporal joint feature vector, thereby improving feature recognition.

5. The video surveillance transmission delay detection system based on image recognition according to claim 2, characterized in that, The algorithm dynamic iterative optimization module includes: The algorithm performance evaluation unit is used to collect the feature recognition rate of the spatiotemporal joint feature extraction module, the matching success rate and delay calculation error of the dynamic feature matching and delay calculation module in real time, so as to establish a three-dimensional evaluation model of feature quality-matching effect-error value. When a certain dimension indicator is lower than the threshold for 5 consecutive frames, the algorithm optimization process is triggered. The multi-algorithm candidate library unit is used to build multiple algorithm solutions adapted to different scenarios and store them in categories; The real-time algorithm switching unit is used to adjust the scene type of the module based on the results of the algorithm effect evaluation unit and the scene adaptive adjustment module, and to call the optimal algorithm from the candidate library.

6. The video surveillance transmission delay detection system based on image recognition according to claim 2, characterized in that, The dynamic feature matching and time delay calculation module includes: The dynamic template generation unit is used to package the spatiotemporal joint feature vector and the local clock of the transmission time into a feature template at the transmitting end; The cross-end feature matching unit is used to match the feature template received at the receiving end with the feature vector extracted locally in real time using the FLANN algorithm, and the matching threshold is adaptively adjusted. The dual-calibration delay calculation unit first performs basic delay calculation by subtracting the clock signal from the local clock signal at the time of reception to obtain the original delay. Then, clock offset calibration is performed by comparing the clock difference between the transmitting and receiving ends during the daily network idle period to generate a calibration coefficient and correct the original delay. After that, jitter compensation is performed by using a sliding window to calculate the average delay, eliminating the instantaneous deviation caused by network jitter, and finally outputting the actual calibrated delay.

7. The video surveillance transmission delay detection system based on image recognition according to claim 2, characterized in that, The edge-cloud collaborative computing module includes: The collaborative strategy decision-making unit is used to receive scene parameters from the scene adaptive adjustment module and the real-time computing power of the edge device, and dynamically allocate computing tasks. Edge computing acceleration unit for deploying lightweight algorithms on edge devices; The bandwidth dynamic allocation unit is used to monitor the edge-to-cloud transmission bandwidth in real time. When bandwidth is insufficient, it prioritizes the data transmission of high-priority scenarios and adopts an inter-frame compression + priority queuing strategy for low-priority scenario data to ensure that critical latency data is not lost.

8. A video surveillance transmission delay detection system based on image recognition according to claim 2, characterized in that, The scene adaptive adjustment module includes: The scene recognition unit is used to identify the type of monitored scene through a pre-trained CNN model; The detection parameter dynamic configuration unit is used to output configuration parameters according to the scene type; The resource scheduling unit is used to monitor the CPU and memory usage of the backend server in real time. If the CPU usage exceeds 80%, the detection frequency of non-critical scenarios will be temporarily reduced to prioritize the detection accuracy of traffic and industrial scenarios and avoid server lag.

9. A video surveillance transmission delay detection system based on image recognition according to claim 2, characterized in that, The alarm and data management module includes: The latency level determination unit is used to preset latency thresholds for various scenarios and classify the calibrated latency into three levels: normal, slightly exceeding the standard, and severely exceeding the standard. Slightly exceeding the standard triggers an early warning, and severely exceeding the standard triggers an emergency alarm. Multi-channel alarm unit, used to realize emergency alarms and early warnings; The encrypted storage and analysis unit is used to first encrypt and store latency data using AES-256, retaining 6 months of historical data; then, it generates a latency trend report every week to identify high-frequency cameras exceeding the standard and assist in operation and maintenance decisions.

10. A video surveillance transmission delay detection system based on image recognition according to claim 2, characterized in that, The multi-dimensional fault tracing module includes: The multi-source data acquisition unit is used to collect latency data, device status data, environmental data, and network data. The fault correlation analysis unit is used to establish the correlation between latency exceedance and multi-source data using a Bayesian network model. The source tracing report and remediation suggestion unit is used to generate a source tracing report that includes the root cause type, related data evidence, and scope of impact, and to push targeted remediation suggestions.

Images