An edge-computing-based video monitoring device running state diagnosis method and system

Abstract

This application discloses a method and system for diagnosing the operational status of video surveillance equipment based on edge computing, belonging to the field of image processing technology. The method includes: S100, setting a sampling time window, using the timestamp inherent in each image to calculate the frame drop rate, and issuing an image anomaly alert when the frame drop rate exceeds a smoothness threshold; S200, when the frame drop rate within the sampling time window is less than or equal to the smoothness threshold, acquiring the first grayscale distribution vector of the first sampled frame image and the second grayscale distribution vector of the last sampled frame image within the sampling time window; S300, calculating the relative deviation between the first and second grayscale distribution vectors, and issuing an image anomaly alert when the relative deviation exceeds a grayscale fluctuation threshold; S400, when the relative deviation is less than or equal to the grayscale fluctuation threshold, downsampling the last frame image and performing edge strength analysis using an improved Laplacian operator, and issuing an image anomaly alert when the edge strength is less than a sharpness threshold.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and system for diagnosing the operating status of video surveillance equipment based on edge computing. Background Technology

[0002] In industrial production, image recognition technology is increasingly being applied to manufacturing and quality inspection processes. For example, in aluminum foil production lines, some existing technologies utilize image recognition for defect detection. In such applications, malfunctions in video surveillance equipment can prevent the timely detection of defects in the aluminum foil, disrupting the entire production line's workflow; therefore, a rapid response is crucial.

[0003] In existing technologies, fault detection of monitoring equipment largely relies on regular manual inspections. This not only requires a significant investment of manpower but also suffers from severe delays in fault detection. Often, equipment malfunctions are only discovered after defects have left the production line or even resulted in batches of scrap, causing unnecessary economic losses for manufacturing companies. While some improved existing technologies incorporate simple disconnection alarm mechanisms, these only trigger when the equipment is completely disconnected. They cannot identify soft faults that do not completely interrupt the connection, such as color distortion, blurry focus, or sudden frame rate drops. Therefore, they still fail to address the need for early fault detection and handling, and cannot meet the demands of production scenarios like aluminum foil defect detection, which require high image acquisition stability. Summary of the Invention

[0004] This application is proposed to address the technical problem that existing video surveillance technologies cannot quickly identify soft faults such as blurriness, color distortion, and frame drops in devices. The purpose of this application is achieved through the following technical solution:

[0005] In a first aspect, embodiments of this application provide a method for diagnosing the operational status of video surveillance equipment based on edge computing, comprising the following steps:

[0006] S100. Set a sampling time window and use the timestamp of each image to count the frame drop rate within this sampling time window. When the frame drop rate is greater than the smoothness threshold, issue an image abnormality prompt.

[0007] S200. When the frame dropping rate within the sampling time window is less than or equal to the smoothness threshold, the first gray-level distribution vector of the first frame image sampled within the sampling time window and the second gray-level distribution vector of the last frame image are obtained respectively.

[0008] S300. Calculate the relative deviation between the first gray-level distribution vector and the second gray-level distribution vector. When the relative deviation is greater than the gray-level fluctuation threshold, issue an image abnormality prompt.

[0009] S400. When the relative deviation is less than or equal to the grayscale fluctuation threshold, the last frame image is downsampled, and edge intensity analysis is performed using the improved Laplacian operator. When the edge intensity is less than the sharpness threshold, an image abnormality prompt is issued.

[0010] In the above-mentioned method for diagnosing the operating status of video surveillance equipment based on edge computing, step S100 further includes: calculating the cumulative frame loss rate of multiple consecutive sampling time windows, and issuing an image anomaly prompt when the cumulative frame loss rate exceeds the cumulative threshold.

[0011] In the above-mentioned method for diagnosing the operating status of video surveillance equipment based on edge computing, in step S200, the grayscale distribution vector is generated by converting the RGB three-color channels into grayscale values ​​and then counting the number of pixels according to the grayscale level.

[0012] In the above-mentioned method for diagnosing the operational status of video surveillance equipment based on edge computing, step S300 further includes:

[0013] When the relative deviation does not exceed the grayscale fluctuation threshold but exceeds the warning threshold, the third grayscale distribution vector of the intermediate frame image within the sampling time window is obtained. If the relative deviation between the third grayscale distribution vector and the first grayscale distribution vector or the second grayscale distribution vector also exceeds the warning threshold, an image abnormality prompt is issued.

[0014] In the above-mentioned method for diagnosing the operating status of video surveillance equipment based on edge computing, the following operations are performed before or in parallel with step S300:

[0015] The average values ​​of the three RGB color channels of the first and last frame images within the sampling time window are obtained. The average value is compared with a standard color template learned in advance through normal images. When the average value of any channel deviates from the preset normal fluctuation range of the corresponding channel in the standard color template, an image abnormality prompt is issued.

[0016] In the above-mentioned method for diagnosing the operational status of video surveillance equipment based on edge computing, the edge strength analysis in step S400 includes:

[0017] The edge intensity of the last frame image is compared with the pre-stored reference edge intensity. When the edge intensity is lower than the proportional threshold of the reference edge intensity, an image anomaly warning is issued. And / or, the change in edge intensity of the last frame image compared with the edge intensity of the previous frame image is calculated. When the change in edge intensity is continuously negative within multiple consecutive sampling time windows and the cumulative decrease exceeds the cumulative change threshold, an image anomaly warning is issued.

[0018] In the above-mentioned method for diagnosing the operating status of video surveillance equipment based on edge computing, the reference edge strength is pre-established in the following way: when the video surveillance equipment is confirmed to be operating normally, at least one clear image is acquired, the edge strength of at least one key region in the clear image is extracted using an improved Laplacian operator, and it is stored as the reference edge strength.

[0019] In the above-mentioned method for diagnosing the operating status of a video surveillance device based on edge computing, the triggering method for steps S100-S400 is as follows: step S100 is executed at a first sampling frequency; when no image abnormality prompt is issued after a preset number of consecutive steps S100, steps S200 and S300 are triggered at a second sampling frequency lower than the first sampling frequency; when no image abnormality prompt is issued after a preset number of consecutive steps S200 and S300, step S400 is triggered at a third sampling frequency lower than the second sampling frequency.

[0020] Secondly, embodiments of this application provide a video surveillance equipment operation status diagnosis system based on edge computing, used to implement the above-mentioned video surveillance equipment operation status diagnosis method, including the following modules:

[0021] The first detection module is used to set a sampling time window, use the timestamps of the video image to count the frame drop rate within the sampling time window, and issue an image abnormality prompt when the frame drop rate is greater than the smoothness threshold.

[0022] The grayscale distribution acquisition module is used to acquire the first grayscale distribution vector of the first frame image and the second grayscale distribution vector of the last frame image within the sampling time window when the frame loss rate within the sampling time window is less than or equal to the smoothness threshold.

[0023] The grayscale fluctuation analysis module is used to calculate the relative deviation between the first grayscale distribution vector and the second grayscale distribution vector. When the relative deviation is greater than the grayscale fluctuation threshold, an image anomaly prompt is issued.

[0024] The second detection module is used to downsample the last frame image when the relative deviation is less than or equal to the grayscale fluctuation threshold, and to perform edge intensity analysis using the improved Laplacian operator. When the edge intensity is less than the sharpness threshold, an image abnormality prompt is issued.

[0025] In the aforementioned edge computing-based video surveillance equipment operation status diagnosis system, the system further includes: a hierarchical scheduling module, used to trigger the first detection module to run at a first sampling frequency; when the first detection module fails to issue an image abnormality prompt for a preset number of consecutive times, the grayscale distribution acquisition module and the grayscale fluctuation analysis module are triggered to run at a second sampling frequency lower than the first sampling frequency; when the grayscale fluctuation analysis module fails to issue an image abnormality prompt for a preset number of consecutive times, the second detection module is triggered to run at a third sampling frequency lower than the second sampling frequency.

[0026] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0027] This application utilizes multi-level image feature detection to accurately identify soft faults in monitoring equipment, such as frame drops, color casts, overexposure, underexposure, blurring, and occlusion, overcoming the blind spot of existing technologies that can only detect connection interruptions. Leveraging edge computing, it achieves lightweight local diagnostics and employs a hierarchical scheduling mechanism to allocate computing power on demand, significantly reducing resource consumption at the edge. Through sliding time windows, grayscale distribution comparison, and improved Laplacian edge analysis, it achieves millisecond-level fault response and progressive early warning of faults, significantly improving detection real-time performance and accuracy. It can replace regular manual inspections, saving on manpower costs and preventing missed production defects and batch scrap losses caused by equipment soft faults, effectively improving the operational stability and maintenance efficiency of industrial vision monitoring systems. Attached Figure Description

[0028] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof.

[0029] Figure 1 This is a schematic diagram illustrating the steps of a video surveillance equipment operation status diagnosis method based on edge computing provided in an embodiment of this application;

[0030] Figure 2 This is a schematic diagram of a video surveillance equipment operation status diagnosis system based on edge computing provided in an embodiment of this application. Detailed Implementation

[0031] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only a part of the embodiments of this application, and this application is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] This application takes an industrial visual monitoring scenario of a continuous aluminum foil production line as an example. The edge computing hardware used is an edge computing gateway equipped with an ARM Cortex-A72 architecture, a main frequency of 1.8GHz, 2GB of memory and 8GB of eMMC storage. The connected video monitoring equipment is an industrial area scan camera with a resolution of 1920×1080, a frame rate of 25fps and progressive scan. The camera is directly connected to the edge computing gateway via Gigabit Ethernet. The image data is transmitted to the edge in real time via the RTSP protocol for local diagnosis. The diagnosis results are uploaded to the cloud management platform via the MQTT protocol to realize real-time fault reporting and remote operation and maintenance.

[0034] Example 1

[0035] This embodiment provides a video surveillance equipment operation status diagnosis method that can operate lightweightly at the edge and comprehensively identify soft faults such as frame loss, color cast, overexposure / underexposure, blurring, and occlusion. It is applicable to continuous material production scenarios with extremely high image stability requirements, such as aluminum foil production lines, thin film production lines, and photovoltaic glass inspection. Figure 1 As shown, the diagnostic method includes steps S100-S400.

[0036] In step S100, a sampling time window is set, and the frame dropping rate within this sampling time window is calculated using the timestamp of each image. When the frame dropping rate is greater than the smoothness threshold, an image abnormality prompt is issued.

[0037] Specifically, the inventors discovered that existing monitoring systems only trigger alarms when devices are completely disconnected, failing to identify soft faults such as intermittent frame drops and sudden frame rate drops caused by network jitter, insufficient bandwidth, or driver anomalies. Furthermore, frame loss in continuous production lines directly leads to missed defects and batch scrap. Therefore, it is necessary to verify frame stream continuity in real time with extremely lightweight computation, prioritizing the interception of sudden frame anomalies to avoid unnecessary computational waste. To address this issue, this application employs a fixed-duration, fixed-step sliding overlapping time window mechanism. The time window length T is set to 1 second, adapted to a camera frame rate of 25fps, ensuring a fixed theoretical number of frames within each time window, facilitating frame drop rate calculation. The sliding step size... The time window is set to 50ms, meaning that every 50ms, the time window slides forward once, creating a new window position. Since the step size (50ms) is much smaller than the window length (1000ms), there is a significant overlap between adjacent time windows (overlap rate of 95%). This ensures the real-time performance (outputting a diagnostic result every 50ms) and smoothness of fault detection, avoiding detection blind spots caused by window jumps. The edge computing gateway extracts the UTC timestamp of each frame from the video stream's RTP encapsulation header or the private data area of ​​the image frame, accurate to the millisecond level. At the current sampling time window position (i.e., the time range of T hours backwards from the current moment), the number of valid frames actually received in that interval is counted. Next, the frame drop rate is calculated for the theoretical frame rate. = Camera rated frame rate × Time window length, in this embodiment = 25 × 1 = 25 frames; frame drop rate The following formula is used for calculation:

[0038] ;

[0039] In the above formula, This represents the frame drop rate within the current sampling time window. This represents the theoretical number of frames that should be received within the time window. The actual number of valid frames received within the time window; the smoothness threshold. The threshold is set at 8% (this value is determined statistically based on standard experimental conditions (camera 25fps, resolution 1080p, normal illumination of 200 lux on the aluminum foil production line). The calibration process is as follows: collect 10 hours of video stream under normal equipment operation, and statistically analyze the frame drop rate distribution per second; take the mean + 3 times the standard deviation as the anomaly threshold, and calculate 8%; for different frame rates f, the theoretical number of frames = f × T, the frame drop rate threshold remains unchanged (8%), but the absolute frame drop number threshold needs to be proportionally converted); compare the frame drop rate within the current sampling time window. With smoothness threshold ,when > When the frame smoothness is abnormal, the edge device generates an abnormal prompt message, including the start and end time of the time window, the frame drop rate, the theoretical number of frames, and the actual number of frames. The alarm is synchronized with the cloud platform through local indicator lights.

[0040] The step S100 further includes: calculating the cumulative frame loss rate of multiple consecutive sampling time windows, and issuing an image anomaly prompt when the cumulative frame loss rate exceeds the cumulative threshold.

[0041] To avoid false alarms due to momentary jitter and to capture progressive frame drop degradation, multiple consecutive sampling time windows (e.g., 3) are set as cumulative statistical periods to calculate the cumulative frame drop rate. The formula is as follows:

[0042] ;

[0043] In the formula, The cumulative frame drop rate over three consecutive sampling time windows; Let be the theoretical number of frames for the i-th time window. Let i be the number of actually received frames in the i-th time window. For summation operations;

[0044] Cumulative threshold Set to 5% (the cumulative frame drop rate over three consecutive time windows, calculated as the mean plus three times the standard deviation); when > When a persistent frame drop fault is detected, a Level 2 smoothness anomaly alert is issued, which differs from the Level 1 smoothness anomaly alert for instantaneous frame drops within a single time window, making it easier for operations and maintenance personnel to distinguish the severity of the fault. Through the above methods, only timestamp-based statistical calculations are required, with computing power consumption of less than 5%, meeting the lightweight requirements of edge devices. Millisecond-level response can detect frame drops caused by network congestion, driver anomalies, and camera hardware failures within 1 second. The combination of single-window judgment and cumulative judgment ensures that instantaneous faults are not falsely reported and progressive faults are not missed. After an anomaly is triggered, subsequent complex image calculations are blocked, significantly reducing the unnecessary computing power consumption at the edge.

[0045] In step S200, when the frame drop rate within the sampling time window is less than or equal to the smoothness threshold, the first gray-level distribution vector of the first frame image sampled within the sampling time window and the second gray-level distribution vector of the last frame image are obtained respectively.

[0046] The grayscale distribution vector is generated by converting the RGB three color channels into grayscale values ​​and then counting the number of pixels according to the grayscale level.

[0047] Specifically, since simple frame drop detection cannot cover image content anomalies such as overexposure, underexposure, large-area occlusion, and color cast; industrial scene backgrounds are fixed and materials are continuous, the grayscale distribution of normal images is highly stable, and sudden changes in grayscale distribution directly correspond to image anomalies; at the same time, comparing the first and last frames can eliminate instantaneous noise interference and improve the robustness of judgment; this application makes the following processing: if the frame drop rate within the current sampling time window is... ≤ Smoothness threshold And the cumulative frame drop rate over three consecutive sampling time windows ≤Cumulative threshold At that time, extract the first frame image within the current sampling time window. With the last frame image To reduce computational load and focus on the effective area, an 80%×80% area from the center of the image was selected as the analysis region to eliminate border interference. The RGB three channels were converted to grayscale values ​​using an industrial vision standard grayscale conversion formula, as follows:

[0048] ;

[0049] In this model, R, G, and B all range from 0 to 255, and Gray also ranges from 0 to 255. The grayscale values ​​from 0 to 255 are then evenly divided into 32 grayscale levels, with each level containing 8 grayscale values. The number of pixels at each grayscale level within each region is statistically analyzed to form a 32-dimensional grayscale distribution vector. : ;in, The vector represents the total number of pixels within the i-th gray level; the vector is normalized to eliminate the effect of resolution.

[0050] ;

[0051] Finally, the normalized grayscale distribution vector of the first frame image is obtained. (First grayscale distribution vector), normalized grayscale distribution vector of the last frame image (Second grayscale distribution vector); The grayscale distribution integrates RGB three-channel information, reflecting both brightness and color distribution, and can cover various anomalies such as overexposure, underexposure, occlusion, and color cast; The 32-dimensional vector balances accuracy and computational load, and can be generated in real time at the edge; The comparison between the first and last frames eliminates single-frame noise and instantaneous light interference, improving the stability of anomaly judgment;

[0052] Next, in step S300, the relative deviation between the first gray-level distribution vector and the second gray-level distribution vector is calculated. When the relative deviation is greater than the gray-level fluctuation threshold, an image abnormality prompt is issued.

[0053] Because the grayscale distribution fluctuation is minimal in a normal scene, lens contamination, occlusion, lighting malfunctions, and camera parameter drift can cause significant shifts in the grayscale distribution. Quantifying the degree of fluctuation through relative deviation allows for stable identification of soft faults in the image content. In this step, the relative deviation is first calculated, using cosine distance to calculate the normalized grayscale distribution vector of the first frame image. With the normalized grayscale distribution vector of the last frame image relative deviation The formula is as follows:

[0054] ;

[0055] In the formula, It is the dot product of two vectors; , Let L2 norms be the L2 norms of the two vectors, respectively. The value represents the relative deviation, ranging from 0 to 1. A larger value indicates a greater difference in grayscale distribution.

[0056] Grayscale fluctuation threshold Set to 0.12 (collect 1000 sets of grayscale distribution vectors of the first and last frames under normal conditions, calculate the cosine distance, and obtain the deviation sequence; take the 95th percentile of this sequence as the threshold); when the relative deviation > Grayscale fluctuation threshold When this occurs, it is determined that the grayscale distribution is abnormal, and an image abnormality warning is issued;

[0057] Step S300 further includes:

[0058] When the relative deviation does not exceed the grayscale fluctuation threshold but exceeds the warning threshold, the third grayscale distribution vector of the intermediate frame image within the sampling time window is obtained. If the relative deviation between the third grayscale distribution vector and the first grayscale distribution vector or the second grayscale distribution vector also exceeds the warning threshold, an image abnormality prompt is issued.

[0059] Specifically, set early warning thresholds. =0.07 (obtained by taking the 75th percentile of the deviation sequence); when the warning threshold Relative Deviation ≤ Grayscale fluctuation threshold At that time, extract the intermediate frames of the time window. Generate the third grayscale distribution vector ; calculate separately and , and The relative deviation, if any deviation > If this is detected, it is determined to be an anomaly in the gradual grayscale, and an early warning is issued to enable early detection of the fault.

[0060] Specifically, the following operations are performed before or in parallel with step S300:

[0061] The average values ​​of the three RGB color channels of the first and last frame images within the sampling time window are obtained. The average value is compared with a standard color template learned in advance through normal images. When the average value of any channel deviates from the preset normal fluctuation range of the corresponding channel in the standard color template, an image abnormality prompt is issued.

[0062] Specifically, the RGB channel color cast detection is performed as follows: RGB three-channel mean detection is performed in parallel: the first frame image within the current sampling time window is calculated respectively. With the last frame image Mean values ​​of R, G, and B channels , , Compared with the pre-stored standard color template, the template is learned from images obtained during 10 minutes of continuous operation of the device. Each channel is set with ±10 as the preset normal fluctuation range. If the mean value of any channel exceeds the range, it is judged as a color cast fault and an abnormality prompt is issued. The cosine distance is not affected by the vector amplitude, is not sensitive to the overall brightness shift, but is sensitive to changes in the distribution structure, making it suitable for fixed industrial scenarios. A three-level judgment mechanism (normal → warning → abnormal) realizes graded fault response. Grayscale distribution + RGB channel dual-dimensional detection provides full coverage of color cast, overexposure, underexposure, and occlusion, with no detection blind spots.

[0063] Finally, in step S400, when the relative deviation is less than or equal to the grayscale fluctuation threshold, the last frame image is downsampled, and edge intensity analysis is performed using the improved Laplacian operator. When the edge intensity is less than the sharpness threshold, an image abnormality prompt is issued.

[0064] Since frame dropping, grayscale, and color cast detection cannot identify edge detail loss faults such as focus blur, lens fogging, and slow defocusing; these faults do not cause connection drops, obvious frame drops, or significant color casts, but they directly lead to defect detection failure; in edge computing or industrial vision, the mainstream approach to blur detection is to analyze high-frequency information (edges and details) in the image; when the image is blurry, this high-frequency information is significantly reduced; the Laplacian variance method is currently the most suitable method for edge computing because it has extremely low computational cost and stable performance. The principle is: the Laplacian operator is a second-order differential operator specifically used to extract edges in an image; sharp image edges are sharp and have large response value fluctuations (high variance); blurry image edges are smooth and have small response value fluctuations (low variance); this method is often used in the autofocus system of digital cameras, which uses the Sobel operator to extract gradients in the horizontal and vertical directions; edge strength is the core indicator for judging sharpness, and the improved Laplacian operator is suitable for lightweight computation at the edge;

[0065] In this step, when the relative deviation ≤ Grayscale fluctuation threshold At that time, for the last frame A 4x downsampling process reduces the resolution from 1920×1080 to 480×270, preserving edge features while reducing computational load by 75%, thus adapting to edge computing power. It should be noted that although downsampling reduces high-frequency details, for the industrial continuous material monitoring scenario (such as aluminum foil surface) targeted in this application, the texture features exhibit scale invariance; that is, the distinction between sharp and blurry images remains at low resolution. Specifically, according to the Nyquist sampling theorem, image blurring is essentially the attenuation of high-frequency components. After downsampling, the effective bandwidth decreases, but the ratio of the variance of the Laplacian response to the sharp image to the blurry image remains essentially unchanged after resolution reduction (experiments show that the variance ratio at 480×270 is correlated with the ratio at 1080p by a coefficient > 0.95). Therefore, using downsampled images for edge intensity analysis can relatively determine sharpness while significantly reducing computational load. The improved Laplacian operator introduces a direction weight factor based on the standard 8-neighborhood Laplacian kernel to enhance the edge response to specific industrial textures. The kernel matrix is ​​as follows:

[0066] ;

[0067] Here, a and b are directional weighting coefficients used to enhance the edge response to specific industrial textures (such as aluminum foil rolling textures, unidirectional wire drawing textures, etc.); the kernel matrix naturally satisfies the normalization condition that the sum of its elements is zero, regardless of the specific values ​​of a and b. This property ensures that the kernel's response is zero in flat areas of the image (where pixel values ​​do not change), avoiding false edge responses; when a=1 and b=1, the above matrix degenerates into a standard eight-neighbor Laplacian kernel:

[0068] ;

[0069] In this embodiment, to further adapt to edge computing power, a=b=0.25 is chosen, that is, the coefficients of the standard kernel are scaled by 1 / 4, reducing the dynamic range of the convolution operation without changing the relative distribution of edge intensity. The kernel matrix of the improved Laplacian operator described above also satisfies anisotropic or isotropic configurability: by adjusting the ratio of weight coefficients a and b, this kernel matrix can flexibly adapt to different industrial texture detection scenarios.

[0070] When configured as a = b, the kernel matrix exhibits isotropic properties and has a uniform response to edges in all directions in the image, making it suitable for detecting surface defects of continuous materials with uniform texture distribution, such as aluminum foil.

[0071] When configured as a ≠ b, the kernel matrix exhibits anisotropy, which can enhance the edge response sensitivity of textures in specific directions. For example, for a unidirectional brushed metal surface, a=0.5 and b=0 can be set. At this time, the kernel matrix only detects the edge gradients in the horizontal and vertical directions, effectively suppressing noise interference in the diagonal direction and improving the blur detection accuracy in specific texture scenes.

[0072] This application does not limit the specific values ​​of a and b, as long as the normalization condition that the sum of the elements of the kernel matrix is ​​zero is met. The specific values ​​can be adaptively set by those skilled in the art based on the computing power characteristics of the edge device and the texture features of the scene.

[0073] Next, edge strength calculation is performed: the downsampled grayscale image is convolved with the improved Laplacian kernel described above to obtain the Laplacian response map; the variance of the response map is calculated as the edge strength. Clear images exhibit large fluctuations and high variance in edge response; blurry images show smooth edges and low variance.

[0074] The edge strength analysis in step S400 includes:

[0075] The edge intensity of the last frame image is compared with the pre-stored reference edge intensity. When the edge intensity is lower than the proportional threshold of the reference edge intensity, an image anomaly warning is issued. And / or, the change in edge intensity of the last frame image compared with the edge intensity of the previous frame image is calculated. When the change in edge intensity is continuously negative within multiple consecutive sampling time windows and the cumulative decrease exceeds the cumulative change threshold, an image anomaly warning is issued.

[0076] Edge strength analysis can perform a two-dimensional judgment. The first dimension is a benchmark comparison: a clear image is acquired when the equipment is in normal working order, and the benchmark edge strength is calculated. , proportion threshold The value is 0.65 (by artificially creating test images with varying degrees of blur (defocus, motion blur) and comparing their edge strength with a normal image. The ratio corresponding to the critical point where blur is detected (unacceptable to the human eye) is taken; the empirical value is 0.65). When the sharpness threshold is less than the specified threshold, the image is considered blurry. The sharpness threshold is calibrated by the downsampling ratio. In this embodiment, for example, the original reference edge intensity is 1200 at 1080P. After downsampling by 4 times, the reference edge intensity... The measured strength is the original reference edge strength. (Base value obtained after downsampling through actual calibration), i.e., 75, then the sharpness threshold after downsampling is 75 × 0.65 ≈ 49; The second dimension is to determine the trend: calculated over 5 consecutive time windows. Change ,like Continuously negative and cumulative decrease >Cumulative change threshold (The 95th percentile of the cumulative decrease in edge intensity over 5 consecutive time windows) determines progressive ambiguity and issues an anomaly warning.

[0077] The reference edge strength is pre-established in the following way: when the video surveillance equipment is confirmed to be operating normally, at least one clear image is acquired, the edge strength of at least one key region in the clear image is extracted using the improved Laplacian operator, and it is stored as the reference edge strength.

[0078] After the equipment is installed and debugged, automatically or manually delineate 3 to 5 rectangular regions in the image. These regions should contain rich textures / edges (such as aluminum foil edges or roll boundaries) and avoid large flat areas. In this embodiment, a fixed 200×200 pixel region at the center of the image and 100×100 pixel regions near each of the four corners are selected. Under standard lighting, correct focus, and a clean lens, continuously acquire multiple frames (e.g., 30 frames) of clear images. For each frame, downsample each key region by 4 times, and then use the improved Laplacian operator to calculate the edge intensity (response variance). Take the mean of all frames and all regions as the baseline edge intensity. ,Will The coordinates of key areas are stored in the local configuration file of the edge device. If the device installation location, lighting, or lens undergoes permanent changes, the above calibration process must be re-executed. The improved Laplacian operator has low computational cost and strong robustness, and can run in real time at the edge; downsampling + key area analysis consumes less than 10% of computing power; benchmark comparison + trend judgment can identify both instantaneous and progressive blur; effectively solves the problem of detecting soft faults caused by lens fogging, focus drift, and slight vibration.

[0079] The triggering method for steps S100-S400 is as follows: step S100 is executed at a first sampling frequency; when no image abnormality prompt is issued after a preset number of consecutive steps S100, steps S200 and S300 are triggered at a second sampling frequency lower than the first sampling frequency; when no image abnormality prompt is issued after a preset number of consecutive steps S200 and S300, step S400 is triggered at a third sampling frequency lower than the second sampling frequency.

[0080] Since edge devices have limited computing power, performing all detections at high frequencies would waste computing power and cause system lag. This application uses a fault-based hierarchical scheduling approach, performing high-frequency detection for sudden faults and low-frequency detection for gradual faults, minimizing computing power consumption while ensuring detection coverage. Specifically, the first sampling frequency... =20Hz (every 50ms / time), high-frequency execution of frame loss detection in step S100 ensures real-time response to instantaneous faults; if no abnormality is found in step S100 for several consecutive times (e.g., 10 times), switch to the second sampling frequency. =1Hz (once per second), perform fast grayscale and color cast detection in steps S200 and S300; if no abnormality is found in steps S200 and S300 for multiple consecutive times (e.g., 30 times), switch to the third sampling frequency. =0.2Hz (once every 5 seconds), perform edge intensity depth inspection in step S400; if an anomaly is detected in any step, immediately revert to High-frequency detection ensures continuous fault tracking. Through a hierarchical scheduling trigger mechanism, the average computing power consumption during normal operation is less than 15%, and edge devices can process multiple video streams simultaneously; hierarchical scheduling achieves the optimal balance between real-time performance and computing power efficiency; and an automatic rollback mechanism ensures that no fault is missed and no response is delayed.

[0081] It should be noted that although adjacent sliding windows highly overlap, steps S200 and subsequent steps in this application are not executed in every sliding window. Instead, they are triggered at a lower frequency (the second sampling frequency of 1Hz) after the frame drop rate detection (S100) has failed to trigger an anomaly multiple times. That is, grayscale contrast and edge analysis are only performed on a portion of the windows (e.g., taking a non-overlapping window every 1 second), thereby avoiding redundant calculations and invalid contrasts caused by window overlap. Therefore, the "current sampling time window" in step S100 and the "sampling time window" in step S200 can be the same physical window, but in terms of time scheduling, the sampling frequency of steps S200 / S300 / S400 is much lower than that of S100. The actual analysis window is equivalent to equally spaced non-overlapping sampling. This design utilizes the high real-time performance of sliding windows for frame drop detection while avoiding interference from overlapping windows on grayscale contrast.

[0082] Example 2

[0083] This embodiment provides a video surveillance equipment operation status diagnosis system based on edge computing, used to implement any step of the method described in Embodiment 1. The system adopts a modular architecture, is deployed on an edge computing gateway, and is adapted to the parallel diagnosis needs of multiple monitoring devices in industrial sites. Figure 2 As shown, the system includes: a first detection module 100, a grayscale distribution acquisition module 200, a grayscale fluctuation analysis module 300, a second detection module 400, and a hierarchical scheduling module 500. The system may also include an anomaly reporting module 600; the functions of each module are as follows:

[0084] The first detection module 100 is used to perform frame drop rate detection, configure a sliding sampling time window, extract image timestamps, and calculate the frame drop rate of a single sampling time window and the cumulative frame drop rate of multiple sampling time windows. The inputs are the original frames and timestamps of the video stream; the outputs are frame smoothness judgment results and anomaly flags. The timestamp parsing unit extracts millisecond-level timestamps from the RTP header of the RTSP stream; the frame count unit updates the sliding window in 50ms steps and counts the actual number of frames; the frame drop rate calculation unit calculates the frame drop rate within the current sampling time window according to the formula in Example 1. Cumulative frame drop rate over three consecutive sampling time windows The threshold judgment unit will , With smoothness threshold Cumulative threshold Compare and output an abnormal signal;

[0085] The grayscale distribution acquisition module 200 is used to generate grayscale distribution vectors. When frame dropping is normal, it extracts the first and last frame images within the time window, converts them to grayscale, calculates and normalizes the 32-level grayscale distribution; the output is the normalized grayscale distribution vector of the first frame image. With the normalized grayscale distribution vector of the last frame image The color conversion unit performs RGB to grayscale conversion; the grayscale statistics unit counts pixels according to 32 levels; and the normalization unit generates a standardized distribution vector.

[0086] The grayscale fluctuation analysis module 300 is used to perform relative deviation calculation and anomaly detection, calculate the vector cosine distance, and determine grayscale fluctuations; it also detects RGB channel color cast in parallel; the input is the normalized grayscale distribution vector of the first frame image. With the normalized grayscale distribution vector of the last frame image The output includes grayscale distribution anomaly flags and color cast anomaly flags; the deviation calculation unit calculates the normalized grayscale distribution vector of the first frame image. With the normalized grayscale distribution vector of the last frame image relative deviation Multi-level threshold units enable normal / warning / abnormal judgment; color cast detection unit reads the standard color template, compares the channel average, and then determines whether there is an abnormality;

[0087] The second detection module 400 is used to perform edge strength analysis (downsampling, improved Laplacian convolution, edge strength calculation, and two-dimensional blur detection); output a sharpness anomaly flag; the downsampling unit reduces the image to 1 / 4 size; the edge extraction unit uses an 8-neighbor kernel for convolution; and the intensity calculation unit calculates the response variance as the edge strength. The benchmark comparison unit will determine the edge strength. With reference edge strength By comparison, the trend unit indicates a continuous decline;

[0088] The hierarchical scheduling module 500 schedules the operation of each module according to three levels of frequency, and automatically reverts to a higher frequency in case of an anomaly; control logic: based on the first sampling frequency. =20Hz scheduling for the first detection module 100 to run; the first detection module 100 detects no abnormalities for 10 consecutive times → then at the second sampling frequency =1Hz scheduling grayscale distribution acquisition module 200 and grayscale fluctuation analysis module 300 run; grayscale distribution acquisition module 200 and grayscale fluctuation analysis module 300 detect no abnormalities for 30 consecutive times → use the third sampling frequency The second detection module 400 is scheduled to run at 0.2Hz; if any abnormality is detected, the frequency is immediately switched back to the first sampling frequency. Schedule the first detection module 100 to run.

[0089] The system may also include an anomaly reporting module 600, which provides local audio-visual alerts and cloud-based MQTT reporting, including fault type, time, parameters, and device ID; outputting: Level 1 / Level 2 / Early Warning Level 3 alarm information.

[0090] Those skilled in the art will understand that the specific functions and operations of each module and each unit in the system of this embodiment have been described in detail in the above embodiment 1, which describes a method for diagnosing the operating status of a video surveillance device based on edge computing. Therefore, the repeated description will be omitted here.

[0091] The basic principles of this application have been described above with reference to specific embodiments. It should be understood that the specific details disclosed above are for illustrative and illustrative purposes only, and are not intended to limit the scope of protection of this invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for diagnosing the operational status of video surveillance equipment based on edge computing, characterized in that, Includes the following steps: S100. Set a sampling time window and use the timestamp of each image to count the frame drop rate within this sampling time window. When the frame drop rate is greater than the smoothness threshold, issue an image abnormality prompt. S200. When the frame dropping rate within the sampling time window is less than or equal to the smoothness threshold, the first gray-level distribution vector of the first frame image sampled within the sampling time window and the second gray-level distribution vector of the last frame image are obtained respectively. S300. Calculate the relative deviation between the first gray-level distribution vector and the second gray-level distribution vector. When the relative deviation is greater than the gray-level fluctuation threshold, issue an image abnormality prompt. S400. When the relative deviation is less than or equal to the grayscale fluctuation threshold, the last frame image is downsampled, and edge intensity analysis is performed using the improved Laplacian operator. When the edge intensity is less than the sharpness threshold, an image abnormality prompt is issued.

2. The method according to claim 1, characterized in that, Step S100 further includes: calculating the cumulative frame loss rate of multiple consecutive sampling time windows, and issuing an image anomaly prompt when the cumulative frame loss rate exceeds the cumulative threshold.

3. The method according to claim 1, characterized in that, In step S200, the grayscale distribution vector is generated by converting the RGB three color channels into grayscale values ​​and then counting the number of pixels according to the grayscale level.

4. The method according to claim 1, characterized in that, Step S300 further includes: When the relative deviation does not exceed the grayscale fluctuation threshold but exceeds the warning threshold, the third grayscale distribution vector of the intermediate frame image within the sampling time window is obtained. If the relative deviation between the third grayscale distribution vector and the first grayscale distribution vector or the second grayscale distribution vector also exceeds the warning threshold, an image abnormality prompt is issued.

5. The method according to claim 1, characterized in that, The following operations are performed before or in parallel with step S300: The average values ​​of the three RGB color channels of the first and last frame images within the sampling time window are obtained. The average value is compared with a standard color template learned in advance through normal images. When the average value of any channel deviates from the preset normal fluctuation range of the corresponding channel in the standard color template, an image abnormality prompt is issued.

6. The method according to claim 1, characterized in that, The edge strength analysis in step S400 includes: The edge intensity of the last frame image is compared with the pre-stored reference edge intensity. When the edge intensity is lower than the proportional threshold of the reference edge intensity, an image anomaly warning is issued. And / or, the change in edge intensity of the last frame image compared with the edge intensity of the previous frame image is calculated. When the change in edge intensity is continuously negative within multiple consecutive sampling time windows and the cumulative decrease exceeds the cumulative change threshold, an image anomaly warning is issued.

7. The method according to claim 6, characterized in that, The baseline edge strength is pre-established in the following way: when the video surveillance equipment is confirmed to be operating normally, at least one clear image is acquired, the edge strength of at least one key region in the clear image is extracted using the improved Laplacian operator, and it is stored as the baseline edge strength.

8. The method according to claim 1, characterized in that, The triggering method for steps S100-S400 is as follows: Step S100 is executed at a first sampling frequency; when no image abnormality prompt is issued for a preset number of consecutive steps S100, steps S200 and S300 are triggered to be executed at a second sampling frequency lower than the first sampling frequency; when no image abnormality prompt is issued for a preset number of consecutive steps S200 and S300, step S400 is triggered to be executed at a third sampling frequency lower than the second sampling frequency.

9. A video surveillance equipment operation status diagnosis system based on edge computing, used to implement the method as described in any one of claims 1-8, characterized in that, Includes the following modules: The first detection module is used to set a sampling time window, use the timestamps of the video image to count the frame drop rate within the sampling time window, and issue an image abnormality prompt when the frame drop rate is greater than the smoothness threshold. The grayscale distribution acquisition module is used to acquire the first grayscale distribution vector of the first frame image and the second grayscale distribution vector of the last frame image within the sampling time window when the frame loss rate within the sampling time window is less than or equal to the smoothness threshold. The grayscale fluctuation analysis module is used to calculate the relative deviation between the first grayscale distribution vector and the second grayscale distribution vector. When the relative deviation is greater than the grayscale fluctuation threshold, an image anomaly prompt is issued. The second detection module is used to downsample the last frame image when the relative deviation is less than or equal to the grayscale fluctuation threshold, and to perform edge intensity analysis using the improved Laplacian operator. When the edge intensity is less than the sharpness threshold, an image abnormality prompt is issued.

10. The system according to claim 9, characterized in that, The system further includes: a hierarchical scheduling module, used to trigger the first detection module to run at a first sampling frequency; when the first detection module does not issue an image abnormality prompt for a preset number of consecutive times, it triggers the grayscale distribution acquisition module and the grayscale fluctuation analysis module to run at a second sampling frequency lower than the first sampling frequency; when the grayscale fluctuation analysis module does not issue an image abnormality prompt for a preset number of consecutive times, it triggers the second detection module to run at a third sampling frequency lower than the second sampling frequency.

Images