Intelligent monitoring method and device for video monitoring system based on cloud edge collaboration

Abstract

The invention discloses an intelligent monitoring method and device for a video monitoring system based on cloud edge collaboration, and relates to a video monitoring technology. An edge computing platform is deployed on one side of the video monitoring system, so that the computing load of a central cloud platform is reduced; through combination of video fault data, historical operation data and real-time operation data, a correlation index between a picture parameter and a video fault type is calculated so as to accurately determine a video picture fault reason. Constructing a panoramic canvas through equipment parameters, and constructing a panoramic background model based on historical operation data; the real-time operation data and the panoramic background model are combined to continuously monitor faults of the holder and the zoom lens, and monitoring blind areas or picture loss caused by faults of the video monitoring system are avoided; the fault evaluation platform receives the detected suspected fault data and suspected fault indexes and feeds back the suspected fault data and suspected fault indexes to the fault alarm system in time, corresponding processing measures are taken, and the reliability of the monitoring camera system can be ensured.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and specifically to an intelligent monitoring method and device for a video surveillance system based on cloud-edge collaboration. Background Art

[0002] At present, with the rapid development of video surveillance system technology, video surveillance systems have been used in more and more places, and with the development of network technology, the country's investment in security and the participation of operators, many large-scale video surveillance systems have been established, and the monitoring network can cover a city, a province, or even a country; in the video surveillance system, more and more various extended functions are added, connecting multiple systems together, realizing some new functions and expanding the application scope of the original system.

[0003] Existing video surveillance systems mainly rely on manual troubleshooting to identify faults in video images. This approach is not only inefficient, but also easily affected by human factors, making it difficult to ensure the accuracy of video image fault judgment, and to a certain extent increases the difficulty and cost of fault handling. In the video surveillance system, the pan / tilt of the surveillance camera can automatically identify and track moving targets based on the video analysis algorithm, but the video surveillance system lacks real-time monitoring capabilities for the operating status of the pan / tilt and zoom lens. When the pan / tilt or zoom lens fails (such as stuck, unable to rotate, abnormal zoom, etc.), manual monitoring cannot promptly judge the operating status of the pan / tilt and zoom lens, and cannot promptly discover potential faults, which may result in monitoring blind spots or image loss, affecting the monitoring effect.

[0004] In order to solve this problem, we propose an intelligent monitoring method and device for video surveillance system based on cloud-edge collaboration. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a method and device for intelligent monitoring of a video surveillance system based on cloud-edge collaboration, which can effectively solve the problems in the above-mentioned background technology.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] An intelligent monitoring method for a video surveillance system based on cloud-edge collaboration, comprising:

[0008] Acquire online device information of the monitoring device system, and based on the online device information, record multiple monitoring cameras connected to the same central switch as the same monitoring camera group;

[0009] Construct multiple edge computing platforms, wherein the edge computing platforms obtain video fault data of the monitoring camera group, and obtain correlation indexes and related influencing parameters between picture parameters and video fault types based on the video fault data;

[0010] The edge computing platform obtains the historical operation data and real-time operation data of the corresponding monitoring camera group from the monitoring device system, calculates the average value of the picture parameters of each monitoring camera in different time periods based on the historical operation data, obtains the real-time picture parameters based on the real-time operation data, and calculates the difference index of each real-time picture parameter based on the real-time picture parameters and the picture parameter average;

[0011] Construct a fault assessment model, and calculate the abnormality index of each video fault type in the real-time video stream data based on the real-time picture parameters, difference index, correlation index and related influencing parameters of each video fault type. If the abnormality index is lower than the preset degree threshold, the abnormality index is recorded as 0; otherwise, the abnormality index is used as a suspected fault index, and the video fault type corresponding to the abnormality index is recorded as a suspected fault type and uploaded to the fault assessment model;

[0012] The edge computing platform obtains the device parameters of each surveillance camera in the corresponding surveillance camera group and creates a blank panoramic canvas based on the device parameters;

[0013] Obtain historical video stream data and historical device status data of surveillance cameras in different time periods in the past N unit cycles from historical operation data, and build a panoramic background model of surveillance cameras;

[0014] Obtain the real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle in the real-time operation data;

[0015] Extracting a first background image on the panoramic background model based on the real-time device status data, and extracting a second background image on the panoramic background model based on the real-time video stream data;

[0016] Analyze and process the first background image and the second background image to obtain a comprehensive position offset index and a comprehensive size deviation index, and detect whether there is a fault in the pan / tilt operation and the zoom lens; if there is a fault, generate a suspected fault type and a suspected fault index, and upload them to the fault assessment model;

[0017] The suspected fault index and suspected fault type are input into the fault assessment model to calculate the fault assessment coefficient of each surveillance camera in the current unit cycle, and feedback is given to the fault alarm system based on the fault assessment coefficient, and corresponding processing measures are taken.

[0018] Preferably, the obtaining of online device information of the monitoring device system further includes:

[0019] Use a network scanning tool to scan the network link of the video surveillance system to obtain the network topology structure of the network link;

[0020] Identify all online device information based on network topology;

[0021] Based on the online device information, multiple surveillance cameras connected to the same central switch are recorded as the same surveillance camera group;

[0022] The central switch is connected to the gateway device via optical fiber, and the gateway device is connected to the network. The gateway device is used to receive the real-time operation data of each monitoring camera in the monitoring camera group corresponding to multiple central switches, and store the historical operation data and video fault data of each monitoring camera in the monitoring camera group. The historical operation data includes the historical video stream data and historical device status data of the monitoring camera in different time periods in the past N unit cycles, and the real-time operation data includes the real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle.

[0023] Preferably, the obtaining of the correlation index between the picture parameter and the video fault type and the related influencing parameters based on the video fault data specifically includes:

[0024] Obtain video fault data, the video fault data includes fault video stream data of each surveillance camera in each time period and video fault types, the video fault types include abnormal brightness, abnormal contrast, image color cast, image blur, abnormal noise, image ghosting, image flickering, image tearing, image freezing, image delay, and image frame loss;

[0025] Acquire picture parameters of the fault video stream data at preset time intervals, wherein the picture parameters include resolution, brightness, contrast, white balance, chroma, sharpness, signal-to-noise ratio, motion vector, and inter-frame difference;

[0026] The correlation between the image parameters extracted from each fault video stream data and the video fault type is calculated according to the Pearson correlation coefficient, and the correlation index is calculated to measure the linear correlation between the image parameters and the video fault type;

[0027] The picture parameters corresponding to the correlation index in the interval of (0.3, 1] are recorded as the relevant influencing parameters of the video fault type;

[0028] The calculation formula of the correlation index is:

[0029]

[0030] In the formula, R xy，Q is the correlation index between the picture parameter x and the video fault type y in the Qth time period, r xy,m，Q is the correlation between the image parameter x and the video fault type y in the mth fault video stream data in the Qth time period, M is the total number of fault video stream data, x iis the observed value of the video picture parameter x in the i-th sampling, y i is the sample value of the fault video stream data in the i-th sampling, j is the total number of sampling times, and are the average values ​​of the picture parameter x and the video fault type y, respectively, wherein the y i It is a binary code, 1 indicates the fault exists, 0 indicates it does not exist;

[0031] Among them, R xy The value range is -1 to 1, r xy ∈[0.3,1] indicates correlation, R xy ∈[0, 0.3) indicates no correlation.

[0032] Preferably, recording the video fault type corresponding to the abnormality index as a suspected fault type and uploading it to the fault assessment model specifically includes:

[0033] Obtain the historical video stream data of each surveillance camera in different time periods in the past N unit cycles in the historical operation parameters, and obtain the picture parameters of the historical video stream data at preset time intervals;

[0034] Calculate the mean value of the image parameter x of each surveillance camera in different time periods

[0035] The edge computing platform obtains real-time video stream data, obtains the time period of the current video stream data, and extracts the correlation index between the picture parameter x and the video fault type y in the time period;

[0036] Use OpenCV to read real-time video stream data, obtain real-time picture parameters of real-time video stream data at preset time intervals, compare the real-time picture parameters with the picture parameter mean, and calculate the difference index of each real-time picture parameter;

[0037] Based on the relevant influencing parameters of each video fault type, the real-time picture parameters of the real-time video stream data, the difference index and the correlation index of the real-time picture parameters are analyzed, and the abnormality degree index of each video fault type in the real-time video stream data is calculated. If the abnormality degree index is lower than the preset degree threshold, the abnormality degree index is recorded as 0; otherwise, the abnormality degree index retains the corresponding value;

[0038] If there is at least one abnormality index, the real-time video stream data obtained in the current unit period is recorded in the suspected fault operation data, the obtained abnormality index is used as the suspected fault index, and the video fault type corresponding to the abnormality index is recorded as the suspected fault type; otherwise, the real-time video stream data obtained in the current unit period is recorded in the historical operation data;

[0039] Upload the suspected fault type and the corresponding suspected fault index to the fault assessment model;

[0040] The calculation formula of the abnormality index is:

[0041]

[0042] In the formula, S y is the abnormality index of video fault type y, is the kth video fault type y y The correlation index of the relevant influencing parameters, K y is the number of relevant influencing parameters of video fault type y, For the bth and kth y The difference index of the real-time picture parameters corresponding to the relevant influencing parameters of the video fault type y is B, and B is the sampling number of the real-time picture parameters, that is, the sampling number of the real-time video frame in the real-time video stream data.

[0043] Preferably, the creating a blank panoramic canvas based on device parameters specifically includes:

[0044] The edge computing platform obtains the device parameters of each surveillance camera in the corresponding surveillance camera group and assigns a unique identifier to each surveillance camera. The device parameters include device model, video resolution, pan / tilt rotation angle range, focal length adjustment range, and image sensor size.

[0045] Based on the device parameters, obtain the minimum focal length, maximum horizontal rotation angle of the pan / tilt, and maximum vertical rotation angle of each surveillance camera; based on the minimum focal length and image sensor size, obtain the maximum horizontal viewing angle and maximum vertical viewing angle of the surveillance camera at a fixed angle;

[0046] Calculate the proportion of the maximum horizontal viewing angle and the maximum vertical viewing angle in the maximum horizontal rotation angle of the gimbal and the maximum vertical rotation angle of the gimbal, respectively, and record them as b1 and b2;

[0047] Based on b1, b2 and video resolution of each surveillance camera, a blank panoramic canvas is created for each surveillance camera.

[0048] Preferably, the step of constructing a panoramic background model of a surveillance camera includes:

[0049] Obtain historical video stream data and historical device status data of surveillance cameras in different time periods in the past N unit cycles in the historical operation data, wherein the historical device status data includes historical pan / tilt angle data and historical focal length adjustment data;

[0050] Extracting multiple historical video frames from the historical video stream data based on a preset acquisition interval;

[0051] Based on the historical PTZ angle data and the historical focal length adjustment data, the PTZ angle data and the focal length adjustment data corresponding to each historical video frame are extracted;

[0052] A day is divided into multiple time periods, and based on the timestamp information of the historical video frames, the historical video frames of the same time period in the past N unit cycles are assigned to a group;

[0053] Adaptively adjust the brightness and contrast of multiple historical video frames according to a preset histogram equalization standard;

[0054] Use multiple historical video frames of each group as model training samples;

[0055] According to the pan / tilt angle data and focal length adjustment data corresponding to the historical video frames, the pixel values ​​of multiple historical video frames in the same group are respectively mapped to the corresponding pixel positions of the blank panoramic canvas;

[0056] The blank panorama canvas stores all pixel values ​​mapped to corresponding pixel locations;

[0057] Calculate the mean and standard deviation of the accumulated pixel values ​​at each corresponding pixel position on the blank panoramic canvas;

[0058] Calculate a pixel deviation index of the accumulated pixel values, compare the pixel deviation index with a pixel deviation threshold, and remove pixel values ​​exceeding the pixel deviation threshold;

[0059] Based on the cumulative pixel values ​​at each pixel location, a Gaussian mixture model is used to model the multimodal distribution of each corresponding pixel location;

[0060] A panoramic background dynamic image is constructed by integrating the main Gaussian components of all pixel positions, and for pixel positions in the panoramic background dynamic image that are not mapped to pixel values, a predefined special value is used to indicate pixel missing at the position;

[0061] Based on the panoramic background dynamic images of multiple time periods, a panoramic background model of the surveillance camera is obtained.

[0062] Preferably, the detecting whether the pan / tilt operation and the zoom lens have faults specifically includes:

[0063] The edge computing platform obtains the real-time operation data of each video camera, wherein the real-time operation data includes the real-time video stream data and real-time device status data of the monitoring camera in different time periods within the current unit cycle, wherein the real-time device status data includes real-time pan / tilt angle data and real-time focal length adjustment data;

[0064] Based on the current time period of the acquired real-time video stream data, a panoramic background dynamic image of a corresponding time period in the panoramic background model is selected;

[0065] Acquire multiple real-time video frames in the real-time video stream data based on a preset acquisition interval;

[0066] Adaptively adjust the brightness and contrast of multiple real-time video frames according to the histogram equalization standard;

[0067] Based on the real-time pan / tilt angle data and the real-time focal length adjustment data, the pan / tilt angle data and the focal length adjustment data corresponding to each real-time video frame are obtained;

[0068] Based on the timestamp information of the real-time video frame, it is matched with the panoramic background dynamic image of the corresponding time period in the panoramic background model;

[0069] Calculate the viewing angle transformation parameters based on the pan / tilt angle and adjusted focal length of the real-time video frame, construct a homography matrix, calculate the theoretical edge contour corresponding to the real-time video frame in the panoramic background dynamic image, and extract the image within the theoretical edge contour of the panoramic background dynamic image as the first background image;

[0070] Calculate the matching feature points of the real-time video frame in the panoramic background dynamic image;

[0071] Based on the matching feature points of the real-time video frame on the panoramic background dynamic image, the actual edge contour of the real-time video frame is drawn on the panoramic background dynamic image, and the image of the panoramic background dynamic image within the actual edge contour is extracted as the second background image;

[0072] The first background image and the second background image obtained from each real-time video frame are respectively recorded as a group;

[0073] Determine the positional relationship and pixel size of each group of the first background image and the second background image on the panoramic background dynamic image, and calculate a comprehensive position offset index and a comprehensive size deviation index;

[0074] Set the offset threshold and deviation threshold. If the comprehensive position offset index is within the offset threshold, the current gimbal is operating normally; otherwise, the gimbal is operating faulty.

[0075] If the size deviation is within the deviation threshold, the current zoom lens operates normally; otherwise, the current zoom lens operates malfunctioning;

[0076] If there is a gimbal operation fault, the gimbal operation abnormality is taken as the fault type, and the offset index is uploaded to the fault assessment model as the abnormality index for fault assessment;

[0077] If there is a zoom lens operation failure, the zoom lens operation abnormality is regarded as a suspected fault type, and the deviation index is uploaded to the fault assessment model as a suspected fault index for fault assessment;

[0078] If there is no pan / tilt operation failure or zoom lens operation failure, the real-time video stream data obtained in the current unit period will be recorded in the historical operation data; otherwise, the real-time video stream data obtained in the current unit period will be recorded in the suspected failure operation data.

[0079] Preferably, the feeding back corresponding processing measures to the fault warning system based on the fault assessment coefficient includes:

[0080] Build a fault assessment model on the central cloud platform to obtain the suspected fault index and suspected fault type of each surveillance camera in the current unit cycle;

[0081] Obtaining weight parameters based on suspected fault types, wherein the suspected fault types include abnormal brightness, abnormal contrast, color cast, blurred image, abnormal noise, ghosting, flickering, tearing, freezing, delay, frame loss, abnormal operation of zoom lens, and abnormal operation of gimbal;

[0082] Calculate the fault assessment coefficient of each surveillance camera based on the weight parameter of the suspected fault type and the suspected fault index;

[0083] When the fault assessment coefficient is less than C1, it is recorded as low fault impact and the video surveillance system continues to be monitored;

[0084] When the fault assessment coefficient is greater than C1 and less than C2, it is recorded as a fault impact and fed back to the fault alarm system. Based on the suspected fault type, the video surveillance system is preliminarily investigated and optimized;

[0085] When the fault assessment coefficient is greater than C2, it is recorded as a high fault impact and fed back to the fault alarm system, and the video surveillance system is immediately investigated and maintained based on the suspected fault type;

[0086] The calculation formula of the fault assessment coefficient is:

[0087]

[0088] Where H is the fault assessment coefficient, η l is the weight parameter of the lth suspected fault type, L is the total number of suspected fault types obtained, S l is the suspected fault index of the lth suspected fault type.

[0089] A second aspect of the present invention provides an intelligent monitoring device for a video surveillance system based on cloud-edge collaboration, comprising:

[0090] The edge computing platform is deployed near the surveillance camera group and is responsible for real-time data processing and analysis;

[0091] The central cloud platform is responsible for global data analysis and fault assessment, and provides feedback on treatment measures;

[0092] A video surveillance system, the video surveillance system comprising a gateway device, a fault alarm system, multiple central switches, and multiple surveillance cameras. The central switch is connected to multiple surveillance cameras, aggregates video stream data from each camera, and transmits the aggregated data to the gateway device. The gateway device receives the video stream data from the central switch and forwards it to the edge computing platform;

[0093] Wherein, the central cloud platform includes:

[0094] The fault assessment module is used to process the suspected fault index and suspected fault type from the edge computing platform: obtain weight parameters based on the suspected fault type; calculate the fault assessment coefficient of each surveillance camera based on the weight parameters of the suspected fault type and the suspected fault index;

[0095] The processing measure feedback module determines the impact of the fault based on the size of the fault assessment coefficient, provides feedback to the fault alarm system, and takes corresponding processing measures.

[0096] Preferably, the edge computing platform comprises:

[0097] The device information acquisition module is used to use a network scanning tool to scan the network link of the video surveillance system, obtain the network topology and online device information, and based on the network topology, record multiple surveillance cameras connected to the same central switch as the same surveillance camera group;

[0098] The video data processing module is used to receive real-time video stream data from the surveillance camera, decode the received real-time video stream data, convert it into processable image frames, and extract real-time picture parameters of the video frames at preset time intervals;

[0099] A fault data processing module is used to obtain video fault data from the monitoring device system, decode the received fault video stream data, convert it into processable image frames, extract the picture parameters of the fault video frame at a preset time interval, calculate the correlation index between the picture parameters and the video fault type, and determine the relevant influencing parameters;

[0100] The image fault detection module is used to obtain the historical video stream data and historical equipment status data of the monitoring camera in different time periods in the past N unit cycles from the monitoring equipment system, and calculate the average image parameters of each monitoring camera in different time periods; obtain real-time video stream data and real-time equipment status data, extract the image parameters of the real-time video stream data at preset time intervals, compare the real-time image parameters with the average image parameters, and calculate the difference index; based on the real-time image parameters, the difference index, the correlation index and related influencing parameters, calculate the abnormality index of each video fault type, if the abnormality index is lower than the preset threshold, it is recorded as 0; otherwise, as a suspected fault index, the suspected fault type and the suspected fault index are uploaded to the fault assessment model.

[0101] Panoramic background model building module, which creates a blank panoramic canvas based on device parameters and builds a panoramic background model based on historical operation data;

[0102] The PTZ and zoom lens fault detection module obtains multiple real-time video frames in the real-time video stream data.

[0103] Match the real-time video frame with the panoramic background model, calculate the perspective transformation parameters, and construct the homography matrix; extract the images within the theoretical edge contour and the actual edge contour in the panoramic background model as the first background image and the second background image, respectively; calculate the comprehensive position offset index and the comprehensive size deviation index to determine whether the gimbal and zoom lens have faults, and generate the suspected fault type and suspected fault index and upload them to the fault assessment model.

[0104] Compared with the prior art, the present invention provides a method and device for intelligent monitoring of a video surveillance system based on cloud-edge collaboration, which has the following beneficial effects:

[0105] The present invention provides an intelligent monitoring method and device for a video surveillance system based on cloud-edge collaboration, which reduces the computing load of a central cloud platform by deploying an edge computing platform on one side of the video surveillance system; calculates the correlation index between picture parameters and video fault types by combining video fault data, historical operation data and real-time operation data to accurately determine the cause of video picture faults; constructs a panoramic canvas through equipment parameters, and constructs a panoramic background model based on historical operation data; combines real-time operation data with the panoramic background model to continuously monitor the faults of the pan / tilt and zoom lens; compared with the monitoring of traditional video surveillance systems, this method can analyze real-time operation status data based on the panoramic background model to judge the operation status of the pan / tilt and zoom lens, so as to timely discover potential faults and avoid monitoring blind spots or picture loss caused by video surveillance system failures; and a fault assessment platform receives the detected suspected fault data and suspected fault index and promptly feeds them back to the fault alarm system, and performs corresponding processing measures to ensure the reliability of the monitoring camera system. BRIEF DESCRIPTION OF THE DRAWINGS

[0106] Figure 1 This is a flow chart of an intelligent monitoring method for a video surveillance system based on cloud-edge collaboration according to the present invention;

[0107] Figure 2 A block diagram of an intelligent monitoring device for a video surveillance system based on cloud-edge collaboration according to the present invention;

[0108] Figure 3 The present invention provides a flow chart for detecting the operation of the pan / tilt head and whether the zoom lens has any faults. DETAILED DESCRIPTION

[0109] In order to make the technical means, creative features, objectives and effects of the present invention easy to understand, the present invention will be further described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0110] In order to address the shortcomings of the existing technology, such as Figure 1 , Figure 2 , Figure 3 As shown, the present invention provides an intelligent monitoring method for a video surveillance system based on cloud-edge collaboration, comprising:

[0111] Acquire online device information of the monitoring device system, and based on the online device information, record multiple monitoring cameras connected to the same central switch as the same monitoring camera group;

[0112] Construct multiple edge computing platforms, wherein the edge computing platforms obtain video fault data of the monitoring camera group, and obtain correlation indexes and related influencing parameters between picture parameters and video fault types based on the video fault data;

[0113] The edge computing platform obtains the historical operation data and real-time operation data of the corresponding monitoring camera group from the monitoring device system, calculates the average value of the picture parameters of each monitoring camera in different time periods based on the historical operation data, obtains the real-time picture parameters based on the real-time operation data, and calculates the difference index of each real-time picture parameter based on the real-time picture parameters and the picture parameter average;

[0114] Construct a fault assessment model, and calculate the abnormality index of each video fault type in the real-time video stream data based on the real-time picture parameters, difference index, correlation index and related influencing parameters of each video fault type. If the abnormality index is lower than the preset degree threshold, the abnormality index is recorded as 0; otherwise, the abnormality index is used as a suspected fault index, and the video fault type corresponding to the abnormality index is recorded as a suspected fault type and uploaded to the fault assessment model;

[0115] The edge computing platform obtains the device parameters of each surveillance camera in the corresponding surveillance camera group and creates a blank panoramic canvas based on the device parameters;

[0116] Obtain historical video stream data and historical device status data of surveillance cameras in different time periods in the past N unit cycles from historical operation data, and build a panoramic background model of surveillance cameras;

[0117] Obtain the real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle in the real-time operation data;

[0118] Extracting a first background image on the panoramic background model based on the real-time device status data, and extracting a second background image on the panoramic background model based on the real-time video stream data;

[0119] Analyze and process the first background image and the second background image to obtain a comprehensive position offset index and a comprehensive size deviation index, and detect whether there is a fault in the pan / tilt operation and the zoom lens; if there is a fault, generate a suspected fault type and a suspected fault index, and upload them to the fault assessment model;

[0120] The suspected fault index and suspected fault type are input into the fault assessment model, the fault assessment coefficient of each surveillance camera in the current unit cycle is calculated, feedback is given to the fault alarm system based on the fault assessment coefficient, and corresponding processing measures are taken;

[0121] The disclosed intelligent monitoring method for a video surveillance system based on cloud-edge collaboration, by deploying an edge computing platform on one side of the video surveillance system, can be responsible for local data processing, thereby reducing the computing load of the central cloud platform and achieving optimal resource utilization and load balancing; by analyzing the correlation index and related influencing parameters of picture parameters and video fault types through historical video data, different types of video fault types in the current cycle can be more accurately identified; based on the panoramic background model and intelligent algorithm, the system can more accurately detect the faults of the pan / tilt and zoom lens; the central cloud platform obtains the suspected fault type and suspected fault index of the edge computing platform, calculates the fault assessment coefficient based on the fault assessment model, generates and feeds back corresponding processing measures to the fault alarm system, so as to achieve efficient, intelligent and reliable video surveillance system fault monitoring and response.

[0122] Specifically, the obtaining of online device information of the monitoring device system also includes:

[0123] Use a network scanning tool to scan the network link of the video surveillance system to obtain the network topology structure of the network link;

[0124] Identify all online device information based on network topology;

[0125] Based on the online device information, multiple surveillance cameras connected to the same central switch are recorded as the same surveillance camera group;

[0126] The central switch is connected to the gateway device via optical fiber, and the gateway device is connected to the network. The gateway device is used to receive the real-time operation data of each monitoring camera in the monitoring camera group corresponding to multiple central switches, and store the historical operation data and video fault data of each monitoring camera in the monitoring camera group. The historical operation data includes the historical video stream data and historical device status data of the monitoring camera in different time periods in the past N unit cycles, and the real-time operation data includes the real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle.

[0127] Specifically, the obtaining of the correlation index between the picture parameter and the video fault type and the related influencing parameters based on the video fault data specifically includes:

[0128] Obtain video fault data, which includes fault video stream data of each surveillance camera in each time period and video fault types, wherein the video fault types include but are not limited to abnormal brightness, abnormal contrast, color cast, blurred image, abnormal noise, ghosting, flickering, tearing, freezing, delay, and frame loss;

[0129] Acquire picture parameters of the fault video stream data at preset time intervals, wherein the picture parameters include but are not limited to resolution, brightness, contrast, white balance, chroma, sharpness, signal-to-noise ratio, motion vector, and inter-frame difference;

[0130] The correlation between the image parameters extracted from each fault video stream data and the video fault type is calculated according to the Pearson correlation coefficient, and the correlation index is calculated to measure the linear correlation between the image parameters and the video fault type;

[0131] The picture parameters corresponding to the correlation index in the interval of (0.3, 1] are recorded as the relevant influencing parameters of the video fault type;

[0132] The calculation formula of the correlation index is:

[0133]

[0134] In the formula, R xy，Q is the correlation index between the picture parameter x and the video fault type y in the Qth time period, r xy,m，Q is the correlation between the image parameter x and the video fault type y in the mth fault video stream data in the Qth time period, M is the total number of fault video stream data, x i is the observed value of the video picture parameter x in the i-th sampling, y i is the sample value of the fault video stream data in the i-th sampling, j is the total number of sampling times, and are the average values ​​of the picture parameter x and the video fault type y, respectively, wherein the y i It is a binary code, 1 indicates the fault exists, 0 indicates it does not exist;

[0135] Among them, R xy The value range is -1 to 1, r xy ∈[0.3,1] indicates correlation, R xy ∈[0, 0.3) means no correlation;

[0136] Specifically, recording the video fault type corresponding to the abnormality index as a suspected fault type and uploading it to the fault assessment model specifically includes:

[0137] Obtain the historical video stream data of each surveillance camera in different time periods in the past N unit cycles in the historical operation parameters, and obtain the picture parameters of the historical video stream data at preset time intervals;

[0138] Calculate the mean value of the image parameter x of each surveillance camera in different time periods

[0139] The edge computing platform obtains real-time video stream data, obtains the time period of the current video stream data, and extracts the correlation index between the picture parameter x and the video fault type y in the time period;

[0140] Use OpenCV to read real-time video stream data, obtain real-time picture parameters of real-time video stream data at preset time intervals, compare the real-time picture parameters with the picture parameter mean, and calculate the difference index of each real-time picture parameter;

[0141] It should be noted that the difference index of the real-time picture parameter can be obtained by calculating the relative difference between the real-time picture parameter and the picture parameter mean;

[0142] Based on the relevant influencing parameters of each video fault type, the real-time picture parameters of the real-time video stream data, the difference index and the correlation index of the real-time picture parameters are analyzed, and the abnormality degree index of each video fault type in the real-time video stream data is calculated. If the abnormality degree index is lower than the preset degree threshold, the abnormality degree index is recorded as 0; otherwise, the abnormality degree index retains the corresponding value;

[0143] It should be noted that the degree threshold of each video fault type is set to

[0144] If there is at least one abnormality index, the real-time video stream data obtained in the current unit period is recorded in the suspected fault operation data, the obtained abnormality index is used as the suspected fault index, and the video fault type corresponding to the abnormality index is recorded as the suspected fault type; otherwise, the real-time video stream data obtained in the current unit period is recorded in the historical operation data;

[0145] Upload the suspected fault type and the corresponding suspected fault index to the fault assessment model;

[0146] The calculation formula of the abnormality index is:

[0147]

[0148] In the formula, S y is the abnormality index of video fault type y, is the kth video fault type y y The correlation index of the relevant influencing parameters, K y is the number of relevant influencing parameters of video fault type y, For the bth and kth y The difference index of the real-time picture parameters corresponding to the relevant influencing parameters of the video fault type y is B, and B is the sampling number of the real-time picture parameters, that is, the sampling number of the real-time video frame in the real-time video stream data.

[0149] Specifically, the creating of a blank panoramic canvas based on device parameters includes:

[0150] The edge computing platform obtains the device parameters of each surveillance camera in the corresponding surveillance camera group and assigns a unique identifier to each surveillance camera. The device parameters include device model, video resolution, pan / tilt rotation angle range, focal length adjustment range, and image sensor size.

[0151] Based on the device parameters, obtain the minimum focal length, maximum horizontal rotation angle of the pan / tilt, and maximum vertical rotation angle of each surveillance camera; based on the minimum focal length and image sensor size, obtain the maximum horizontal viewing angle and maximum vertical viewing angle of the surveillance camera at a fixed angle;

[0152] Calculate the proportion of the maximum horizontal viewing angle and the maximum vertical viewing angle in the maximum horizontal rotation angle of the gimbal and the maximum vertical rotation angle of the gimbal, respectively, and record them as b1 and b2;

[0153] Based on b1, b2 and video resolution of each surveillance camera, a blank panoramic canvas is created for each surveillance camera. The horizontal resolution of the blank panoramic canvas is b1·U h *b2·U v , where U h with U v They are the horizontal resolution of the video and the vertical resolution of the video respectively;

[0154] It should be noted that the calculation formulas for the horizontal viewing angle and the vertical viewing angle are:

[0155]

[0156] In the formula, θ i,h is the horizontal viewing angle of the i-th surveillance camera, θ i,v is the vertical viewing angle of the ith surveillance camera, f i is the minimum focal length of the ith surveillance camera, W i is the image sensor width of the i-th surveillance camera, H i is the image sensor height of the i-th surveillance camera;

[0157] In this way, the minimum focal length f of the i-th surveillance camera is obtained i，min and image sensor size, calculate the maximum horizontal viewing angle θ of the i-th surveillance camera i,h，max With the maximum vertical viewing angle θ i,v，max .

[0158] Specifically, the construction of the panoramic background model of the surveillance camera includes:

[0159] Obtain historical video stream data and historical device status data of surveillance cameras in different time periods in the past N unit cycles in the historical operation data, wherein the historical device status data includes historical pan / tilt angle data and historical focal length adjustment data;

[0160] Extracting multiple historical video frames from the historical video stream data based on a preset acquisition interval;

[0161] Based on the historical PTZ angle data and the historical focal length adjustment data, the PTZ angle data and the focal length adjustment data corresponding to each historical video frame are extracted;

[0162] A day is divided into multiple time periods, and based on the timestamp information of the historical video frames, the historical video frames of the same time period in the past N unit cycles are assigned to a group;

[0163] Adaptively adjusting the brightness and contrast of multiple historical video frames according to a preset histogram equalization standard. It should be noted that the preset histogram equalization standard can be dynamically adjusted according to the contrast and brightness of the extracted historical video frames to reduce the impact of the current time and meteorological factors on the historical video frames;

[0164] Use multiple historical video frames of each group as model training samples;

[0165] According to the pan / tilt angle data and focal length adjustment data corresponding to the historical video frames, the pixel values ​​of multiple historical video frames in the same group are respectively mapped to the corresponding pixel positions of the blank panoramic canvas;

[0166] The blank panorama canvas stores all pixel values ​​mapped to corresponding pixel locations;

[0167] Calculate the mean and standard deviation of the accumulated pixel values ​​at each corresponding pixel position on the blank panoramic canvas;

[0168] Calculate a pixel deviation index of the accumulated pixel values, compare the pixel deviation index with a pixel deviation threshold, and remove pixel values ​​exceeding the pixel deviation threshold;

[0169] Based on the cumulative pixel values ​​at each pixel location, a Gaussian mixture model is used to model the multimodal distribution of each corresponding pixel location;

[0170] A panoramic background dynamic image is constructed by integrating the main Gaussian components of all pixel positions, and for pixel positions in the panoramic background dynamic image that are not mapped to pixel values, a predefined special value is used to indicate pixel missing at the position;

[0171] Based on the panoramic background dynamic images of multiple time periods, a panoramic background model of the surveillance camera is obtained;

[0172] It should be noted that the absolute difference between the current pixel value and the mean is calculated and divided by the standard deviation to obtain the pixel deviation index;

[0173] The pixel deviation threshold uses a statistical method to set a Z-Score threshold range. Pixel values ​​exceeding this range are considered abnormal or foreground image pixel values;

[0174] In the present disclosure, the panoramic background dynamic image updates the pixel values ​​mapped to the panoramic canvas in the past N unit periods in each unit period, and updates the mean and variance of the corresponding pixel positions in the panoramic background dynamic image to ensure the adaptability and accuracy of the panoramic background dynamic image;

[0175] Specifically, the detection of whether the pan / tilt operation and the zoom lens are faulty includes:

[0176] The edge computing platform obtains the real-time operation data of each video camera, wherein the real-time operation data includes the real-time video stream data and real-time device status data of the monitoring camera in different time periods within the current unit cycle, wherein the real-time device status data includes real-time pan / tilt angle data and real-time focal length adjustment data;

[0177] Based on the current time period of the acquired real-time video stream data, a panoramic background dynamic image of a corresponding time period in the panoramic background model is selected;

[0178] Acquire multiple real-time video frames in the real-time video stream data based on a preset acquisition interval;

[0179] Adaptively adjusting the brightness and contrast of multiple real-time video frames according to the histogram equalization standard. It should be noted that the preset histogram equalization standard can be dynamically adjusted according to the contrast and brightness of the extracted real-time video frame to reduce the impact of the current time and meteorological factors on the real-time video frame;

[0180] Based on the real-time pan / tilt angle data and the real-time focal length adjustment data, the pan / tilt angle data and the focal length adjustment data corresponding to each real-time video frame are obtained;

[0181] Based on the timestamp information of the real-time video frame, it is matched with the panoramic background dynamic image of the corresponding time period in the panoramic background model;

[0182] Calculate the viewing angle transformation parameters based on the pan / tilt angle and adjusted focal length of the real-time video frame, construct a homography matrix, calculate the theoretical edge contour corresponding to the real-time video frame in the panoramic background dynamic image, and extract the image within the theoretical edge contour of the panoramic background dynamic image as the first background image;

[0183] Calculate the matching feature points of the real-time video frame in the panoramic background dynamic image;

[0184] Based on the matching feature points of the real-time video frame on the panoramic background dynamic image, the actual edge contour of the real-time video frame is drawn on the panoramic background dynamic image, and the image of the panoramic background dynamic image within the actual edge contour is extracted as the second background image;

[0185] The first background image and the second background image obtained from each real-time video frame are respectively recorded as a group;

[0186] Determine the positional relationship and pixel size of each group of the first background image and the second background image on the panoramic background dynamic image, and calculate a comprehensive position offset index and a comprehensive size deviation index;

[0187] It should be noted that the comprehensive position offset index is obtained as follows:

[0188] Get the pixel distance between the center point of the first background image and the center point of the second background image, and convert the pixel distance;

[0189] Based on the real-time focus adjustment data, the current adjustment focus of the real-time video frame is obtained, and the product of the pixel distance and the adjustment focus is divided by the minimum focus as the position offset index;

[0190] The average of the position shift indexes of each real-time video frame extracted from the real-time video stream in the current unit period is used as the comprehensive position shift index;

[0191] The comprehensive size deviation index is obtained as follows:

[0192] Obtain the pixel size of the first background image and the pixel size of the second background image, calculate the relative difference between the pixel height of the first background image and the pixel height of the second background image, calculate the relative difference between the pixel width of the first background image and the pixel width of the second background image, and add the two relative differences as the size deviation index;

[0193] The average value of the size deviation index of each real-time video frame extracted from the real-time video stream in the current unit period is used as the comprehensive size deviation index;

[0194] Set the offset threshold and deviation threshold. If the comprehensive position offset index is within the offset threshold, the current gimbal is operating normally; otherwise, the gimbal is operating faulty.

[0195] If the size deviation is within the deviation threshold, the current zoom lens operates normally; otherwise, the current zoom lens operates malfunctioning;

[0196] It should be noted that the comprehensive offset threshold is set to 0.3, the comprehensive deviation threshold is set to 0.2, and the comprehensive offset threshold and the comprehensive deviation threshold can be adjusted according to actual conditions;

[0197] If there is a gimbal operation fault, the gimbal operation abnormality is taken as the fault type, and the offset index is uploaded to the fault assessment model as the abnormality index for fault assessment;

[0198] If there is a zoom lens operation failure, the zoom lens operation abnormality is regarded as a suspected fault type, and the deviation index is uploaded to the fault assessment model as a suspected fault index for fault assessment;

[0199] If there is no pan / tilt operation failure or zoom lens operation failure, the real-time video stream data obtained in the current unit period will be recorded in the historical operation data; otherwise, the real-time video stream data obtained in the current unit period will be recorded in the suspected failure operation data.

[0200] Specifically, the feedback of corresponding processing measures to the fault warning system based on the fault assessment coefficient includes:

[0201] Build a fault assessment model on the central cloud platform to obtain the suspected fault index and suspected fault type of each surveillance camera in the current unit cycle;

[0202] Obtaining weight parameters based on suspected fault types, wherein the suspected fault types include abnormal brightness, abnormal contrast, color cast, blurred image, abnormal noise, ghosting, flickering, tearing, freezing, delay, frame loss, abnormal operation of zoom lens, and abnormal operation of gimbal;

[0203] Calculate the fault assessment coefficient of each surveillance camera based on the weight parameter of the suspected fault type and the suspected fault index;

[0204] When the fault assessment coefficient is less than C1, it is recorded as low fault impact and the video surveillance system continues to be monitored;

[0205] When the fault assessment coefficient is greater than C1 and less than C2, it is recorded as a medium fault impact and fed back to the fault alarm system. Based on the suspected fault type, the video surveillance system is preliminarily investigated and optimized;

[0206] When the fault assessment coefficient is greater than C2, it is recorded as a high fault impact and fed back to the fault alarm system, and the video surveillance system is immediately investigated and maintained based on the suspected fault type;

[0207] The calculation formula of the fault assessment coefficient is:

[0208]

[0209] Where H is the fault assessment coefficient, η l is the weight parameter of the lth suspected fault type, L is the total number of suspected fault types obtained, S l is the suspected fault index of the lth suspected fault type;

[0210] It should be noted that C1 and C2 are reasonably set based on the distribution of fault assessment coefficients, and the weight parameters of the suspected fault types are obtained through expert scoring;

[0211] The fault type is determined after investigation. If the fault type is a video fault type, the suspected fault data and the corresponding fault type are uploaded to the gateway device as video fault data.

[0212] A second aspect of the present invention provides an intelligent monitoring device for a video surveillance system based on cloud-edge collaboration, comprising:

[0213] The edge computing platform is deployed near the surveillance camera group and is responsible for real-time data processing and analysis;

[0214] The central cloud platform is responsible for global data analysis and fault assessment, and provides feedback on treatment measures;

[0215] A video surveillance system, the video surveillance system comprising a gateway device, a fault alarm system, multiple central switches, and multiple surveillance cameras. The central switch is connected to multiple surveillance cameras, aggregates video stream data from each camera, and transmits the aggregated data to the gateway device. The gateway device receives the video stream data from the central switch and forwards it to the edge computing platform;

[0216] Wherein, the central cloud platform includes:

[0217] The fault assessment module is used to process the suspected fault index and suspected fault type from the edge computing platform: obtain weight parameters based on the suspected fault type; calculate the fault assessment coefficient of each surveillance camera based on the weight parameters of the suspected fault type and the suspected fault index;

[0218] The processing measure feedback module determines the impact of the fault based on the size of the fault assessment coefficient, provides feedback to the fault alarm system, and takes corresponding processing measures.

[0219] Specifically, the edge computing platform includes:

[0220] The device information acquisition module is used to use a network scanning tool to scan the network link of the video surveillance system, obtain the network topology and online device information, and based on the network topology, record multiple surveillance cameras connected to the same central switch as the same surveillance camera group;

[0221] The video data processing module is used to receive real-time video stream data from the surveillance camera, decode the received real-time video stream data, convert it into processable image frames, and extract real-time picture parameters of the video frames at preset time intervals;

[0222] A fault data processing module is used to obtain video fault data from the monitoring device system, decode the received fault video stream data, convert it into processable image frames, extract the picture parameters of the fault video frame at a preset time interval, calculate the correlation index between the picture parameters and the video fault type, and determine the relevant influencing parameters;

[0223] The image fault detection module is used to obtain the historical video stream data and historical equipment status data of the monitoring camera in different time periods in the past N unit cycles from the monitoring equipment system, and calculate the average image parameters of each monitoring camera in different time periods; obtain real-time video stream data and real-time equipment status data, extract the image parameters of the real-time video stream data at preset time intervals, compare the real-time image parameters with the average image parameters, and calculate the difference index; based on the real-time image parameters, the difference index, the correlation index and related influencing parameters, calculate the abnormality index of each video fault type, if the abnormality index is lower than the preset threshold, it is recorded as 0; otherwise, as a suspected fault index, the suspected fault type and the suspected fault index are uploaded to the fault assessment model.

[0224] Panoramic background model building module, which creates a blank panoramic canvas based on device parameters and builds a panoramic background model based on historical operation data;

[0225] The PTZ and zoom lens fault detection module obtains multiple real-time video frames in the real-time video stream data.

[0226] Match the real-time video frame with the panoramic background model, calculate the perspective transformation parameters, and construct the homography matrix; extract the images within the theoretical edge contour and the actual edge contour in the panoramic background model as the first background image and the second background image, respectively; calculate the comprehensive position offset index and the comprehensive size deviation index to determine whether the gimbal and zoom lens have faults, and generate the suspected fault type and suspected fault index and upload them to the fault assessment model.

[0227] To sum up, the advantages of the present invention are as follows: the present invention provides an intelligent monitoring method and device for a video surveillance system based on cloud-edge collaboration, which reduces the computing load of the central cloud platform by deploying an edge computing platform on one side of the video surveillance system; calculates the correlation index between the picture parameters and the video fault type by combining video fault data, historical operation data and real-time operation data to accurately determine the cause of the video picture fault; constructs a panoramic canvas through equipment parameters, and constructs a panoramic background model based on historical operation data; combines real-time operation data with the panoramic background model to continuously monitor the faults of the pan / tilt and zoom lens; the fault assessment platform receives the detected suspected fault data and suspected fault index and promptly feeds back to the fault alarm system, and takes corresponding processing measures to ensure the reliability of the monitoring camera system.

[0228] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. An intelligent monitoring method for a video surveillance system based on cloud-edge collaboration, characterized in that: include: Acquire online device information of the monitoring device system, and based on the online device information, record multiple monitoring cameras connected to the same central switch as the same monitoring camera group; Construct multiple edge computing platforms, wherein the edge computing platforms obtain video fault data of the monitoring camera group, and obtain correlation indexes and related influencing parameters between picture parameters and video fault types based on the video fault data; The edge computing platform obtains the historical operation data and real-time operation data of the corresponding monitoring camera group from the monitoring device system, calculates the average value of the picture parameters of each monitoring camera in different time periods based on the historical operation data, obtains the real-time picture parameters based on the real-time operation data, and calculates the difference index of each real-time picture parameter based on the real-time picture parameters and the picture parameter average; Construct a fault assessment model, and calculate the abnormality index of each video fault type in the real-time video stream data based on the real-time picture parameters, difference index, correlation index and related influencing parameters of each video fault type. If the abnormality index is lower than the preset degree threshold, the abnormality index is recorded as 0; otherwise, the abnormality index is used as a suspected fault index, and the video fault type corresponding to the abnormality index is recorded as a suspected fault type and uploaded to the fault assessment model; The edge computing platform obtains the device parameters of each surveillance camera in the corresponding surveillance camera group and creates a blank panoramic canvas based on the device parameters; Obtain historical video stream data and historical device status data of surveillance cameras in different time periods in the past N unit cycles from historical operation data, and build a panoramic background model of surveillance cameras; Obtain the real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle in the real-time operation data; Extracting a first background image on the panoramic background model based on the real-time device status data, and extracting a second background image on the panoramic background model based on the real-time video stream data; Analyze and process the first background image and the second background image to obtain a comprehensive position offset index and a comprehensive size deviation index, and detect whether there is a fault in the pan / tilt operation and the zoom lens; if there is a fault, generate a suspected fault type and a suspected fault index, and upload them to the fault assessment model; The suspected fault index and suspected fault type are input into the fault assessment model to calculate the fault assessment coefficient of each surveillance camera in the current unit cycle, and feedback is given to the fault alarm system based on the fault assessment coefficient, and corresponding processing measures are taken.

2. According to claim 1, a video surveillance system intelligent monitoring method based on cloud-edge collaboration is characterized in that: The obtaining of online device information of the monitoring device system also includes: Use a network scanning tool to scan the network link of the video surveillance system to obtain the network topology structure of the network link; Identify all online device information based on network topology; Based on the online device information, multiple surveillance cameras connected to the same central switch are recorded as the same surveillance camera group; The central switch is connected to the gateway device via optical fiber, and the gateway device is connected to the network. The gateway device is used to receive the real-time operation data of each monitoring camera in the monitoring camera group corresponding to multiple central switches, and store the historical operation data and video fault data of each monitoring camera in the monitoring camera group. The historical operation data includes the historical video stream data and historical device status data of the monitoring camera in different time periods in the past N unit cycles, and the real-time operation data includes the real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle.

3. According to claim 2, a video surveillance system intelligent monitoring method based on cloud-edge collaboration is characterized in that: The obtaining of the correlation index between the picture parameter and the video fault type and the related influencing parameters based on the video fault data specifically includes: Obtain video fault data, the video fault data includes fault video stream data of each surveillance camera in each time period and video fault types, the video fault types include abnormal brightness, abnormal contrast, image color cast, image blur, abnormal noise, image ghosting, image flickering, image tearing, image freezing, image delay, and image frame loss; Acquire picture parameters of the fault video stream data at preset time intervals, wherein the picture parameters include resolution, brightness, contrast, white balance, chroma, sharpness, signal-to-noise ratio, motion vector, and inter-frame difference; The correlation between the image parameters extracted from each fault video stream data and the video fault type is calculated according to the Pearson correlation coefficient, and the correlation index is calculated to measure the linear correlation between the image parameters and the video fault type; The picture parameters corresponding to the correlation index in the interval of (0.3, 1] are recorded as the relevant influencing parameters of the video fault type; The calculation formula of the correlation index is: In the formula, R xy，Q is the correlation index between the picture parameter x and the video fault type y in the Qth time period, r xy,m,Q is the correlation between the image parameter x and the video fault type y in the mth fault video stream data in the Qth time period, M is the total number of fault video stream data, x i is the observed value of the video picture parameter x in the i-th sampling, y i is the sample value of the fault video stream data in the i-th sampling, j is the total number of sampling times, and are the average values ​​of the picture parameter x and the video fault type y, respectively, wherein the y i It is a binary code, 1 indicates the fault exists, 0 indicates it does not exist; Among them, R xy The value range is from -1 to 1, r xy ∈[0.3,1] indicates correlation, R xy ∈[0, 0.3) means no correlation.

4. According to claim 3, a video surveillance system intelligent monitoring method based on cloud-edge collaboration is characterized in that: The video fault type corresponding to the abnormality index is recorded as a suspected fault type and uploaded to the fault assessment model, specifically including: Obtain the historical video stream data of each surveillance camera in different time periods in the past N unit cycles in the historical operation parameters, and obtain the picture parameters of the historical video stream data at preset time intervals; Calculate the mean value of the image parameter x of each surveillance camera in different time periods The edge computing platform obtains real-time video stream data, obtains the time period of the current video stream data, and extracts the correlation index between the picture parameter x and the video fault type y in the time period; Use OpenCV to read real-time video stream data, obtain real-time picture parameters of real-time video stream data at preset time intervals, compare the real-time picture parameters with the picture parameter mean, and calculate the difference index of each real-time picture parameter; Based on the relevant influencing parameters of each video fault type, the real-time picture parameters of the real-time video stream data, the difference index and the correlation index of the real-time picture parameters are analyzed, and the abnormality degree index of each video fault type in the real-time video stream data is calculated. If the abnormality degree index is lower than the preset degree threshold, the abnormality degree index is recorded as 0; otherwise, the abnormality degree index retains the corresponding value; If there is at least one abnormality index, the real-time video stream data obtained in the current unit period is recorded in the suspected fault operation data, the obtained abnormality index is used as the suspected fault index, and the video fault type corresponding to the abnormality index is recorded as the suspected fault type; otherwise, the real-time video stream data obtained in the current unit period is recorded in the historical operation data; Upload the suspected fault type and the corresponding suspected fault index to the fault assessment model; The calculation formula of the abnormality index is: In the formula, S y is the abnormality index of video fault type y, is the kth video fault type y y The correlation index of the relevant influencing parameters, K y is the number of relevant influencing parameters of video fault type y, For the bth and kth y The difference index of the real-time picture parameters corresponding to the relevant influencing parameters of the video fault type y is B, and B is the sampling number of the real-time picture parameters, that is, the sampling number of the real-time video frame in the real-time video stream data.

5. According to claim 4, a video surveillance system intelligent monitoring method based on cloud-edge collaboration is characterized in that: The creating of a blank panoramic canvas based on device parameters specifically includes: The edge computing platform obtains the device parameters of each surveillance camera in the corresponding surveillance camera group and assigns a unique identifier to each surveillance camera. The device parameters include device model, video resolution, pan / tilt rotation angle range, focal length adjustment range, and image sensor size. Based on the device parameters, obtain the minimum focal length, maximum horizontal rotation angle of the pan / tilt, and maximum vertical rotation angle of each surveillance camera; based on the minimum focal length and image sensor size, obtain the maximum horizontal viewing angle and maximum vertical viewing angle of the surveillance camera at a fixed angle; Calculate the proportion of the maximum horizontal viewing angle and the maximum vertical viewing angle in the maximum horizontal rotation angle of the gimbal and the maximum vertical rotation angle of the gimbal, respectively, and record them as b1 and b2; Based on b1, b2 and video resolution of each surveillance camera, a blank panoramic canvas is created for each surveillance camera.

6. The intelligent monitoring method of a video surveillance system based on cloud-edge collaboration according to claim 5 is characterized in that: The method of constructing a panoramic background model of a surveillance camera includes: Obtain historical video stream data and historical device status data of surveillance cameras in different time periods in the past N unit cycles in the historical operation data, wherein the historical device status data includes historical pan / tilt angle data and historical focal length adjustment data; Extracting multiple historical video frames from the historical video stream data based on a preset acquisition interval; Based on the historical PTZ angle data and the historical focal length adjustment data, the PTZ angle data and the focal length adjustment data corresponding to each historical video frame are extracted; A day is divided into multiple time periods, and based on the timestamp information of the historical video frames, the historical video frames of the same time period in the past N unit cycles are assigned to a group; Adaptively adjust the brightness and contrast of multiple historical video frames according to a preset histogram equalization standard; Use multiple historical video frames of each group as model training samples; According to the pan / tilt angle data and focal length adjustment data corresponding to the historical video frames, the pixel values ​​of multiple historical video frames in the same group are respectively mapped to the corresponding pixel positions of the blank panoramic canvas; The blank panorama canvas stores all pixel values ​​mapped to corresponding pixel locations; Calculate the mean and standard deviation of the accumulated pixel values ​​at each corresponding pixel position on the blank panoramic canvas; Calculate a pixel deviation index of the accumulated pixel values, compare the pixel deviation index with a pixel deviation threshold, and remove pixel values ​​exceeding the pixel deviation threshold; Based on the cumulative pixel values ​​at each pixel location, a Gaussian mixture model is used to model the multimodal distribution of each corresponding pixel location; A panoramic background dynamic image is constructed by integrating the main Gaussian components of all pixel positions, and for pixel positions in the panoramic background dynamic image that are not mapped to pixel values, a predefined special value is used to indicate pixel missing at the position; Based on the panoramic background dynamic images of multiple time periods, a panoramic background model of the surveillance camera is obtained.

7. The intelligent monitoring method of a video surveillance system based on cloud-edge collaboration according to claim 6 is characterized in that: The detecting whether the pan / tilt operation and the zoom lens have faults specifically includes: The edge computing platform obtains the real-time operation data of each video camera, wherein the real-time operation data includes the real-time video stream data and real-time device status data of the monitoring camera in different time periods within the current unit cycle, wherein the real-time device status data includes real-time pan / tilt angle data and real-time focal length adjustment data; Based on the current time period of the acquired real-time video stream data, a panoramic background dynamic image of a corresponding time period in the panoramic background model is selected; Acquire multiple real-time video frames in the real-time video stream data based on a preset acquisition interval; Adaptively adjust the brightness and contrast of multiple real-time video frames according to the histogram equalization standard; Based on the real-time pan / tilt angle data and the real-time focal length adjustment data, the pan / tilt angle data and the focal length adjustment data corresponding to each real-time video frame are obtained; Based on the timestamp information of the real-time video frame, it is matched with the panoramic background dynamic image of the corresponding time period in the panoramic background model; Calculate the viewing angle transformation parameters based on the pan / tilt angle and adjusted focal length of the real-time video frame, construct a homography matrix, calculate the theoretical edge contour corresponding to the real-time video frame in the panoramic background dynamic image, and extract the image within the theoretical edge contour of the panoramic background dynamic image as the first background image; Calculate the matching feature points of the real-time video frame in the panoramic background dynamic image; Based on the matching feature points of the real-time video frame on the panoramic background dynamic image, the actual edge contour of the real-time video frame is drawn on the panoramic background dynamic image, and the image of the panoramic background dynamic image within the actual edge contour is extracted as the second background image; Determine the positional relationship and pixel size of each group of the first background image and the second background image on the panoramic background dynamic image, and calculate the comprehensive position offset index and the comprehensive size deviation index; set the offset threshold and the deviation threshold, if the comprehensive position offset index is within the offset threshold, the current gimbal is operating normally; otherwise, the gimbal is operating faulty; If the size deviation is within the deviation threshold, the current zoom lens operates normally; otherwise, the current zoom lens operates malfunctioning; If there is a gimbal operation fault, the gimbal operation abnormality is taken as the fault type, and the offset index is uploaded to the fault assessment model as the abnormality index for fault assessment; If there is a zoom lens operation failure, the zoom lens operation abnormality is regarded as a suspected fault type, and the deviation index is uploaded to the fault assessment model as a suspected fault index for fault assessment; If there is no pan / tilt operation failure or zoom lens operation failure, the real-time video stream data obtained in the current unit period will be recorded in the historical operation data; otherwise, the real-time video stream data obtained in the current unit period will be recorded in the suspected failure operation data.

8. The intelligent monitoring method of a video surveillance system based on cloud-edge collaboration according to claim 7 is characterized in that: Feedback of corresponding processing measures to the fault warning system based on the fault assessment coefficient includes: Build a fault assessment model on the central cloud platform to obtain the suspected fault index and suspected fault type of each surveillance camera in the current unit cycle; Obtaining weight parameters based on suspected fault types, wherein the suspected fault types include abnormal brightness, abnormal contrast, color cast, blurred image, abnormal noise, ghosting, flickering, tearing, freezing, delay, frame loss, abnormal operation of zoom lens, and abnormal operation of gimbal; Calculate the fault assessment coefficient of each surveillance camera based on the weight parameter of the suspected fault type and the suspected fault index; When the fault assessment coefficient is less than C1, it is recorded as low fault impact and the video surveillance system continues to be monitored; When the fault assessment coefficient is greater than C1 and less than C2, it is recorded as a medium fault impact and fed back to the fault alarm system. Based on the suspected fault type, the video surveillance system is preliminarily investigated and optimized; When the fault assessment coefficient is greater than C2, it is recorded as a high fault impact and fed back to the fault alarm system, and the video surveillance system is immediately investigated and maintained based on the suspected fault type; The calculation formula of the fault assessment coefficient is: Where H is the fault assessment coefficient, η l is the weight parameter of the lth suspected fault type, L is the total number of suspected fault types obtained, S l is the suspected fault index of the lth suspected fault type.

9. An intelligent monitoring device for a video surveillance system based on cloud-edge collaboration, used to implement the monitoring method according to any one of claims 1 to 8, characterized in that: include: The edge computing platform is deployed near the surveillance camera group and is responsible for real-time data processing and analysis; The central cloud platform is responsible for global data analysis and fault assessment, and provides feedback on treatment measures; A video surveillance system, the video surveillance system comprising a gateway device, a fault alarm system, multiple central switches, and multiple surveillance cameras. The central switch is connected to multiple surveillance cameras, aggregates video stream data from each camera, and transmits the aggregated data to the gateway device. The gateway device receives the video stream data from the central switch and forwards it to the edge computing platform; Wherein, the central cloud platform includes: The fault assessment module is used to process the suspected fault index and suspected fault type from the edge computing platform: obtain weight parameters based on the suspected fault type; calculate the fault assessment coefficient of each surveillance camera based on the weight parameters of the suspected fault type and the suspected fault index; The processing measure feedback module determines the impact of the fault based on the size of the fault assessment coefficient, provides feedback to the fault alarm system, and takes corresponding processing measures.

10. The intelligent monitoring method of a video surveillance system based on cloud-edge collaboration according to claim 9 is characterized in that: The edge computing platform includes: The device information acquisition module is used to use a network scanning tool to scan the network link of the video surveillance system, obtain the network topology and online device information, and based on the network topology, record multiple surveillance cameras connected to the same central switch as the same surveillance camera group; The video data processing module is used to receive real-time video stream data from the surveillance camera, decode the received real-time video stream data, convert it into processable image frames, and extract real-time picture parameters of the video frames at preset time intervals; A fault data processing module is used to obtain video fault data from the monitoring device system, decode the received fault video stream data, convert it into processable image frames, extract the picture parameters of the fault video frame at a preset time interval, calculate the correlation index between the picture parameters and the video fault type, and determine the relevant influencing parameters; The image fault detection module is used to obtain the historical video stream data and historical equipment status data of the monitoring camera in different time periods in the past N unit cycles from the monitoring equipment system, and calculate the average image parameters of each monitoring camera in different time periods; obtain real-time video stream data and real-time equipment status data, extract the image parameters of the real-time video stream data at preset time intervals, compare the real-time image parameters with the average image parameters, and calculate the difference index; based on the real-time image parameters, the difference index, the correlation index and related influencing parameters, calculate the abnormality index of each video fault type, if the abnormality index is lower than the preset threshold, it is recorded as 0; otherwise, as a suspected fault index, the suspected fault type is uploaded to the fault assessment model. Panoramic background model building module, which creates a blank panoramic canvas based on device parameters and builds a panoramic background model based on historical operation data; The pan / tilt and zoom lens fault detection module obtains multiple real-time video frames in the real-time video stream data, matches the real-time video frames with the panoramic background model, calculates the perspective transformation parameters, and constructs the homography matrix; The images within the theoretical edge contour and the actual edge contour in the panoramic background model are extracted as the first background image and the second background image respectively; the comprehensive position offset index and the comprehensive size deviation index are calculated to determine whether there are faults in the gimbal and the zoom lens, and the suspected fault type and suspected fault index are generated.

Images