An intelligent monitoring method and device for a video surveillance system based on cloud-edge collaboration

Abstract

The present invention discloses an intelligent monitoring method and device for a video surveillance system based on cloud-edge collaboration, relating to video surveillance technology. The present invention reduces the computing load of the central cloud platform by deploying an edge computing platform on one side of the video surveillance system; calculates the correlation index between the picture parameters and the video fault type by combining video fault data, historical operation data, and real-time operation data to accurately determine the cause of the video picture fault; constructs a panoramic canvas through device parameters and constructs a panoramic background model based on historical operation data; combines real-time operation data with the panoramic background model to continuously monitor the faults of the pan-tilt and zoom lens, avoiding monitoring blind spots or picture loss caused by video surveillance system faults; the fault assessment platform receives the detected suspected fault data and suspected fault index and timely feedbacks them to the fault warning system for corresponding treatment measures, which can ensure the reliability of the monitoring camera system.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance, and specifically to an intelligent monitoring method and device for a video surveillance system based on cloud-edge collaboration. Background Art

[0002] At present, with the rapid development of video surveillance system technology, video surveillance systems have been applied in more and more places. And with the development of network technology, the country's investment in security, and the participation of operators, numerous large-scale video surveillance systems have been established, and the monitoring network can cover a city, a province, or even a country. In video surveillance systems, more and more various extended functions have been added, connecting multiple systems together to realize some new functions and expand the application scope of the original system.

[0003] Existing video surveillance systems mainly rely on manual inspection to identify faults in video images. This method not only has low efficiency but is also easily affected by human factors, making it difficult to ensure the accuracy of video image fault judgment, and increasing the difficulty and cost of fault handling to a certain extent. In a video surveillance system, the pan-tilt of a surveillance camera can automatically identify and track moving targets according to video analysis algorithms. However, the video surveillance system lacks the ability to monitor the operating status of the pan-tilt and zoom lens in real time. When a fault occurs in the pan-tilt or zoom lens (such as jamming, inability to rotate, abnormal zoom, etc.), manual monitoring cannot timely judge the operating status of the pan-tilt and zoom lens, and potential faults cannot be discovered in time, which may lead to phenomena such as monitoring blind spots or image loss, affecting the monitoring effect.

[0004] To solve this problem, we propose an intelligent monitoring method and device for a video surveillance system based on cloud-edge collaboration. Summary of the Invention

[0005] Aiming at the deficiencies of the prior art, the present invention provides an intelligent monitoring method and device for a video surveillance system based on cloud-edge collaboration, which can effectively solve the problems in the above background art.

[0006] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] An intelligent monitoring method for a video surveillance system based on cloud-edge collaboration includes:

[0008] Obtain the online device information of the monitoring device system, and based on the online device information, record multiple monitoring cameras connected to the same central switch as the same monitoring camera group;

[0009] Construct multiple edge computing platforms, and the edge computing platforms obtain the video fault data of the monitoring camera group, and based on the video fault data, obtain the correlation index between the picture parameters and the video fault type and related influencing parameters;

[0010] The edge computing platform obtains the historical operation data and real-time operation data of the corresponding monitoring camera group from the monitoring device system, calculates the average values of the picture parameters of each monitoring camera in different time periods based on the historical operation data, obtains the real-time picture parameters based on the real-time operation data, and calculates the difference index of each real-time picture parameter based on the real-time picture parameters and the average values of the picture parameters;

[0011] Build a fault evaluation model, calculate the abnormality degree index of each video fault type in the real-time video stream data based on the real-time picture parameters, difference index, correlation index and relevant influence parameters of each video fault type. If the abnormality degree index is lower than the preset degree threshold, record the abnormality degree index as 0; otherwise, use the abnormality degree index as the suspected fault index, record the video fault type corresponding to the abnormality degree index as the suspected fault type and upload it to the fault evaluation model;

[0012] The edge computing platform obtains the device parameters of each monitoring camera in the corresponding monitoring camera group and creates a blank panoramic canvas based on the device parameters;

[0013] Obtain the historical video stream data and historical device status data of the monitoring camera in different time periods in the past N unit cycles in the historical operation data, and build a panoramic background model of the monitoring camera;

[0014] Obtain the real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle in the real-time operation data;

[0015] Extract the first background image on the panoramic background model based on the real-time device status data, and extract the second background image on the panoramic background model based on the real-time video stream data;

[0016] Analyze and process the first background image and the second background image to obtain the comprehensive position offset index and the comprehensive size deviation index, and detect whether there are faults in the pan-tilt operation and zoom lens; if there are faults, generate the suspected fault type and the suspected fault index, and upload them to the fault evaluation model;

[0017] Input the suspected fault index and the suspected fault type into the built fault evaluation model, calculate the fault evaluation coefficient of each monitoring camera in the current unit cycle, feedback to the fault alarm system based on the fault evaluation coefficient, and take corresponding treatment measures.

[0018] Preferably, the obtaining of the online device information of the monitoring device system further includes:

[0019] Use a network scanning tool to scan the network link of the video monitoring system and obtain the network topology structure of the network link;

[0020] Identify all online device information based on the network topology structure;

[0021] Based on the online device information, multiple monitoring cameras connected to the same central switch are recorded as the same monitoring camera group;

[0022] The central switch is connected to the gateway device through optical fiber, and the gateway device is connected to the network. The gateway device is used to receive the real-time operation data of each monitoring camera in the monitoring camera group corresponding to multiple central switches, and store the historical operation data and video fault data of each monitoring camera in the monitoring camera group. The historical operation data includes historical video stream data and historical device status data of the monitoring camera at different time periods in the past N unit cycles, and the real-time operation data includes real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle.

[0023] Preferably, obtaining the correlation index and related influence parameters between the picture parameters and the video fault type based on the video fault data specifically includes:

[0024] Obtain video fault data, which includes the fault video stream data and video fault types of each monitoring camera at each time period. The video fault types include abnormal brightness, abnormal contrast, color cast of the picture, blurred picture, abnormal noise, picture ghosting, picture flickering, picture tearing, picture freezing, picture delay, and picture frame loss;

[0025] Obtain the picture parameters of the fault video stream data at a preset time interval. The picture parameters include resolution, brightness, contrast, white balance, chromaticity, sharpness, signal-to-noise ratio, motion vector, and inter-frame difference;

[0026] Calculate the correlation between the picture parameters extracted from each fault video stream data and the video fault type according to the Pearson correlation coefficient, and calculate the correlation index to measure the linear correlation between the picture parameters and the video fault type;

[0027] Record the picture parameters corresponding to the correlation index in the interval (0.3, 1] as the related influence parameters of the video fault type;

[0028] The calculation formula of the correlation index is:

[0029]

[0030] In the formula, R xy，Q is the correlation index between the picture parameter x and the video fault type y in the Qth time period, r xy,m，Q is the correlation between the picture parameter x and the video fault type y in the Qth time period in the mth fault video stream data, M is the total number of fault video stream data, x iLet \(x\) be the observed value of the video frame parameter in the \(i\)-th sampling, and \(y\) i be the sample value of the faulty video stream data in the \(i\)-th sampling, \(j\) be the total number of samplings, and be the average values of the frame parameter \(x\) and the video fault type \(y\) respectively, where the \(y\) i is in binary encoding, 1 indicates the presence of the fault, and 0 indicates the absence;

[0031] where, \(R\) xy ranges from -1 to 1, \(r\) xy ∈[0.3, 1] indicates correlation, \(R\) xy ∈[0, 0.3) indicates no correlation.

[0032] Preferably, recording the video fault type corresponding to the anomaly degree index as the suspected fault type and uploading it to the fault assessment model specifically includes:

[0033] Obtain the historical video stream data of each monitoring camera at different time periods in the past \(N\) unit cycles in the historical operation parameters, and obtain the frame parameters of the historical video stream data at a preset time interval;

[0034] Calculate the average value of the frame parameter \(x\) of each monitoring camera at different time periods

[0035] The edge computing platform obtains the real-time video stream data, obtains the time period where the current video stream data is located, and extracts the correlation index between the frame parameter \(x\) and the video fault type \(y\) in the time period;

[0036] Use OpenCV to read the real-time video stream data, obtain the real-time frame parameters of the real-time video stream data at a preset time interval, compare the real-time frame parameters with the average value of the frame parameters, and calculate the difference index of each real-time frame parameter;

[0037] Based on the relevant influence parameters of each video fault type, analyze the real-time frame parameters of the real-time video stream data, the difference index of the real-time frame parameters, and the correlation index, calculate the anomaly degree index of each video fault type in the real-time video stream data. If the anomaly degree index is lower than the preset degree threshold, then record the anomaly degree index as 0; otherwise, retain the corresponding value of the anomaly degree index;

[0038] If there is at least one anomaly degree index, record the real-time video stream data obtained in the current unit cycle into the suspected fault operation data, use the obtained anomaly degree index as the suspected fault index, and record the video fault type corresponding to the anomaly degree index as the suspected fault type; otherwise, record the real-time video stream data obtained in the current unit cycle into the historical operation data;

[0039] Upload the suspected fault type and the corresponding suspected fault index to the fault assessment model;

[0040] The calculation formula of the abnormality degree index is as follows:

[0041]

[0042] In the formula, S y is the abnormality degree index of the video fault type y, is the correlation index of the k y th relevant influencing parameter of the video fault type y, K y is the number of relevant influencing parameters of the video fault type y, is the difference index of the real-time picture parameter corresponding to the bth and the k y th relevant influencing parameters of the video fault type y, and B is the sampling times of the real-time picture parameter, that is, the sampling times of the real-time video frames in the real-time video stream data.

[0043] Preferably, creating a blank panoramic canvas based on device parameters specifically includes:

[0044] The edge computing platform obtains the device parameters of each monitoring camera in the corresponding monitoring camera group, assigns a unique identifier to each monitoring camera, and the device parameters include device model, video resolution, pan / tilt rotation angle range, focal length adjustment range, and image sensor size;

[0045] Based on the device parameters, obtain the minimum focal length, the maximum horizontal rotation angle of the pan / tilt, and the maximum vertical rotation angle of the pan / tilt of each monitoring camera, and obtain the maximum horizontal viewing angle and the maximum vertical viewing angle of the monitoring camera at a fixed angle based on the minimum focal length and the image sensor size;

[0046] Calculate the ratios of the maximum horizontal viewing angle and the maximum vertical viewing angle in the maximum horizontal rotation angle and the maximum vertical rotation angle of the pan / tilt respectively, and record them as b1 and b2;

[0047] Based on b1, b2 and the video resolution of each monitoring camera, create a blank panoramic canvas for each monitoring camera.

[0048] Preferably, constructing the panoramic background model of the monitoring camera includes:

[0049] Obtain the historical video stream data and historical device status data of the monitoring camera at different time periods in the past N unit cycles in the historical operation data, and the historical device status data includes historical pan / tilt angle data and historical focal length adjustment data;

[0050] Extract multiple historical video frames from the historical video stream data based on a preset acquisition interval;

[0051] Based on the historical pan-tilt angle data and historical focal length adjustment data, extract the pan-tilt angle data and focal length adjustment data corresponding to each historical video frame;

[0052] Divide a day into multiple time periods, and based on the timestamp information of the historical video frames, allocate the historical video frames in the same time period in the past N unit cycles to a group;

[0053] Adjust the picture brightness and contrast of multiple historical video frames adaptively according to a preset histogram equalization criterion;

[0054] Use the multiple historical video frames of each group as model training samples;

[0055] According to the pan-tilt angle data and focal length adjustment data corresponding to the historical video frames, map the pixel values of multiple historical video frames in the same group to the corresponding pixel positions on a blank panoramic canvas;

[0056] The blank panoramic canvas stores all pixel values mapped to the corresponding pixel positions;

[0057] Calculate the mean and standard deviation of the cumulative pixel values at each corresponding pixel position on the blank panoramic canvas;

[0058] Calculate the pixel deviation index of the cumulative pixel values, compare the pixel deviation index with the pixel deviation threshold, and remove the pixel values that exceed the pixel deviation threshold;

[0059] Based on the cumulative pixel values at each pixel position, use a Gaussian mixture model to model the multimodal distribution at each corresponding pixel position;

[0060] By integrating the main Gaussian components at all pixel positions, construct a panoramic background dynamic image. For the pixel positions on the panoramic background dynamic image that are not mapped to pixel values, use a predefined special value to represent the pixel missing at that position;

[0061] Based on the panoramic background dynamic images of multiple time periods, obtain the panoramic background model of the surveillance camera.

[0062] Preferably, the detection of whether the pan-tilt and zoom lens are faulty specifically includes:

[0063] The edge computing platform obtains the real-time operation data of each video camera. The real-time operation data includes the real-time video stream data of the surveillance camera at different time periods in the current unit cycle and the real-time device status data. The real-time device status data includes the real-time pan-tilt angle data and the real-time focal length adjustment data;

[0064] Based on the time period in which the acquired real-time video stream data is currently located, select the panoramic background dynamic image corresponding to the time period in the panoramic background model;

[0065] Obtain multiple real-time video frames from the real-time video stream data based on a preset acquisition interval;

[0066] Adjust the picture brightness and contrast of the multiple real-time video frames adaptively according to the histogram equalization standard;

[0067] Obtain the pan-tilt angle data and focal length adjustment data corresponding to each real-time video frame based on the real-time pan-tilt angle data and real-time focal length adjustment data;

[0068] Match the real-time video frame with the panoramic background dynamic image corresponding to the time period in the panoramic background model based on the timestamp information of the real-time video frame;

[0069] Calculate the perspective transformation parameters based on the pan-tilt angle and adjusted focal length of the real-time video frame, construct a homography matrix, calculate the theoretical edge contour corresponding to the real-time video frame in the panoramic background dynamic image, and extract the image within the theoretical edge contour of the panoramic background dynamic image as the first background image;

[0070] Calculate the matching feature points of the real-time video frame in the panoramic background dynamic image;

[0071] Based on the matching feature points of the real-time video frame on the panoramic background dynamic image, draw the actual edge contour of the real-time video frame on the panoramic background dynamic image, and extract the image within the actual edge contour of the panoramic background dynamic image as the second background image;

[0072] Record the first background image and the second background image obtained from each real-time video frame as a group;

[0073] Judge the positional relationship and pixel size of each group of the first background image and the second background image on the panoramic background dynamic image, and calculate the comprehensive position offset index and the comprehensive size deviation index;

[0074] Set an offset threshold and a deviation threshold. If the comprehensive position offset index is within the offset threshold, the current pan-tilt runs normally; otherwise, the pan-tilt runs faultily;

[0075] If the size deviation is within the deviation threshold, the current zoom lens runs normally; otherwise, the current zoom lens runs faultily;

[0076] If there is a pan-tilt running fault, take the pan-tilt running anomaly as the fault type, and upload the offset index as the anomaly index to the fault assessment model for fault assessment;

[0077] If there is a malfunction in the zoom lens operation, the abnormal operation of the zoom lens is taken as the suspected fault type, and the deviation index is uploaded as the suspected fault index to the fault assessment model for fault assessment;

[0078] If there are no pan-tilt malfunctions and no zoom lens malfunctions, the real-time video stream data obtained in the current unit cycle is recorded into the historical operation data; otherwise, the real-time video stream data obtained in the current unit cycle is recorded into the suspected fault operation data.

[0079] Preferably, the feedback of corresponding processing measures to the fault warning system based on the fault assessment coefficient includes:

[0080] Build a fault assessment model on the central cloud platform to obtain the suspected fault index and suspected fault type of each monitoring camera in the current unit cycle;

[0081] Obtain the weight parameters based on the suspected fault type, and the suspected fault types include abnormal brightness, abnormal contrast, color cast of the picture, blurred picture, abnormal noise, picture ghosting, picture flickering, picture tearing, picture freezing, picture delay, picture frame loss, abnormal operation of the zoom lens, and abnormal operation of the pan-tilt;

[0082] Calculate the fault assessment coefficient of each monitoring camera based on the weight parameter of the suspected fault type and the suspected fault index;

[0083] When the fault assessment coefficient is less than C1, it is recorded as low fault impact, and the video monitoring system continues to be monitored;

[0084] When the fault assessment coefficient is greater than C1 and less than C2, it is recorded as fault impact, feedback to the fault warning system, and a preliminary investigation and optimization of the video monitoring system are carried out based on the suspected fault type;

[0085] When the fault assessment coefficient is greater than C2, it is recorded as high fault impact, feedback to the fault warning system, and an immediate investigation and maintenance of the video monitoring system are carried out based on the suspected fault type;

[0086] The calculation formula of the fault assessment coefficient is:

[0087]

[0088] In the formula, H is the fault assessment coefficient, η l is the weight parameter of the l-th suspected fault type, L is the total amount of suspected fault types obtained, S l is the suspected fault index of the l-th suspected fault type.

[0089] The second aspect of the present invention provides an intelligent monitoring device for a video monitoring system based on cloud-edge collaboration, including:

[0090] An edge computing platform, which is used to be deployed on one side close to the monitoring camera group and is responsible for real-time data processing and analysis;

[0091] A central cloud platform, which is used to be responsible for global data analysis and fault assessment and provide feedback on processing measures;

[0092] A video monitoring system, which includes a gateway device, a fault warning system, multiple central switches, and multiple monitoring cameras. The central switches are connected to multiple monitoring cameras, aggregate video stream data from each camera, and transmit the aggregated data to the gateway device. The gateway device receives the video stream data from the central switches and forwards it to the edge computing platform;

[0093] Among them, the central cloud platform includes:

[0094] A fault assessment module, which is used to process the suspected fault index and suspected fault type from the edge computing platform: obtain weight parameters based on the suspected fault type; calculate the fault assessment coefficient of each monitoring camera based on the weight parameters of the suspected fault type and the suspected fault index;

[0095] A processing measure feedback module, which judges the degree of fault impact based on the magnitude of the fault assessment coefficient, gives feedback to the fault warning system, and takes corresponding processing measures.

[0096] Preferably, the edge computing platform includes:

[0097] A device information acquisition module, which is used to scan the network link of the video monitoring system using a network scanning tool, obtain the network topology structure and online device information, and based on the network topology structure, record multiple monitoring cameras connected to the same central switch as the same monitoring camera group;

[0098] A video data processing module, which is used to receive real-time video stream data from the monitoring cameras, decode the received real-time video stream data, convert it into processable image frames, and extract the real-time picture parameters of the video frames at a preset time interval;

[0099] A fault data processing module, which is used to obtain video fault data from the monitoring device system, decode the received fault video stream data, convert it into processable image frames, extract the picture parameters of the fault video frames at a preset time interval, calculate the correlation index between the picture parameters and the video fault type, and determine the relevant influence parameters;

[0100] The screen fault detection module is used to obtain the historical video stream data and historical device status data of the monitoring cameras at different time periods in the past N unit cycles from the monitoring device system, and calculate the average values of the screen parameters of each monitoring camera at different time periods; obtain the real-time video stream data and real-time device status data, extract the screen parameters of the real-time video stream data at a preset time interval, compare the real-time screen parameters with the average values of the screen parameters, and calculate the difference index; based on the real-time screen parameters, difference index, correlation index and related influence parameters, calculate the abnormal degree index of each video fault type. If the abnormal degree index is lower than the preset threshold, it is recorded as 0; otherwise, it is used as the suspected fault index, and the suspected fault type and suspected fault index are uploaded to the fault evaluation model.

[0101] The panoramic background model construction module creates a blank panoramic canvas based on the device parameters and constructs a panoramic background model based on the historical operation data;

[0102] The pan-tilt and zoom lens fault detection module obtains multiple real-time video frames in the real-time video stream data,

[0103] matches the real-time video frames with the panoramic background model, calculates the perspective transformation parameters, and constructs a homography matrix; extracts the images within the theoretical edge contour and the actual edge contour in the panoramic background model as the first background image and the second background image respectively; calculates the comprehensive position offset index and the comprehensive size deviation index, determines whether there are faults in the pan-tilt and zoom lens, and generates the suspected fault type and suspected fault index and uploads them to the fault evaluation model.

[0104] Compared with the prior art, the present invention provides an intelligent monitoring method and device for a video monitoring system based on cloud-edge collaboration, having the following beneficial effects:

[0105] The present invention provides an intelligent monitoring method and device for a video monitoring system based on cloud-edge collaboration. By deploying an edge computing platform on one side of the video monitoring system, the computing load of the central cloud platform is reduced; by combining video fault data, historical operation data and real-time operation data, the correlation index between the screen parameters and the video fault type is calculated to accurately determine the cause of the video screen fault; a panoramic canvas is constructed based on the device parameters, and a panoramic background model is constructed based on the historical operation data; the real-time operation data is combined with the panoramic background model to continuously monitor the faults of the pan-tilt and zoom lens; compared with the monitoring of the traditional video monitoring system, this method can analyze the real-time operation state data based on the panoramic background model to judge the operation state of the pan-tilt and zoom lens, discover potential faults in time, and avoid monitoring blind spots or picture loss caused by the failure of the video monitoring system; and the fault evaluation platform receives the detected suspected fault data and suspected fault index and timely feedbacks them to the fault alarm system for corresponding processing measures to ensure the reliability of the monitoring camera system. Brief Description of the Drawings

[0106] Figure 1 It is a flowchart of an intelligent monitoring method for a video surveillance system based on cloud-edge collaboration according to the present invention;

[0107] Figure 2 It is a block diagram of an intelligent monitoring device for a video surveillance system based on cloud-edge collaboration according to the present invention;

[0108] Figure 3 It is a flowchart for detecting whether there are faults in the operation of the detection pan-tilt and the zoom lens provided by the present invention. Detailed Embodiments

[0109] In order to make the technical means, creative features, achieved purposes and effects of the present invention easy to understand, the following will further elaborate on the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.[[ID=IS]]

[0110] Aiming at the deficiencies of the prior art, such as Figure 1 、 Figure 2 、 Figure 3 as shown, the present invention provides an intelligent monitoring method for a video surveillance system based on cloud-edge collaboration, including:

[0111] Obtain the online device information of the monitoring device system, and record multiple monitoring cameras connected to the same central switch as the same monitoring camera group based on the online device information;

[0112] Construct multiple edge computing platforms, and the edge computing platforms obtain the video fault data of the monitoring camera group, and obtain the correlation index between the picture parameters and the video fault type and the relevant influence parameters based on the video fault data;

[0113] The edge computing platform obtains the historical operation data and real-time operation data of the corresponding monitoring camera group from the monitoring device system, calculates the average value of the picture parameters of each monitoring camera at different time periods based on the historical operation data, obtains the real-time picture parameters based on the real-time operation data, and calculates the difference index of each real-time picture parameter based on the real-time picture parameters and the average value of the picture parameters;

[0114] Build a fault assessment model. Based on real-time video parameters, difference indices, correlation indices, and relevant influence parameters of each video fault type, calculate the abnormality degree index of each video fault type in the real-time video stream data. If the abnormality degree index is lower than the preset degree threshold, record the abnormality degree index as 0; otherwise, use the abnormality degree index as the suspected fault index, record the video fault type corresponding to the abnormality degree index as the suspected fault type, and upload it to the fault assessment model.

[0115] The edge computing platform obtains the device parameters of each monitoring camera in the corresponding monitoring camera group and creates a blank panoramic canvas based on the device parameters.

[0116] Obtain the historical video stream data and historical device status data of the monitoring camera at different time periods in the past N unit cycles in the historical operation data, and build a panoramic background model of the monitoring camera.

[0117] Obtain the real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle in the real-time operation data.

[0118] Extract the first background image on the panoramic background model based on the real-time device status data, and extract the second background image on the panoramic background model based on the real-time video stream data.

[0119] Analyze and process the first background image and the second background image to obtain the comprehensive position offset index and the comprehensive size deviation index, and detect whether there are faults in the pan-tilt operation and zoom lens; if there are faults, generate the suspected fault type and the suspected fault index, and upload them to the fault assessment model.

[0120] Input the suspected fault index and the suspected fault type into the built fault assessment model, calculate the fault assessment coefficient of each monitoring camera in the current unit cycle, feedback to the fault warning system based on the fault assessment coefficient, and take corresponding treatment measures.

[0121] The intelligent monitoring method of the video monitoring system based on cloud-edge collaboration in the present disclosure can be responsible for local data processing by deploying an edge computing platform on one side of the video monitoring system, reducing the computing load of the central cloud platform, achieving optimized utilization of resources and load balancing; by analyzing the correlation index and relevant influence parameters between the video parameters and video fault types through historical video data, it can more accurately identify different types of video fault types in the current cycle; based on the panoramic background model and intelligent algorithms, the system can more precisely detect faults in the pan-tilt and zoom lens; the central cloud platform obtains the suspected fault type and the suspected fault index of the edge computing platform, calculates the fault assessment coefficient based on the fault assessment model, generates and feedbacks corresponding treatment measures to the fault warning system to achieve efficient, intelligent, and reliable fault monitoring and response of the video monitoring system.

[0122] Specifically, the obtaining of the online device information of the monitoring device system further includes:

[0123] Use a network scanning tool to scan the network link of the video monitoring system to obtain the network topology structure of the network link;

[0124] Identify all online device information based on the network topology structure;

[0125] Based on the online device information, multiple monitoring cameras connected to the same central switch are recorded as the same monitoring camera group;

[0126] The central switch is connected to the gateway device through optical fiber, and the gateway device is connected to the network. The gateway device is used to receive the real-time operation data of each monitoring camera in the monitoring camera groups corresponding to multiple central switches, and store the historical operation data and video fault data of each monitoring camera in the monitoring camera groups. The historical operation data includes historical video stream data and historical device status data of the monitoring camera in different time periods in the past N unit cycles, and the real-time operation data includes real-time video stream data and real-time device status data of the monitoring camera in the current unit cycle.

[0127] Specifically, the obtaining of the correlation index and related influencing parameters between the picture parameters and the video fault type based on the video fault data specifically includes:

[0128] Obtain video fault data, which includes the fault video stream data and video fault types of each monitoring camera in each time period. The video fault types include but are not limited to abnormal brightness, abnormal contrast, color cast of the picture, blurred picture, abnormal noise, picture ghosting, picture flickering, picture tearing, picture freezing, picture delay, and picture frame loss;

[0129] Obtain the picture parameters of the fault video stream data at a preset time interval. The picture parameters include but are not limited to resolution, brightness, contrast, white balance, chromaticity, sharpness, signal-to-noise ratio, motion vector, and inter-frame difference;

[0130] Calculate the correlation between the picture parameters extracted from each fault video stream data and the video fault type according to the Pearson correlation coefficient, and calculate the correlation index to measure the linear correlation between the picture parameters and the video fault type;

[0131] Record the picture parameters corresponding to the correlation index in the interval (0.3, 1] as the related influencing parameters of the video fault type;

[0132] The calculation formula of the correlation index is:

[0133]

[0134] In the formula, R xy，Q is the correlation index of the screen parameter x and the video fault type y in the Qth time period, r xy,m，Q is the correlation between the screen parameter x and the video fault type y in the mth fault video stream data in the Qth time period, M is the total number of fault video stream data, x i is the observed value of the video screen parameter x in the ith sampling, y i is the sample value of the fault video stream data in the ith sampling, j is the total number of samplings, and are the average values of the screen parameter x and the video fault type y respectively, where the y i is in binary coding, 1 indicates the existence of the fault, and 0 indicates the non-existence;

[0135] Among them, the value range of R xy is from -1 to 1, r xy ∈[0.3, 1] indicates correlation, R xy ∈[0, 0.3) indicates no correlation;

[0136] Specifically, recording the video fault type corresponding to the abnormality degree index as the suspected fault type and uploading it to the fault evaluation model specifically includes:

[0137] Obtain the historical video stream data of each monitoring camera in different time periods in the past N unit cycles in the historical operation parameters, and obtain the screen parameters of the historical video stream data at a preset time interval;

[0138] Calculate the screen parameter mean value of the screen parameter x of each monitoring camera in different time periods

[0139] The edge computing platform obtains the real-time video stream data, obtains the time period where the current video stream data is located, and extracts the correlation index of the screen parameter x and the video fault type y in the time period;

[0140] Use OpenCV to read the real-time video stream data, obtain the real-time screen parameters of the real-time video stream data at a preset time interval, compare the real-time screen parameters with the screen parameter mean value, and calculate the difference index of each real-time screen parameter;

[0141] It should be noted that the difference index of the real-time screen parameter can be obtained by calculating the relative difference between the real-time screen parameter and the screen parameter mean value;

[0142] Based on the relevant impact parameters of each video fault type, analyze the real-time picture parameters of the real-time video stream data, the difference index and the correlation index of the real-time picture parameters, calculate the abnormality degree index of each video fault type in the real-time video stream data. If the abnormality degree index is lower than the preset degree threshold, record the abnormality degree index as 0; otherwise, retain the corresponding value of the abnormality degree index;

[0143] It should be noted that the degree threshold of each video fault type is set to

[0144] If there is at least one abnormality degree index, record the real-time video stream data obtained in the current unit cycle into the suspected fault operation data, use the obtained abnormality degree index as the suspected fault index, and record the video fault type corresponding to the abnormality degree index as the suspected fault type; otherwise, record the real-time video stream data obtained in the current unit cycle into the historical operation data;

[0145] Upload the suspected fault type and the corresponding suspected fault index to the fault evaluation model;

[0146] The calculation formula of the abnormality degree index is:

[0147]

[0148] In the formula, S y is the abnormality degree index of the video fault type y, is the correlation index of the k y th relevant impact parameter of the video fault type y, K y is the number of relevant impact parameters of the video fault type y, is the difference index of the real-time picture parameter corresponding to the bth and the k y th relevant impact parameters of the video fault type y, and B is the sampling number of the real-time picture parameters, that is, the sampling number of the real-time video frames in the real-time video stream data.

[0149] Specifically, the creation of the blank panoramic canvas based on the device parameters specifically includes:

[0150] The edge computing platform obtains the device parameters of each monitoring camera in the corresponding monitoring camera group, assigns a unique identifier to each monitoring camera, and the device parameters include device model, video resolution, pan rotation angle range, focal length adjustment range, and image sensor size;

[0151] Based on the device parameters, obtain the minimum focal length, the maximum horizontal rotation angle of the pan, and the maximum vertical rotation angle of the tilt of each monitoring camera, and obtain the maximum horizontal viewing angle and the maximum vertical viewing angle of the monitoring camera at a fixed angle based on the minimum focal length and the image sensor size;

[0152] Calculate the proportions of the maximum horizontal viewing angle and the maximum vertical viewing angle in the maximum horizontal rotation angle and the maximum vertical rotation angle of the pan-tilt respectively, denoted as b1 and b2 respectively;

[0153] Based on b1, b2 and the video resolution of each surveillance camera, create a blank panoramic canvas for each surveillance camera. The horizontal resolution of the blank panoramic canvas is b1·U h *b2·U v where U h and U v are the video horizontal resolution and the video vertical resolution respectively;

[0154] It should be noted that the calculation formulas for the horizontal viewing angle and the vertical viewing angle are:

[0155]

[0156] In the formula, θ i,h is the horizontal viewing angle of the i-th surveillance camera, θ i,v is the vertical viewing angle of the i-th surveillance camera, f i is the minimum focal length of the i-th surveillance camera, W i is the width of the image sensor of the i-th surveillance camera, H i is the height of the image sensor of the i-th surveillance camera;

[0157] In this way, obtain the minimum focal length f i，min and the image sensor size of the i-th surveillance camera, and calculate the maximum horizontal viewing angle θ i,h，max and the maximum vertical viewing angle θ i,v，max .

[0158] Specifically, the construction of the panoramic background model of the surveillance camera includes:

[0159] Obtain the historical video stream data and historical device status data of the surveillance camera at different time periods in the past N unit cycles in the historical operation data. The historical device status data includes historical pan-tilt angle data and historical focal length adjustment data;

[0160] Extract multiple historical video frames from the historical video stream data based on a preset acquisition interval;

[0161] Extract the pan-tilt angle data and focal length adjustment data corresponding to each historical video frame based on the historical pan-tilt angle data and historical focal length adjustment data;

[0162] Divide a day into multiple time periods. Based on the timestamp information of historical video frames, assign the historical video frames in the same time period in the past N unit cycles to a group;

[0163] Adjust the picture brightness and contrast of multiple historical video frames adaptively according to a preset histogram equalization criterion. It should be noted that the preset histogram equalization criterion can be dynamically adjusted according to the contrast and brightness of the extracted historical video frame picture, so as to reduce the influence of the current time and meteorological factors on the historical video frame picture;

[0164] Take the multiple historical video frames in each group as model training samples;

[0165] According to the pan-tilt angle data and focal length adjustment data corresponding to the historical video frames, map the pixel values of multiple historical video frames in the same group to the corresponding pixel positions on a blank panoramic canvas respectively;

[0166] The blank panoramic canvas stores all the pixel values mapped to the corresponding pixel positions;

[0167] Calculate the mean and standard deviation of the cumulative pixel values at each corresponding pixel position on the blank panoramic canvas;

[0168] Calculate the pixel deviation index of the cumulative pixel values, compare the pixel deviation index with the pixel deviation threshold, and remove the pixel values that exceed the pixel deviation threshold;

[0169] Based on the cumulative pixel values at each pixel position, use a Gaussian mixture model to model the multimodal distribution at each corresponding pixel position;

[0170] By integrating the main Gaussian components at all pixel positions, construct a panoramic background dynamic image. For the pixel positions on the panoramic background dynamic image that are not mapped to pixel values, use a predefined special value to represent the pixel missing at that position;

[0171] Based on the panoramic background dynamic images in multiple time periods, obtain the panoramic background model of the surveillance camera;

[0172] It should be noted that for the current pixel value, calculate the absolute difference between it and the mean, and divide it by the standard deviation to obtain the pixel deviation index;

[0173] The pixel deviation threshold uses a statistical method to set a threshold range of Z-Score. Pixel values exceeding this range are considered to be pixel values of abnormal or foreground images;

[0174] In the present disclosure, the panoramic background dynamic image updates the pixel values mapped to the panoramic canvas in the past N unit cycles every unit cycle, and updates the mean and variance at the corresponding pixel positions in the panoramic background dynamic image to ensure the adaptability and accuracy of the panoramic background dynamic image;

[0175] Specifically, detecting whether there are faults in the operation of the detection pan-tilt and the zoom lens specifically includes:

[0176] The edge computing platform obtains the real-time operation data of each video camera. The real-time operation data includes the real-time video stream data of the monitoring camera at different time periods within the current unit cycle and the real-time device status data. The real-time device status data includes the real-time pan-tilt angle data and the real-time focal length adjustment data;

[0177] Based on the time period in which the obtained real-time video stream data is located, select the panoramic background dynamic image corresponding to the time period in the panoramic background model;

[0178] Obtain multiple real-time video frames from the real-time video stream data based on the preset acquisition interval;

[0179] Adaptively adjust the picture brightness and contrast of multiple real-time video frames according to the histogram equalization standard. It should be noted that the preset histogram equalization standard can be dynamically adjusted according to the contrast and brightness of the extracted real-time video frame picture to reduce the influence of the current time and meteorological factors on the real-time video frame picture;

[0180] Obtain the pan-tilt angle data and the focal length adjustment data corresponding to each real-time video frame based on the real-time pan-tilt angle data and the real-time focal length adjustment data;

[0181] Match the real-time video frame with the panoramic background dynamic image corresponding to the time period in the panoramic background model based on the timestamp information of the real-time video frame;

[0182] Calculate the perspective transformation parameters based on the pan-tilt angle and the adjusted focal length of the real-time video frame, construct a homography matrix, calculate the theoretical edge contour corresponding to the real-time video frame in the panoramic background dynamic image, and extract the image within the theoretical edge contour of the panoramic background dynamic image as the first background image;

[0183] Calculate the matching feature points of the real-time video frame in the panoramic background dynamic image;

[0184] Based on the matching feature points of the real-time video frame on the panoramic background dynamic image, draw the actual edge contour of the real-time video frame on the panoramic background dynamic image, and extract the image within the actual edge contour of the panoramic background dynamic image as the second background image;

[0185] Record the first background image and the second background image obtained from each real-time video frame as a group;

[0186] Judge the positional relationship and pixel size of each group of the first background image and the second background image on the panoramic background dynamic image, and calculate the comprehensive position offset index and the comprehensive size deviation index;

[0187] It should be noted that the method for obtaining the comprehensive position offset index is as follows:

[0188] Obtain the pixel distance between the center point of the first background image and the center point of the second background image, and take the pixel distance;

[0189] Based on the real-time focal length adjustment data, obtain the adjusted focal length of the current real-time video frame, and divide the product of the pixel distance and the adjusted focal length by the minimum focal length as the position offset index;

[0190] Take the average value of the position offset indices of each real-time video frame extracted from the real-time video stream in the current unit cycle as the comprehensive position offset index;

[0191] The method for obtaining the comprehensive size deviation index is as follows:

[0192] Obtain the pixel sizes of the first background image and the second background image, calculate the relative difference between the pixel height of the first background image and the pixel height of the second background image, calculate the relative difference between the pixel width of the first background image and the pixel width of the second background image, and add the two relative differences as the size deviation index;

[0193] Take the average value of the size deviation indices of each real-time video frame extracted from the real-time video stream in the current unit cycle as the comprehensive size deviation index;

[0194] Set the offset threshold and the deviation threshold. If the comprehensive position offset index is within the offset threshold, the current pan-tilt unit operates normally; otherwise, the pan-tilt unit has a failure;

[0195] If the size deviation is within the deviation threshold, the current zoom lens operates normally; otherwise, the current zoom lens has a failure;

[0196] It should be noted that the comprehensive offset threshold is set to 0.3, and the comprehensive deviation threshold is set to 0.2. The comprehensive offset threshold and the comprehensive deviation threshold can be adjusted according to the actual situation;

[0197] If there is a failure in the operation of the pan-tilt unit, take the abnormal operation of the pan-tilt unit as the failure type, and upload the offset index as the abnormal index to the failure assessment model for failure assessment;

[0198] If there is a failure in the operation of the zoom lens, take the abnormal operation of the zoom lens as the suspected failure type, and upload the deviation index as the suspected failure index to the failure assessment model for failure assessment;

[0199] If there are no failures in the operation of the pan-tilt unit and the zoom lens, record the real-time video stream data obtained in the current unit cycle into the historical operation data; otherwise, record the real-time video stream data obtained in the current unit cycle into the suspected failure operation data.

[0200] Specifically, the feedback of corresponding processing measures to the fault warning system based on the fault evaluation coefficient includes:

[0201] Build a fault evaluation model on the central cloud platform to obtain the suspected fault index and suspected fault type of each monitoring camera in the current unit cycle;

[0202] Obtain weight parameters based on the suspected fault type, and the suspected fault type includes abnormal brightness, abnormal contrast, color deviation of the picture, blurred picture, abnormal noise, picture ghosting, picture flickering, picture tearing, picture freezing, picture delay, picture frame loss, abnormal operation of the zoom lens, and abnormal operation of the pan-tilt head;

[0203] Calculate the fault evaluation coefficient of each monitoring camera based on the weight parameter of the suspected fault type and the suspected fault index;

[0204] When the fault evaluation coefficient is less than C1, it is recorded as low fault impact, and continue to monitor the video monitoring system;

[0205] When the fault evaluation coefficient is greater than C1 and less than C2, it is recorded as medium fault impact, feedback to the fault warning system, and conduct a preliminary investigation and optimization of the video monitoring system based on the suspected fault type;

[0206] When the fault evaluation coefficient is greater than C2, it is recorded as high fault impact, feedback to the fault warning system, and immediately conduct an investigation and maintenance of the video monitoring system based on the suspected fault type;

[0207] The calculation formula of the fault evaluation coefficient is:

[0208]

[0209] In the formula, H is the fault evaluation coefficient, η l is the weight parameter of the l-th suspected fault type, L is the total amount of suspected fault types obtained, and S l is the suspected fault index of the l-th suspected fault type;

[0210] It should be noted that C1 and C2 are reasonably set based on the distribution of the fault evaluation coefficient, and the weight parameters of the suspected fault type are obtained through expert scoring;

[0211] After investigation, determine the fault type. If the fault type is a video fault type, upload the suspected fault data and the corresponding fault type as video fault data to the gateway device.

[0212] The second aspect of the present invention provides an intelligent monitoring device for a video monitoring system based on cloud-edge collaboration, including:

[0213] An edge computing platform, which is used to be deployed on one side close to the monitoring camera group and is responsible for real-time data processing and analysis;

[0214] A central cloud platform, which is used to be responsible for global data analysis and fault assessment, and provide feedback on processing measures;

[0215] A video monitoring system, which includes a gateway device, a fault warning system, multiple central switches, and multiple monitoring cameras. The central switches are connected to multiple monitoring cameras, converge the video stream data from each camera, and transmit the converged data to the gateway device. The gateway device receives the video stream data from the central switches and forwards it to the edge computing platform;

[0216] Among them, the central cloud platform includes:

[0217] A fault assessment module, which is used to process the suspected fault index and suspected fault type from the edge computing platform: obtain weight parameters based on the suspected fault type; calculate the fault assessment coefficient of each monitoring camera based on the weight parameter of the suspected fault type and the suspected fault index;

[0218] A processing measure feedback module, which judges the degree of fault impact based on the size of the fault assessment coefficient, gives feedback to the fault warning system, and takes corresponding processing measures.

[0219] Specifically, the edge computing platform includes:

[0220] A device information acquisition module, which is used to scan the network link of the video monitoring system using a network scanning tool, obtain the network topology structure and online device information, and based on the network topology structure, record multiple monitoring cameras connected to the same central switch as the same monitoring camera group;

[0221] A video data processing module, which is used to receive real-time video stream data from the monitoring cameras, decode the received real-time video stream data, convert it into processable image frames, and extract the real-time picture parameters of the video frames at a preset time interval;

[0222] A fault data processing module, which is used to obtain video fault data from the monitoring device system, decode the received fault video stream data, convert it into processable image frames, extract the picture parameters of the fault video frames at a preset time interval, calculate the correlation index between the picture parameters and the video fault type, and determine the relevant influence parameters;

[0223] The screen fault detection module is used to obtain the historical video stream data and historical device status data of the monitoring cameras at different time periods in the past N unit cycles from the monitoring device system, and calculate the average values of the screen parameters of each monitoring camera at different time periods; obtain the real-time video stream data and real-time device status data, extract the screen parameters of the real-time video stream data at a preset time interval, compare the real-time screen parameters with the average values of the screen parameters, and calculate the difference index; based on the real-time screen parameters, difference index, correlation index and related influencing parameters, calculate the abnormal degree index of each video fault type. If the abnormal degree index is lower than the preset threshold, it is recorded as 0; otherwise, it is used as a suspected fault index, and the suspected fault type and suspected fault index are uploaded to the fault assessment model.

[0224] The panoramic background model construction module creates a blank panoramic canvas based on the device parameters and constructs a panoramic background model based on the historical operation data;

[0225] The pan-tilt and zoom lens fault detection module obtains multiple real-time video frames in the real-time video stream data,

[0226] matches the real-time video frames with the panoramic background model, calculates the perspective transformation parameters, and constructs a homography matrix; extracts the images within the theoretical edge contour and the actual edge contour in the panoramic background model as the first background image and the second background image respectively; calculates the comprehensive position offset index and the comprehensive size deviation index, determines whether there are faults in the pan-tilt and zoom lens, and generates the suspected fault type and suspected fault index and uploads them to the fault assessment model.

[0227] In summary, the advantages of the present invention are as follows: The present invention provides an intelligent monitoring method and device for a video monitoring system based on cloud-edge collaboration, which reduces the computing load of the central cloud platform by deploying an edge computing platform on one side of the video monitoring system; by combining video fault data, historical operation data and real-time operation data, calculates the correlation index between the screen parameters and the video fault types to accurately determine the cause of the video screen fault; constructs a panoramic canvas based on the device parameters and constructs a panoramic background model based on the historical operation data; combines the real-time operation data with the panoramic background model to continuously monitor the faults of the pan-tilt and zoom lens; the fault assessment platform receives the detected suspected fault data and suspected fault index and timely feedbacks them to the fault alarm system for corresponding processing measures, which can ensure the reliability of the monitoring camera system.

[0228] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification only illustrate the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed.

Claims

1. An intelligent monitoring method for a video surveillance system based on cloud-edge collaboration, characterized in that: include: Obtain online device information of the monitoring device system, and based on the online device information, record multiple monitoring cameras connected to the same central switch as the same monitoring camera group; Construct multiple edge computing platforms, wherein the edge computing platforms obtain video fault data of the monitoring camera group, and obtain the correlation index between the image parameters and the video fault type and related influencing parameters based on the video fault data; The edge computing platform obtains historical and real-time operating data of the corresponding surveillance camera group from the monitoring equipment system. Based on the historical operating data, it calculates the average image parameters of each surveillance camera in different time periods. Based on the real-time operating data, it obtains real-time image parameters. Based on the real-time image parameters and the average image parameters, it calculates the difference index of each real-time image parameter. A fault assessment model is constructed. Based on real-time image parameters, difference index, correlation index, and relevant influencing parameters of each video fault type, the abnormality index of each video fault type in the real-time video stream data is calculated. If the abnormality index is lower than the preset threshold, the abnormality index is recorded as 0; otherwise, the abnormality index is used as a suspected fault index, and the video fault type corresponding to the abnormality index is recorded as a suspected fault type and uploaded to the fault assessment model. The edge computing platform obtains the device parameters of each surveillance camera in the corresponding surveillance camera group and creates a blank panoramic canvas based on the device parameters; Obtain historical video stream data and historical device status data of surveillance cameras in different time periods over the past N unit cycles from historical operation data, and build a panoramic background model of surveillance cameras; Obtain real-time video stream data and real-time device status data of the surveillance camera in the current unit cycle in real-time operation data; Extracting a first background image on the panoramic background model based on the real-time device status data, and extracting a second background image on the panoramic background model based on the real-time video stream data; Analyze and process the first and second background images to obtain a comprehensive position offset index and a comprehensive size deviation index, and detect whether there are any faults in the pan / tilt operation and zoom lens. If a fault is present, generate a suspected fault type and suspected fault index, and upload them to the fault assessment model. The suspected fault index and suspected fault type are input into the fault assessment model to calculate the fault assessment coefficient of each surveillance camera in the current unit cycle. Based on the fault assessment coefficient, feedback is given to the fault alarm system and corresponding processing measures are taken.

2. The intelligent monitoring method for a video surveillance system based on cloud-edge collaboration according to claim 1 is characterized in that: The obtaining of online device information of the monitoring device system further includes: Use a network scanning tool to scan the network link of the video surveillance system to obtain the network topology structure of the network link; Identify all online device information based on network topology; Based on online device information, multiple surveillance cameras connected to the same central switch are recorded as the same surveillance camera group; The central switch is connected to the gateway device via optical fiber, and the gateway device is connected to the network. The gateway device is used to receive the real-time operation data of each surveillance camera in the surveillance camera group corresponding to multiple central switches, and store the historical operation data and video fault data of each surveillance camera in the surveillance camera group. The historical operation data includes the historical video stream data and historical device status data of the surveillance camera in different time periods in the past N unit cycles, and the real-time operation data includes the real-time video stream data and real-time device status data of the surveillance camera in the current unit cycle.

3. The intelligent monitoring method for a video surveillance system based on cloud-edge collaboration according to claim 2 is characterized in that: The obtaining of the correlation index between the image parameters and the video fault type and related influencing parameters based on the video fault data specifically includes: Obtain video fault data, which includes fault video stream data and video fault types of each surveillance camera in each time period. The video fault types include abnormal brightness, abnormal contrast, color cast, blurred image, abnormal noise, ghosting, flickering, tearing, freezing, delay, and frame loss. Obtaining picture parameters of the fault video stream data at preset time intervals, wherein the picture parameters include resolution, brightness, contrast, white balance, chroma, sharpness, signal-to-noise ratio, motion vector, and inter-frame difference; The correlation between the image parameters extracted from each fault video stream data and the video fault type is calculated based on the Pearson correlation coefficient, and the correlation index is calculated to measure the linear correlation between the image parameters and the video fault type; The picture parameters corresponding to the correlation index in the interval (0.3, 1] are recorded as the relevant influencing parameters of the video fault type; The calculation formula of the correlation index is: Where R xy，Q is the correlation index between the image parameter x and the video fault type y in the Qth time period, r xy,m,Q is the correlation between the image parameter x and the video fault type y in the mth fault video stream data in the Qth time period, M is the total number of fault video stream data, x i is the observed value of the video image parameter x in the i-th sampling, y i is the sample value of the fault video stream data in the i-th sampling, j is the total number of sampling times, and are the average values of the picture parameter x and the video fault type y, respectively, wherein the y i It is a binary code, 1 indicates the fault exists, 0 indicates it does not exist; Among them, R xy The value range is -1 to 1, r xy ∈[0.3,1] indicates correlation, R xy ∈[0, 0.3) means no correlation.

4. The intelligent monitoring method for a video surveillance system based on cloud-edge collaboration according to claim 3 is characterized in that: The video fault type corresponding to the abnormality index is recorded as a suspected fault type and uploaded to the fault assessment model, specifically including: Obtain historical video stream data of each surveillance camera in different time periods in the past N unit cycles in the historical operation parameters, and obtain image parameters of the historical video stream data at preset time intervals; Calculate the mean value of the image parameter x of each surveillance camera in different time periods The edge computing platform obtains real-time video stream data, obtains the time period of the current video stream data, and extracts the correlation index between the image parameter x and the video fault type y in the time period; Use OpenCV to read real-time video stream data, obtain real-time picture parameters of real-time video stream data at preset time intervals, compare the real-time picture parameters with the picture parameter mean, and calculate the difference index of each real-time picture parameter; Based on the relevant influencing parameters of each video fault type, the real-time picture parameters of the real-time video stream data, the difference index of the real-time picture parameters, and the correlation index are analyzed to calculate the abnormality degree index of each video fault type in the real-time video stream data. If the abnormality degree index is lower than a preset degree threshold, the abnormality degree index is recorded as 0; otherwise, the abnormality degree index retains the corresponding value; If there is at least one abnormality index, the real-time video stream data obtained in the current unit period is recorded in the suspected fault operation data, the obtained abnormality index is used as the suspected fault index, and the video fault type corresponding to the abnormality index is recorded as the suspected fault type; otherwise, the real-time video stream data obtained in the current unit period is recorded in the historical operation data; Upload the suspected fault type and the corresponding suspected fault index to the fault assessment model; The calculation formula of the abnormality index is: Where S y is the abnormality index of video fault type y, is the kth video fault type y y The correlation index of the relevant influencing parameters, K y is the number of relevant influencing parameters of video fault type y, For the bth and kth y The difference index of the real-time picture parameters corresponding to the relevant influencing parameters of the video fault type y is represented by B, which is the sampling number of the real-time picture parameters, that is, the sampling number of the real-time video frame in the real-time video stream data.

5. The intelligent monitoring method for a video surveillance system based on cloud-edge collaboration according to claim 4 is characterized in that: The process of creating a blank panoramic canvas based on device parameters specifically includes: The edge computing platform obtains the device parameters of each surveillance camera in the corresponding surveillance camera group and assigns a unique identifier to each surveillance camera. The device parameters include device model, video resolution, pan / tilt rotation angle range, focal length adjustment range, and image sensor size. Based on the device parameters, obtain the minimum focal length, maximum horizontal rotation angle of the pan / tilt, and maximum vertical rotation angle of each surveillance camera. Based on the minimum focal length and image sensor size, obtain the maximum horizontal viewing angle and maximum vertical viewing angle of the surveillance camera at a fixed angle. Calculate the ratio of the maximum horizontal viewing angle and the maximum vertical viewing angle to the maximum horizontal rotation angle and the maximum vertical rotation angle of the gimbal, respectively, and record them as b1 and b2; Based on b1, b2 and video resolution of each surveillance camera, a blank panoramic canvas is created for each surveillance camera.

6. The intelligent monitoring method for a video surveillance system based on cloud-edge collaboration according to claim 5 is characterized in that: The method of constructing a panoramic background model of a surveillance camera includes: Obtain historical video stream data and historical device status data of surveillance cameras in different time periods within the past N unit cycles in historical operation data, wherein the historical device status data includes historical pan / tilt angle data and historical focal length adjustment data; Extracting multiple historical video frames from the historical video stream data based on a preset acquisition interval; Based on the historical pan / tilt angle data and the historical focus adjustment data, the pan / tilt angle data and the focus adjustment data corresponding to each historical video frame are extracted; Divide a day into multiple time periods, and assign historical video frames of the same time period in the past N unit periods to a group based on the timestamp information of the historical video frames; Adaptively adjust the brightness and contrast of multiple historical video frames according to a preset histogram equalization standard; Use multiple historical video frames of each group as model training samples; According to the pan / tilt angle data and focus adjustment data corresponding to the historical video frames, the pixel values of multiple historical video frames in the same group are mapped to the corresponding pixel positions of the blank panoramic canvas; The blank panorama canvas stores all pixel values mapped to corresponding pixel locations; Calculate the mean and standard deviation of the accumulated pixel values at each corresponding pixel position on the blank panoramic canvas; Calculating a pixel deviation index of the accumulated pixel values, comparing the pixel deviation index with a pixel deviation threshold, and removing pixel values exceeding the pixel deviation threshold; Based on the cumulative pixel values at each pixel location, a Gaussian mixture model is used to model the multimodal distribution of each corresponding pixel location; A panoramic background dynamic image is constructed by integrating the main Gaussian components of all pixel positions, and for pixel positions in the panoramic background dynamic image that are not mapped to pixel values, a predefined special value is used to indicate pixel absence at the position; Based on the panoramic background dynamic images of multiple time periods, a panoramic background model of the surveillance camera is obtained.

7. The intelligent monitoring method for a video surveillance system based on cloud-edge collaboration according to claim 6 is characterized in that: The detection of whether the pan / tilt operation and the zoom lens have faults specifically includes: The edge computing platform obtains real-time operating data of each video camera, including real-time video stream data and real-time device status data of the surveillance camera at different time periods within the current unit cycle, including real-time pan / tilt angle data and real-time focus adjustment data; Based on the current time period of the acquired real-time video stream data, a panoramic background dynamic image of the corresponding time period in the panoramic background model is selected; Acquire multiple real-time video frames in the real-time video stream data based on a preset acquisition interval; Adaptively adjust the brightness and contrast of multiple real-time video frames according to the histogram equalization standard; Acquire the pan / tilt angle data and focus adjustment data corresponding to each real-time video frame based on the real-time pan / tilt angle data and the real-time focus adjustment data; Matching the real-time video frame with the panoramic background dynamic image of the corresponding time period in the panoramic background model based on the timestamp information of the real-time video frame; Calculate the perspective transformation parameters based on the pan / tilt angle and adjusted focal length of the real-time video frame, construct a homography matrix, calculate the theoretical edge contour corresponding to the real-time video frame in the panoramic background dynamic image, and extract the image within the theoretical edge contour of the panoramic background dynamic image as the first background image; Calculate the matching feature points of real-time video frames in the panoramic background dynamic image; Based on the matching feature points of the real-time video frame on the panoramic background dynamic image, the actual edge contour of the real-time video frame is drawn on the panoramic background dynamic image, and the image of the panoramic background dynamic image within the actual edge contour is extracted as the second background image; Determine the positional relationship and pixel size of each set of the first background image and the second background image on the panoramic background dynamic image, and calculate a comprehensive position offset index and a comprehensive size deviation index; set an offset threshold and a deviation threshold; if the comprehensive position offset index is within the offset threshold, the current pan-tilt operation is normal; otherwise, the pan-tilt operation is faulty; If the size deviation is within the deviation threshold, the current zoom lens is operating normally; otherwise, the current zoom lens is operating abnormally; If there is a gimbal operation fault, the gimbal operation abnormality is regarded as the fault type, and the offset index is uploaded as the abnormality index to the fault assessment model for fault assessment; If there is a zoom lens operation failure, the zoom lens operation abnormality is regarded as a suspected fault type, and the deviation index is uploaded as a suspected fault index to the fault assessment model for fault assessment; If there is no pan / tilt operation failure or zoom lens operation failure, the real-time video stream data obtained in the current unit period will be recorded in the historical operation data; otherwise, the real-time video stream data obtained in the current unit period will be recorded in the suspected failure operation data.

8. The intelligent monitoring method for a video surveillance system based on cloud-edge collaboration according to claim 7 is characterized in that: Feedback of corresponding processing measures to the fault warning system based on the fault assessment coefficient includes: Build a fault assessment model on the central cloud platform to obtain the suspected fault index and suspected fault type of each surveillance camera in the current unit cycle; Obtaining weight parameters based on suspected fault types, wherein the suspected fault types include abnormal brightness, abnormal contrast, color cast, blurred image, abnormal noise, ghosting, flickering, tearing, freezing, delay, frame loss, abnormal zoom lens operation, and abnormal pan / tilt operation; Calculate the fault assessment coefficient of each surveillance camera based on the weight parameter of the suspected fault type and the suspected fault index; When the fault assessment coefficient is less than C1, it is recorded as a low fault impact and the video surveillance system continues to be monitored; When the fault assessment coefficient is greater than C1 and less than C2, it is recorded as a medium fault impact and fed back to the fault alarm system. Based on the suspected fault type, a preliminary investigation and optimization of the video surveillance system is conducted; When the fault assessment coefficient is greater than C2, it is recorded as a high fault impact and fed back to the fault alarm system. The video surveillance system is immediately investigated and maintained based on the suspected fault type. The calculation formula of the fault assessment coefficient is: Where H is the fault assessment coefficient, η l is the weight parameter of the lth suspected fault type, L is the total number of suspected fault types obtained, S l is the suspected fault index of the lth suspected fault type.

9. An intelligent monitoring device for a video surveillance system based on cloud-edge collaboration, used to implement the monitoring method according to any one of claims 1 to 8, characterized in that: include: The edge computing platform is deployed near the surveillance camera group and is responsible for real-time data processing and analysis; The central cloud platform is responsible for global data analysis and fault assessment, and provides feedback on treatment measures; A video surveillance system includes a gateway device, a fault alarm system, multiple central switches, and multiple surveillance cameras. The central switch connects to the multiple surveillance cameras, aggregates video stream data from each camera, and transmits the aggregated data to the gateway device. The gateway device receives the video stream data from the central switch and forwards it to the edge computing platform. The central cloud platform includes: The fault assessment module is used to process the suspected fault index and suspected fault type from the edge computing platform: obtain weight parameters based on the suspected fault type; and calculate the fault assessment coefficient of each surveillance camera based on the weight parameters and suspected fault index of the suspected fault type; The treatment measure feedback module determines the degree of fault impact based on the size of the fault assessment coefficient, provides feedback to the fault alarm system, and takes corresponding treatment measures.

10. The intelligent monitoring device for a video surveillance system based on cloud-edge collaboration according to claim 9, characterized in that: The edge computing platform includes: The device information acquisition module is used to use a network scanning tool to scan the network link of the video surveillance system to obtain the network topology and online device information. Based on the network topology, multiple surveillance cameras connected to the same central switch are recorded as the same surveillance camera group; The video data processing module is used to receive real-time video stream data from the surveillance camera, decode the received real-time video stream data, convert it into processable image frames, and extract real-time image parameters of the video frames at preset time intervals; The fault data processing module is used to obtain video fault data from the monitoring device system, decode the received fault video stream data, convert it into processable image frames, extract the image parameters of the fault video frames at preset time intervals, calculate the correlation index between the image parameters and the video fault type, and determine the relevant influencing parameters; The image fault detection module is used to obtain historical video stream data and historical device status data of surveillance cameras in different time periods over the past N unit cycles from the monitoring device system, and calculate the average image parameters of each surveillance camera in different time periods; obtain real-time video stream data and real-time device status data, extract the image parameters of the real-time video stream data at preset time intervals, compare the real-time image parameters with the average image parameters, and calculate the difference index; based on the real-time image parameters, difference index, correlation index, and related influencing parameters, calculate the abnormality index of each video fault type. If the abnormality index is lower than the preset threshold, it is recorded as 0; otherwise, it is used as a suspected fault index and the suspected fault type is uploaded to the fault assessment model; Panoramic background model building module, which creates a blank panoramic canvas based on device parameters and builds a panoramic background model based on historical operation data; The pan / tilt and zoom lens fault detection module obtains multiple real-time video frames from the real-time video stream data, matches the real-time video frames with the panoramic background model, calculates the perspective transformation parameters, and constructs the homography matrix; The images within the theoretical edge contour and the actual edge contour in the panoramic background model are extracted as the first background image and the second background image, respectively. The comprehensive position offset index and the comprehensive size deviation index are calculated to determine whether there are faults in the gimbal and zoom lens, and the suspected fault type and suspected fault index are generated.

Images