Image processing system and method applied to intelligent video monitoring

Abstract

The invention discloses an image processing system and method applied to intelligent video monitoring, and relates to the technical field of video image processing, and the method comprises the following steps: firstly, capturing video data through a monitoring camera, analyzing the obtained video data, detecting the change between video frames, and determining the change between the video frames; marking video segments which are still for a long time or have no obvious change in pictures; carrying out quantitative analysis on the definition of each frame of image in the video segment, and marking the image as a blurred image when the image blurring degree exceeds a preset threshold value; then combining a static and change result and a fuzzy evaluation result, automatically screening out redundant video segments, and carrying out hiding processing and compression storage on the redundant video segments; and finally, carrying out key frame extraction on the hidden video data and generating and storing a video abstract.

Description

Technical Field

[0001] The present invention relates to the technical field of video image processing, and in particular to an image processing system and method applied to intelligent video monitoring. Background Art

[0002] In the field of intelligent video surveillance, with the popularization of high-definition cameras and the continuous expansion of surveillance networks, the amount of video data generated every day is exploding. Although these massive data contain rich security monitoring information, they are also mixed with a large amount of redundant content that is of no substantial help for analysis, such as long static scenes, blurred or low-quality images, etc. This not only greatly increases the cost of data storage and transmission, but also brings considerable challenges to subsequent video retrieval and analysis.

[0003] Traditional video processing technologies, such as simple video compression, can alleviate storage pressure to a certain extent, but they often sacrifice video clarity and are difficult to meet the needs of high-definition video surveillance for capturing details. In addition, when faced with video content that can easily last for hours or even dozens of hours, manual browsing is not only time-consuming and laborious, but also prone to omission of important information due to fatigue. Therefore, intelligent video surveillance technology has begun to explore the automatic generation of video summaries. As an efficient way to express video content, video summaries can present the core information of the video in a brief form by extracting key frames or fragments from the video, greatly facilitating users' quick browsing and retrieval. However, most existing video summarization technologies rely on complex algorithm models or manually set rules. When processing surveillance videos containing rich scene changes and complex events, they often find it difficult to accurately identify and extract the most representative content. Summary of the invention

[0004] The purpose of the present invention is to provide an image processing system and method for intelligent video surveillance to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] An image processing method applied to intelligent video surveillance, characterized in that the method comprises the following steps:

[0007] Step S100: Capturing video data through a surveillance camera;

[0008] Step S200: Analyze the acquired video data, detect changes between video frames, and mark video segments that are static for a long time or have no obvious changes in the picture;

[0009] Step S300: quantifying and analyzing the clarity of each frame of the video segment, and marking the image as a blurred image when the blurriness of the image exceeds a preset threshold;

[0010] Step S400: combining the static and changing results and the fuzzy evaluation results, automatically screening out the redundant video segments and performing hidden processing and compressed storage on them;

[0011] Step S500: extract key frames from the hidden video data and generate a video summary for storage.

[0012] The step S200 includes the following steps:

[0013] Step S201 reads the acquired video data frame by frame and performs preprocessing, and then calculates the pixel difference between two adjacent frames according to the formula:

[0014]

[0015] Where D(t) represents the difference between the tth frame and the t-1th frame, M and N are the width and height of the image respectively, P(x,y,t) represents the pixel value of the tth frame image at the coordinate (x,y), P(x,y,t-1) represents the pixel value of the t-1th frame image at the coordinate (x,y), and a frame difference threshold K is set;

[0016] Step S202 determines the static segment based on the inter-frame difference D(t) and the time threshold T, sets a time window W, and calculates the average inter-frame difference D in the window. avg , according to the formula:

[0017]

[0018] where t 0 is the starting frame of the time window; if the average difference between frames D avg If the time interval is less than the set threshold value K and the duration exceeds the set time threshold T, the time period is marked as a static period and S is set. j A flag indicating whether the jth frame is a still frame.

[0019] In the above technical solution, the acquired video data is analyzed, the changes between video frames are detected, and the video segments that are still for a long time or have no obvious changes in the picture are marked. This step can effectively identify the redundant parts in the video, and by calculating the pixel difference between two adjacent frames and setting the frame difference threshold and time threshold, it can accurately determine which video segments are still or have little change, which can significantly improve the efficiency of video processing, reduce the pressure of storage and transmission, and also help to improve the accuracy of video retrieval and analysis.

[0020] The step S300 includes the following steps:

[0021] Step S301 traverses each frame of the video segment and uses the Laplace gradient method to perform a clarity quantitative evaluation on each frame of the image according to the formula:

[0022] L(x,y)=|f(x+1,y)+f(x-1,y)-2f(x,y)|+|f(x,y+1)+f(x,y-1)-2f(x,y)|;

[0023] Where L(x,y) represents the Laplace gradient value of the image at the coordinate (x,y), and f(x,y) represents the pixel value of the image at the coordinate (x,y);

[0024] Step S302 calculates the overall image clarity value according to the formula:

[0025]

[0026] Where F represents the overall clarity value of the image; a blur threshold P is set. When the overall clarity of the image is lower than this threshold, the image is marked as blurry. R is set j A flag indicating whether the jth frame is a blurry frame.

[0027] In the above technical solution, the clarity of each frame in the video segment is quantitatively analyzed and blurred images are marked. This step can ensure that each frame in the video has sufficient clarity to meet the needs of subsequent analysis and processing. By using the Laplace gradient method to quantitatively evaluate the clarity of the image and setting the blur threshold, the system can accurately identify blurred images, thereby avoiding incorporating low-quality content into subsequent processing procedures, improving the overall quality of the video, reducing misjudgments or omissions caused by image blur, and also helping to optimize the efficiency of video storage and transmission.

[0028] The step S400 includes the following steps:

[0029] Step S401 traverses all frames and automatically filters out redundant video segments according to the filtering formula:

[0030]

[0031] Where H represents the preset continuous frame number threshold, S j and R j The value rule of is: if the jth frame is marked as a still frame, then when S j =1, otherwise S j = 0; if the jth frame is marked as a blurred frame, then when R j =1, otherwise R j=0; It means traversing all frames in the video, starting from the i-th frame, calculating the total number of still frames and blurred frames in H consecutive frames. If the total number is greater than or equal to H, the video from the i-th frame to the i+H-1-th frame is regarded as a redundant video segment mark and these frames are marked as redundant frames;

[0032] Step S402: for the video segments marked as redundant frames, during the video playback, the system will automatically skip these frames according to the frame marking information and display a prompt message;

[0033] In step S403, after the hiding process, the redundant video segments are individually compressed and uploaded to the cloud platform for storage.

[0034] In the above technical solution, by setting a filtering formula, redundant video segments are automatically identified and marked, and these frames are automatically skipped during video playback, while a short prompt message is displayed. The redundant video segments are separately compressed and uploaded to the cloud platform for storage, which can further save storage space, reduce storage costs, and help improve the overall performance and reliability of the system.

[0035] The step S500 includes the following steps:

[0036] Step S501 traverses each frame of the hidden video, extracts color features, texture features, and shape features of each frame, calculates the similarity between adjacent frames through feature vectors, and sets a similarity threshold. When the similarity is lower than the threshold, it is determined that a shot switch occurs in the video, and the switched frames are marked and the video is divided into different shot segments.

[0037] Step S502: in each shot segment, the amount of information contained in each frame of the image is obtained by calculating the entropy value of the grayscale histogram of the image, and the frame with the largest entropy value in the shot segment is selected as the key frame of the shot;

[0038] Step S503 combines the key frames of all shots in order to form a video summary, encodes and compresses the video summary, and uploads the compressed video summary to the cloud platform for storage.

[0039] In the above technical solution, the hidden video data is stored as a video summary, which can provide users with a convenient and efficient way to retrieve and analyze videos. By extracting key frames from the video and combining them into a video summary, the core content of the video can be summarized to help users quickly understand the main information of the video; at the same time, encoding and compressing the video summary can further save storage space, improve storage efficiency, and help improve the practicality of the system and user satisfaction.

[0040] An image processing system for intelligent video surveillance, the system comprising a video change analysis module, an image clarity analysis module, a redundant video screening module, and a video summary module;

[0041] The video change analysis module captures video data through a surveillance camera and analyzes the acquired video data, detects changes between video frames, and marks video segments that are static for a long time or have no obvious changes in the picture;

[0042] The image clarity analysis module performs quantitative analysis on the clarity of each frame of the video segment, and when the blurriness of the image exceeds a preset threshold, marks the image as a blurred image;

[0043] The redundant video screening module combines the static and changing results and the fuzzy evaluation results to automatically screen out the redundant video segments and perform hidden processing and compressed storage on them;

[0044] The video summary module performs key frame extraction and shot segmentation operations on the hidden video data to generate a video summary for storage.

[0045] The video change analysis module is used to read the acquired video data frame by frame and perform preprocessing, and then calculate the pixel difference between two adjacent frames according to the formula:

[0046]

[0047] Where D(t) represents the difference between the tth frame and the t-1th frame, M and N are the width and height of the image respectively, P(x,y,t) represents the pixel value of the tth frame image at the coordinate (x,y), P(x,y,t-1) represents the pixel value of the t-1th frame image at the coordinate (x,y), and a frame difference threshold K is set;

[0048] According to the inter-frame difference D(t) and the time threshold T, the static segment is determined, a time window W is set, and the average inter-frame difference D in the window is calculated. avg , according to the formula:

[0049]

[0050] where t 0 is the starting frame of the time window; if the average difference between frames D avg If the time interval is less than the set threshold value K and the duration exceeds the set time threshold T, the time period is marked as a static period and S is set. j A flag indicating whether the jth frame is a still frame;

[0051] The image clarity analysis module is used to traverse each frame of the video segment and use the Laplace gradient method to perform a clarity quantitative evaluation on each frame of the image according to the formula: L(x,y)=|f(x+1,y)+f(x-1,y)-2f(x,y)|+|f(x,y+1)+f(x,y-1)-

[0052] 2f(x,y)|;

[0053] Where L(x,y) represents the Laplace gradient value of the image at the coordinate (x,y), and f(x,y) represents the pixel value of the image at the coordinate (x,y);

[0054] Calculate the overall image clarity value according to the formula:

[0055]

[0056] Where F represents the overall clarity value of the image; a blur threshold P is set. When the overall clarity of the image is lower than this threshold, the image is marked as blurry. R is set j A flag indicating whether the jth frame is a blurry frame.

[0057] The redundant video screening module is used to traverse all frames and automatically screen out redundant video segments according to the screening formula:

[0058]

[0059] Where H represents the preset continuous frame number threshold, S j and R j The value rule of is: if the jth frame is marked as a still frame, then when S j =1, otherwise S j = 0; if the jth frame is marked as a blurred frame, then when R j =1, otherwise R j =0; It means traversing all frames in the video, starting from the i-th frame, calculating the total number of still frames and blurred frames in H consecutive frames. If the total number is greater than or equal to H, the video from the i-th frame to the i+H-1-th frame is regarded as a redundant video segment mark and these frames are marked as redundant frames;

[0060] For video segments marked as redundant frames, during video playback, the system will automatically skip these frames based on the frame marking information and display a prompt message; after hiding, the redundant video segments are separately compressed and uploaded to the cloud platform for storage.

[0061] The video summary module is used to traverse each frame in the hidden video, extract color features, texture features, and shape features of each frame image, and calculate the similarity between adjacent frames through feature vectors. At the same time, a similarity threshold is set. When the similarity is lower than the threshold, it is determined that a shot switch occurs in the video, and the switched frames are marked and the video is divided into different shot sections. In each shot section, the amount of information contained in each frame image is obtained by calculating the entropy value of the grayscale histogram of the image, and the frame with the largest entropy value in the shot section is selected as the key frame of the shot; the key frames of all shots are combined in sequence to form a video summary, the video summary is encoded and compressed, and the compressed video summary is uploaded to the cloud platform for storage.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] The present invention automatically screens out video segments that are static for a long time or have no obvious changes in the picture by analyzing the changes between video frames, and performs hidden processing and compressed storage, effectively reducing the amount of video data that needs to be processed, thereby improving the efficiency of video processing; through the quantitative analysis of clarity, blurred images can be identified and marked, which helps to exclude low-quality images in subsequent video processing or analysis, thereby improving the quality of the overall video;

[0064] The present invention compresses and stores redundant video segments, which can significantly reduce the storage space requirements of video data, which helps to reduce the storage cost of large-scale video surveillance systems; at the same time, the video summary generated by the present invention contains key frames of each shot, which can reflect important information and changes in the video, allowing users to quickly browse and retrieve video content, thereby improving the efficiency and accuracy of video retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 The present invention is a method flow chart of an image processing method applied to intelligent video surveillance. DETAILED DESCRIPTION

[0066] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0067] Example: Figure 1 As shown, the present invention provides a technical solution, an image processing method applied to intelligent video surveillance, characterized in that the method comprises the following steps:

[0068] Step S100: Capturing video data through a surveillance camera;

[0069] Step S200: Analyze the acquired video data, detect changes between video frames, and mark video segments that are static for a long time or have no obvious changes in the picture;

[0070] Step S300: quantifying and analyzing the clarity of each frame of the video segment, and marking the image as a blurred image when the blurriness of the image exceeds a preset threshold;

[0071] Step S400: combining the static and changing results and the fuzzy evaluation results, automatically screening out the redundant video segments and performing hidden processing and compressed storage on them;

[0072] Step S500: extract key frames from the hidden video data and generate a video summary for storage.

[0073] The step S200 includes the following steps:

[0074] Step S201 reads the acquired video data frame by frame and performs preprocessing, and then calculates the pixel difference between two adjacent frames according to the formula:

[0075]

[0076] Where D(t) represents the difference between the tth frame and the t-1th frame, M and N are the width and height of the image respectively, P(x,y,t) represents the pixel value of the tth frame image at the coordinate (x,y), P(x,y,t-1) represents the pixel value of the t-1th frame image at the coordinate (x,y), and a frame difference threshold K is set;

[0077] Step S202 determines the static segment based on the inter-frame difference D(t) and the time threshold T, sets a time window W, and calculates the average inter-frame difference D in the window. avg , according to the formula:

[0078]

[0079] where t 0 is the starting frame of the time window; if the average difference between frames D avg If the time interval is less than the set threshold value K and the duration exceeds the set time threshold T, the time period is marked as a static period and S is set. j A flag indicating whether the jth frame is a still frame;

[0080] In the above technical solution, the acquired video data is analyzed, the changes between video frames are detected, and the video segments that are still for a long time or have no obvious changes in the picture are marked. This step can effectively identify the redundant parts in the video, and by calculating the pixel difference between two adjacent frames and setting the frame difference threshold and time threshold, it can accurately determine which video segments are still or have little change, which can significantly improve the efficiency of video processing, reduce the pressure of storage and transmission, and also help to improve the accuracy of video retrieval and analysis.

[0081] The step S300 includes the following steps:

[0082] Step S301 traverses each frame of the video segment and uses the Laplace gradient method to perform a clarity quantitative evaluation on each frame of the image according to the formula:

[0083] L(x,y)=|f(x+1,y)+f(x-1,y)-2f(x,y)|+|f(x,y+1)+f(x,y-1)-2f(x,y)|;

[0084] Where L(x,y) represents the Laplace gradient value of the image at the coordinate (x,y), and f(x,y) represents the pixel value of the image at the coordinate (x,y);

[0085] Step S302 calculates the overall image clarity value according to the formula:

[0086]

[0087] Where F represents the overall clarity value of the image; a blur threshold P is set. When the overall clarity of the image is lower than this threshold, the image is marked as blurry. R is set j A flag indicating whether the jth frame is a blurry frame.

[0088] In the above technical solution, the clarity of each frame in the video segment is quantitatively analyzed and blurred images are marked. This step can ensure that each frame in the video has sufficient clarity to meet the needs of subsequent analysis and processing. By using the Laplace gradient method to quantitatively evaluate the clarity of the image and setting the blur threshold, the system can accurately identify blurred images, thereby avoiding incorporating low-quality content into subsequent processing procedures, improving the overall quality of the video, reducing misjudgments or omissions caused by image blur, and also helping to optimize the efficiency of video storage and transmission.

[0089] The step S400 includes the following steps:

[0090] Step S401 traverses all frames and automatically filters out redundant video segments according to the filtering formula:

[0091]

[0092] Where H represents the preset continuous frame number threshold, S j and R j The value rule of is: if the jth frame is marked as a still frame, then when S j =1, otherwise S j = 0; if the jth frame is marked as a blurred frame, then when R j =1, otherwise R j =0; It means traversing all frames in the video, starting from the i-th frame, calculating the total number of still frames and blurred frames in H consecutive frames. If the total number is greater than or equal to H, the video from the i-th frame to the i+H-1-th frame is regarded as a redundant video segment mark and these frames are marked as redundant frames;

[0093] Step S402: for the video segments marked as redundant frames, during the video playback, the system will automatically skip these frames according to the frame marking information and display a prompt message;

[0094] In step S403, after the hiding process, the redundant video segments are individually compressed and uploaded to the cloud platform for storage.

[0095] In the above technical solution, by setting a filtering formula, redundant video segments are automatically identified and marked, and these frames are automatically skipped during video playback. At the same time, prompt information is displayed, and the redundant video segments are separately compressed and uploaded to the cloud platform for storage. This can further save storage space, reduce storage costs, and help improve the overall performance and reliability of the system.

[0096] The step S500 includes the following steps:

[0097] Step S501 traverses each frame of the hidden video, extracts color features, texture features, and shape features of each frame, calculates the similarity between adjacent frames through feature vectors, and sets a similarity threshold. When the similarity is lower than the threshold, it is determined that a shot switch occurs in the video, and the switched frames are marked and the video is divided into different shot segments.

[0098] Step S502: in each shot segment, the amount of information contained in each frame of the image is obtained by calculating the entropy value of the grayscale histogram of the image, and the frame with the largest entropy value in the shot segment is selected as the key frame of the shot;

[0099] Step S503 combines the key frames of all shots in order to form a video summary, encodes and compresses the video summary, and uploads the compressed video summary to the cloud platform for storage.

[0100] In the above technical solution, the hidden video data is stored as a video summary, which can provide users with a convenient and efficient way to retrieve and analyze videos. By extracting key frames from the video and combining them into a video summary, the core content of the video can be summarized to help users quickly understand the main information of the video; at the same time, encoding and compressing the video summary can further save storage space, improve storage efficiency, and help improve the practicality of the system and user satisfaction.

[0101] An image processing system for intelligent video surveillance, the system comprising a video change analysis module, an image clarity analysis module, a redundant video screening module, and a video summary module;

[0102] The video change analysis module captures video data through a surveillance camera and analyzes the acquired video data, detects changes between video frames, and marks video segments that are static for a long time or have no obvious changes in the picture;

[0103] The image clarity analysis module performs quantitative analysis on the clarity of each frame of the video segment, and when the blurriness of the image exceeds a preset threshold, marks the image as a blurred image;

[0104] The redundant video screening module combines the static and changing results and the fuzzy evaluation results to automatically screen out the redundant video segments and perform hidden processing and compressed storage on them;

[0105] The video summary module performs key frame extraction and shot segmentation operations on the hidden video data to generate a video summary for storage.

[0106] The video change analysis module is used to read the acquired video data frame by frame and perform preprocessing, and then calculate the pixel difference between two adjacent frames according to the formula:

[0107]

[0108] Where D(t) represents the difference between the tth frame and the t-1th frame, M and N are the width and height of the image respectively, P(x,y,t) represents the pixel value of the tth frame image at the coordinate (x,y), P(x,y,t-1) represents the pixel value of the t-1th frame image at the coordinate (x,y), and a frame difference threshold K is set;

[0109] According to the inter-frame difference D(t) and the time threshold T, the static segment is determined, a time window W is set, and the average inter-frame difference D in the window is calculated. avg , according to the formula:

[0110]

[0111] where t 0 is the starting frame of the time window; if the average difference between frames D avg If the time interval is less than the set threshold value K and the duration exceeds the set time threshold T, the time period is marked as a static period and S is set. j A flag indicating whether the jth frame is a still frame;

[0112] The image clarity analysis module is used to traverse each frame of the video segment and use the Laplace gradient method to perform a clarity quantitative evaluation on each frame of the image according to the formula: L(x,y)=|f(x+1,y)+f(x-1,y)-2f(x,y)|+|f(x,y+1)+f(x,y-1)-

[0113] 2f(x,y)|;

[0114] Where L(x,y) represents the Laplace gradient value of the image at the coordinate (x,y), and f(x,y) represents the pixel value of the image at the coordinate (x,y);

[0115] Calculate the overall image clarity value according to the formula:

[0116]

[0117] Where F represents the overall clarity value of the image; a blur threshold P is set. When the overall clarity of the image is lower than this threshold, the image is marked as blurry. R is set j A flag indicating whether the jth frame is a blurry frame.

[0118] The redundant video screening module is used to traverse all frames and automatically screen out redundant video segments according to the screening formula:

[0119]

[0120] Where H represents the preset continuous frame number threshold, S j and R j The value rule of is: if the jth frame is marked as a still frame, then when S j =1, otherwise S j = 0; if the jth frame is marked as a blurred frame, then when R j =1, otherwise R j =0; It means traversing all frames in the video, starting from the i-th frame, calculating the total number of still frames and blurred frames in H consecutive frames. If the total number is greater than or equal to H, the video from the i-th frame to the i+H-1-th frame is regarded as a redundant video segment mark and these frames are marked as redundant frames;

[0121] For video segments marked as redundant frames, during video playback, the system will automatically skip these frames according to the frame marking information and display a prompt message at the same time; after the hiding process, the redundant video segments are separately compressed and uploaded to the cloud platform for storage.

[0122] The video summary module is used to traverse each frame in the video after the hiding process, extract color features, texture features, and shape features from each frame image, calculate the similarity between adjacent frames through feature vectors, and set a similarity threshold at the same time. When the similarity is lower than this threshold, it is determined that a shot transition has occurred in the video, mark the transition frames, and split the video into different shot segments; in each shot segment, calculate the information amount contained in each frame image by calculating the entropy value of the grayscale histogram of the image, and select the frame with the largest entropy value in the shot segment as the key frame of this shot; combine the key frames of all shots in sequence to form a video summary, perform encoding and compression processing on the video summary, and upload the compressed video summary to the cloud platform for storage.

[0123] Embodiment: In a community, in order to monitor the security situation of the community, multiple surveillance cameras are installed. These cameras continuously capture video data and upload it to the central monitoring room for analysis and processing; capture a 1-hour video data from the surveillance camera, with a resolution of 1920x1080 and a frame rate of 30fps;

[0124] Step S201: Read the video data frame by frame. For each frame image, calculate the pixel difference between it and the previous frame image at each coordinate (x, y) where M = 1920 and N = 1080; set the inter-frame difference threshold K = 1000;

[0125] Step S202: Determine the stationary segment according to the inter-frame difference D(t) and the time threshold T. Set the time window W = 60, that is, the number of frames within 1 second, and calculate the average value Davg of the inter-frame difference within the window avg ; set the time threshold T = 300, that is, the number of frames within 5 seconds; if Davg < K and the duration exceeds T, then mark this time period as a stationary segment;

[0126] Step S203: It is detected that from the 5000th frame to the 5500th frame of the video, D avg < K and the duration exceeds T, then mark the 5000th frame to the 5500th frame as stationary frames;

[0127] Step S301: Traverse each frame image in the video segment, calculate the clarity of each frame image using the Laplace gradient method, and for each frame image, calculate the Laplace gradient value L(x, y) at each coordinate (x, y);

[0128] Step S302: Calculate the overall clarity value F of the image, set the blur threshold P = 10000. If F < P, mark the image as a blurred image;

[0129] Step S401: Set the threshold H for the number of consecutive frames to 100, that is, if the total number of static frames and blurred frames within 100 consecutive frames reaches or exceeds 100 frames, then this video segment is considered redundant; filter out the redundant video segments according to the formula;

[0130] Step S402: For the video segments marked as redundant frames, automatically skip these frames during video playback and display a prompt message "The redundant part of the video has been skipped";

[0131] Step S403: Perform separate compression processing on the redundant video segments using H.265 encoding and upload them to the cloud platform for storage;

[0132] Step S500: Traverse the video after the hiding process, decompose the video into a series of individual frames. For each frame image, extract color features, texture features, and shape features, and evaluate the similarity between adjacent frames by calculating the cosine similarity; set the similarity threshold to 0.6, that is, when the similarity is lower than 0.6, it is considered that a shot transition has occurred in the video; in each shot segment, calculate the gray histogram entropy value of each frame image to evaluate the amount of information contained in each frame image, and select the frame with the largest entropy value in the shot segment as the key frame of this shot; combine the key frames of all shots in order to form a video summary, perform encoding and compression processing on the video summary, and upload the compressed video summary to the cloud platform for storage.

[0133] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An image processing method applied to intelligent video surveillance, characterized in that: The method comprises the following steps: Step S100: Capturing video data through a surveillance camera; Step S200: Analyze the acquired video data, detect changes between video frames, and mark video segments that are static for a long time or have no obvious changes in the picture; Step S300: quantifying and analyzing the clarity of each frame of the video segment, and marking the image as a blurred image when the blurriness of the image exceeds a preset threshold; Step S400: combining the static and changing results and the fuzzy evaluation results, automatically screening out the redundant video segments and performing hidden processing and compressed storage on them; Step S500: extract key frames from the hidden video data and generate a video summary for storage.

2. The image processing method for intelligent video surveillance according to claim 1, characterized in that: The step S200 includes the following steps: Step S201 reads the acquired video data frame by frame and performs preprocessing, and then calculates the pixel difference between two adjacent frames according to the formula: Where D(t) represents the difference between the tth frame and the t-1th frame, M and N are the width and height of the image respectively, P(x,y,t) represents the pixel value of the tth frame image at the coordinate (x,y), P(x,y,t-1) represents the pixel value of the t-1th frame image at the coordinate (x,y), and a frame difference threshold K is set; Step S202 determines the static segment based on the inter-frame difference D(t) and the time threshold T, sets a time window W, and calculates the average inter-frame difference D in the window. avg , according to the formula: Where t0 is the starting frame of the time window; if the average difference between frames D avg If the time interval is less than the set threshold K and the duration exceeds the set time threshold T, the time period is marked as a static period and S is set. j A flag indicating whether the jth frame is a still frame.

3. The image processing method for intelligent video surveillance according to claim 1, characterized in that: The step S300 includes the following steps: Step S301 traverses each frame of the video segment and uses the Laplace gradient method to perform a clarity quantitative evaluation on each frame of the image according to the formula: L(x,y)=|f(x+1,y)+f(x-1,y)-2f(x,y)|+|f(x,y+1)+f(x,y-1)-2f(x,y)|; Where L(x,y) represents the Laplace gradient value of the image at the coordinate (x,y), and f(x,y) represents the pixel value of the image at the coordinate (x,y); Step S302 calculates the overall image clarity value according to the formula: Where F represents the overall clarity value of the image; a blur threshold P is set. When the overall clarity of the image is lower than this threshold, the image is marked as blurry. R is set j A flag indicating whether the jth frame is a blurry frame.

4. The image processing method for intelligent video surveillance according to claim 1, characterized in that: The step S400 includes the following steps: Step S401 traverses all frames and automatically filters out redundant video segments according to the filtering formula: Where H represents the preset continuous frame number threshold, S j and R j The value rule of is: if the jth frame is marked as a still frame, then when S j =1, otherwise S j =0; If the jth frame is marked as a blurry frame, then when R j =1, otherwise R j =0; It means traversing all frames in the video, starting from the i-th frame, calculating the total number of still frames and blurred frames in H consecutive frames. If the total number is greater than or equal to H, the video from the i-th frame to the i+H-1-th frame is regarded as a redundant video segment mark and these frames are marked as redundant frames; Step S402: for the video segments marked as redundant frames, during the video playback, the system will automatically skip these frames according to the frame marking information and display a prompt message; In step S403, after the hiding process, the redundant video segments are individually compressed and uploaded to the cloud platform for storage.

5. The image processing method for intelligent video surveillance according to claim 1, characterized in that: The step S500 includes the following steps: Step S501 traverses each frame of the hidden video, extracts color features, texture features, and shape features of each frame, calculates the similarity between adjacent frames through feature vectors, and sets a similarity threshold. When the similarity is lower than the threshold, it is determined that a shot switch occurs in the video, and the switched frames are marked and the video is divided into different shot segments. Step S502: in each shot segment, the amount of information contained in each frame of the image is obtained by calculating the entropy value of the grayscale histogram of the image, and the frame with the largest entropy value in the shot segment is selected as the key frame of the shot; Step S503 combines the key frames of all shots in order to form a video summary, encodes and compresses the video summary, and uploads the compressed video summary to the cloud platform for storage.

6. An image processing system for intelligent video surveillance, characterized in that: The system includes a video change analysis module, an image clarity analysis module, a redundant video screening module, and a video summary module; The video change analysis module captures video data through a surveillance camera and analyzes the acquired video data, detects changes between video frames, and marks video segments that are static for a long time or have no obvious changes in the picture; The image clarity analysis module performs quantitative analysis on the clarity of each frame of the video segment, and when the blurriness of the image exceeds a preset threshold, marks the image as a blurry image; The redundant video screening module combines the static and changing results and the fuzzy evaluation results to automatically screen out the redundant video segments and perform hidden processing and compressed storage on them; The video summary module performs key frame extraction and shot segmentation operations on the hidden video data to generate a video summary for storage.

7. The image processing system for intelligent video surveillance according to claim 6, characterized in that: The video change analysis module is used to read the acquired video data frame by frame and perform preprocessing, and then calculate the pixel difference between two adjacent frames according to the formula: Where D(t) represents the difference between the tth frame and the t-1th frame, M and N are the width and height of the image respectively, P(x,y,t) represents the pixel value of the tth frame image at the coordinate (x,y), P(x,y,t-1) represents the pixel value of the t-1th frame image at the coordinate (x,y), and a frame difference threshold K is set; According to the inter-frame difference D(t) and the time threshold T, the static segment is determined, a time window W is set, and the average inter-frame difference D in the window is calculated. avg , according to the formula: Where t0 is the starting frame of the time window; if the average difference between frames D avg If the time interval is less than the set threshold K and the duration exceeds the set time threshold T, the time period is marked as a static period and S is set. j A flag indicating whether the jth frame is a still frame; The image clarity analysis module is used to traverse each frame of the video segment and use the Laplace gradient method to perform a clarity quantitative evaluation on each frame of the image according to the formula: L(x,y)=|f(x+1,y)+f(x-1,y)-2f(x,y)|+|f(x,y+1)+f(x,y-1)-2f(x,y|; Where L(x,y) represents the Laplace gradient value of the image at the coordinate (x,y), and f(x,y) represents the pixel value of the image at the coordinate (x,y); Calculate the overall image clarity value according to the formula: Where F represents the overall clarity value of the image; a blur threshold P is set. When the overall clarity of the image is lower than this threshold, the image is marked as blurry. R is set j A flag indicating whether the jth frame is a blurry frame.

8. The image processing system for intelligent video surveillance according to claim 6, characterized in that: The redundant video screening module is used to traverse all frames and automatically screen out redundant video segments according to the screening formula: Where H represents the preset continuous frame number threshold, S j and R j The value rule of is: if the jth frame is marked as a still frame, then when S j =1, otherwise S j =0; If the jth frame is marked as a blurry frame, then when R j =1, otherwise R j =0; It means traversing all frames in the video, starting from the i-th frame, calculating the total number of still frames and blurred frames in H consecutive frames. If the total number is greater than or equal to H, the video from the i-th frame to the i+H-1-th frame is regarded as a redundant video segment mark and these frames are marked as redundant frames; For video segments marked as redundant frames, during video playback, the system will automatically skip these frames based on the frame marking information and display a prompt message; after hiding, the redundant video segments are separately compressed and uploaded to the cloud platform for storage.

9. The image processing system for intelligent video surveillance according to claim 6, characterized in that: The video summary module is used to traverse each frame in the hidden video, extract color features, texture features, and shape features of each frame image, and calculate the similarity between adjacent frames through feature vectors. At the same time, a similarity threshold is set. When the similarity is lower than the threshold, it is determined that a shot switch occurs in the video, and the switched frames are marked and the video is divided into different shot sections. In each shot section, the amount of information contained in each frame image is obtained by calculating the entropy value of the grayscale histogram of the image, and the frame with the largest entropy value in the shot section is selected as the key frame of the shot; the key frames of all shots are combined in sequence to form a video summary, the video summary is encoded and compressed, and the compressed video summary is uploaded to the cloud platform for storage.

Images