A monitoring video recommendation method and system based on dynamic background removal

Abstract

The application relates to a campus monitoring video recommendation method and system based on dynamic background removal, which comprises the following steps: dynamic background elimination: performing background shielding processing on the monitoring video; motion target detection: performing motion target detection on the monitoring video after the background shielding processing; if a motion target exists, the monitoring video is transmitted to a server for other function detection, or is transmitted to a large screen in a monitoring center for display; and a low-complexity motion target detection algorithm based on background elimination is used to overcome the influence of dynamic backgrounds such as leaves, vegetation, flags and flickering lights in the video stream of an outdoor camera on motion target detection, so that effective detection of the motion target is realized; and through screening and recommendation of massive videos, the application reduces the demand of a business function based on deep learning for computing power and reduces the average processing cost of each video stream.

Description

Technical Field

[0001] This invention relates to the fields of data processing and computer vision technology, and in particular to a monitoring recommendation method and system based on dynamic background removal. Background Technology

[0002] Surveillance, as one of the most widely used systems in security systems, is deployed in various scenarios and plays an increasingly important role, becoming an indispensable security barrier in social life. Therefore, there is a pressing need for surveillance in public areas such as banks, supermarkets, shopping malls, factories, schools, and communities. University campuses are large and require high levels of security. Security personnel patrols can only cover limited areas and time periods. Therefore, video surveillance systems are of great significance for comprehensive management and monitoring of the campus, ensuring the personal and property safety of teachers, students, and other staff, and strengthening the campus's ability to prevent and control security risks.

[0003] Currently, the surveillance systems used on university campuses are extremely large-scale, typically consisting of thousands to tens of thousands of cameras. Traditional real-time monitoring by on-duty personnel can only cover a few extremely important key camera locations, making it impossible to handle the massive video streams generated by the vast majority of cameras. Moreover, the footage from most cameras remains static for extended periods, resulting in a high proportion of invalid data, making it impractical to rely solely on on-duty personnel for monitoring.

[0004] In recent years, machine learning has been widely used in image and video processing, and computers can now automatically analyze and process surveillance data. However, effective machine learning methods typically require powerful computing capabilities, so using deep learning technology to automatically analyze and process all video streams would result in exorbitant system costs, exceeding the affordability of users.

[0005] Our investigation revealed that many surveillance cameras on campus recorded little to no pedestrian or vehicle traffic for most of the time, resulting in a low proportion of valid data. Directly analyzing the video streams using deep learning would be a significant waste of computing power. Therefore, if we could perform preliminary analysis of the video streams, recommending only footage showing pedestrian or vehicle traffic to the monitoring center's large screen for unified display, while simultaneously sending relevant video clips or extracted images to the server for automatic machine learning analysis, we could not only effectively improve the efficiency of personnel monitoring but also significantly reduce the overall system's computing power requirements.

[0006] In the aforementioned system, the algorithm for preliminary analysis of the video stream is crucial. Designing lightweight, low-complexity analysis algorithms enables real-time analysis and processing of large volumes of video streams using simple edge computing devices, which plays a decisive role in reducing the overall system cost. Image processing methods based on inter-frame difference of video images can effectively analyze moving targets with very low computational complexity and are commonly used methods. However, videos generated by outdoor cameras contain dynamic backgrounds such as leaves, vegetation, flags, and flashing lights. Simple inter-frame difference algorithms are greatly affected by dynamic backgrounds and cannot function properly. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a low-cost method and system for recommending surveillance videos based on dynamic background removal.

[0008] Terminology Explanation:

[0009] 1. Dynamic background: In campus and outdoor surveillance videos, background elements such as leaves, flags, and vegetation that affect the detection of moving targets are usually in a localized state of motion.

[0010] 2. Morphological dilation / erosion: Morphology is a series of image processing operations based on shape. The most basic operations are dilation and erosion. Dilation makes the highlighted area larger and is an operation to find the local maximum value. Erosion makes the highlighted area smaller and is an operation to find the local minimum value.

[0011] 3. Asymmetric expansion: The mask area expansion method proposed in this invention aims to increase the mask area, enhance the occlusion of dynamic backgrounds, and improve the efficiency of subsequent moving target detection.

[0012] The technical solution of this invention is as follows:

[0013] A method for recommending surveillance videos based on dynamic background removal includes:

[0014] Dynamic background removal: This function removes background occlusion from surveillance videos.

[0015] Moving target detection: The monitoring video after background occlusion is processed is used to detect moving targets. The monitoring video with moving targets is sent to the server for other function testing, or transmitted to the large screen in the monitoring center for display.

[0016] According to a preferred embodiment of the present invention, dynamic background elimination includes:

[0017] Activity point statistics: Acquire monitoring video stream data, calculate the inter-frame difference image of grayscale images between two adjacent frames, and statistically analyze the pixels with more than 100 image values ​​of 255 in the inter-frame difference image of 255 consecutive frames. Generate a mask image based on the statistical characteristics of these pixels, and apply it to the analysis of the subsequent 255 frames. Generate a new mask image for the subsequent 255 frames, and so on.

[0018] Noise reduction: Noise reduction is performed on each mask image.

[0019] Asymmetric dilation: Asymmetric dilation is performed on the denoised mask image to increase the area of ​​the mask image and achieve occlusion of dynamic backgrounds in the video.

[0020] Further optimization includes activity point statistics, including:

[0021] 1) Obtain the grayscale difference image of the surveillance video: Convert the three-channel image of the original surveillance image into a grayscale image. The formula for calculating the grayscale value (Gray) of each pixel is shown in Equation (I):

[0022] Gray=R*0.299+G*0.587+B*0.144 (I)

[0023] The processed image is a grayscale image, and the data is stored as a two-dimensional array. The grayscale value of each pixel is represented by the value at the corresponding position in the array.

[0024] Calculate the grayscale image f of the current image. n , and the grayscale image f of the next frame. n+1 The absolute value difference between the two is calculated to obtain the current grayscale difference image f. dif That is, to calculate the absolute value of the values ​​at corresponding positions in two two-dimensional arrays corresponding to two grayscale images, and save the result as a two-dimensional array of the same size;

[0025] 2) Binarize the grayscale difference image to obtain image f. dif-thr Create a two-dimensional array f with the same size as the image. count This is used to record statistical information of 250 consecutive binarized grayscale difference images, f count Each data a in ij For the 250 images at the corresponding positions f dif-thr The corresponding position b ij The number of values ​​that are 255; a ij The calculation formula is shown in equation (II):

[0026]

[0027] From equation (II), we can see that aij The value range is 0-250, when a ij When the value is greater than 65, this location in the image is considered to be dynamic background and should be removed. ij Set to 255; for a ij When the value is less than 65, the location in the image is considered to be either a stationary part or a part through which the target has passed, and a is set to... ij Set to 0 to obtain the initial mask image f mask .

[0028] Further preferred, the noise reduction process, the initial mask image noise reduction, includes:

[0029] Denoising is achieved using morphological dilation and erosion: First, an opening operation is performed, followed by erosion to eliminate individual pixels. Then, a dilation operation is performed to restore the original area of ​​the normal portion, resulting in the denoised mask image f'. mask .

[0030] Further preferred, asymmetric expansion includes:

[0031] 1) Calculate the centroid coordinates of the mask image.

[0032] The calculation process for the centroid's x-coordinate or y-coordinate is as follows:

[0033] First, count the number of data points with a value of 255 in all vertical or horizontal directions to obtain an array of length 720. That is, each data point in the array is the number of data points with the same x or y coordinate that are 255.

[0034] Then, multiply the data in the obtained array by the corresponding x-coordinate or y-coordinate, and then calculate the average of all the data in the array, which is the x-coordinate or y-coordinate of the centroid.

[0035] According to the calculation process, the x-coordinate of the centroid ranges from 1 to 720, and the y-coordinate ranges from 1 to 576. Since the data in the mask image only includes 255 and 0, the x-coordinate of the centroid is calculated using equations (III) and (IV). i , centroid ordinate x j :

[0036]

[0037]

[0038] 2) Mask area expansion; the specific calculation process is as follows:

[0039] ① Read the image data from the leftmost column of the image array, with a length of 576. Iterate through each column of the image data from left to right. When the data of the mask image is 255, set all subsequent image data corresponding to the y-coordinate to 255. Stop iterating when the centroid x-coordinate is reached. This gives you the mask expansion image f, which was traversed from the left. ml ;

[0040] ② Read the image data from the rightmost column of the image array, which has a length of 576. Iterate through each column of the image data from right to left. When the data of the mask image is 255, set all subsequent image data corresponding to the vertical coordinate to 255. Stop iterating when the horizontal coordinate of the centroid is reached. This gives the mask expansion image f, which was traversed from the left. mr ;

[0041] ③ Read the image data from the top row of the image array, which has a length of 720. Iterate through each row of the image data from top to bottom. When the data of the mask image is 255, set all subsequent image data with the corresponding horizontal coordinate to 255. Stop iterating when the horizontal coordinate of the centroid is reached. This will give you the mask expansion image f, which is traversed from the left. mu ;

[0042] ④ Read the image data from the bottom row of the image array, which has a length of 720. Iterate through each row of the image data from bottom to top. When the data of the mask image is 255, set all subsequent image data with the corresponding horizontal coordinate to 255. Stop iterating when the horizontal coordinate of the centroid is reached. This will give you the mask expansion image f, which is traversed from the left. md ;

[0043] Traverse the image data from four directions to the centroid center, and perform a union operation on the portions of the data with a value of 255 in the four images, as shown in equation (V):

[0044] f = f ml ∪f mr ∪f mu ∪f md (V)

[0045] The final image f is a mask image after asymmetric dilation, which is used for dynamic background removal in subsequent videos.

[0046] According to a preferred embodiment of the present invention, moving target detection includes:

[0047] For each frame of image after background occlusion processing, activity points are counted. If the area of ​​active points in 10 consecutive frames exceeds 10% of the area of ​​the unoccluded image, then a moving target is considered to exist in the last frame. When a moving target is detected, the relevant monitoring information is recommended to the server for subsequent complex function testing, or recommended to the monitoring center's large screen for display.

[0048] Further preferred moving target detection includes:

[0049] When processing the calculated mask image, first set the part of the image f with data of 255 to 0, and set the part with data of 0 to 255 to obtain the final mask image f1.

[0050] The video to be detected is converted into a grayscale image. The mask image f1 obtained by asymmetric dilation is logically ANDed with the video image to obtain the video image data with dynamic background removed.

[0051] Active point statistics are performed on video images with dynamic backgrounds removed, that is, the proportion of data with a value greater than 40 in the two-dimensional array corresponding to each frame of the image to the total data corresponding to all unoccluded areas.

[0052] Let f be the grayscale image of the surveillance camera after occlusion. c f is obtained after performing a difference operation on two adjacent frames. c-dif Values ​​greater than 40 are designated as active points. The proportion r of all data points with values ​​greater than 40 to all unoccluded image data is calculated. x Calculate the proportion of active points in 10 consecutive frames, r1, r2, ..., r 10 When all proportions are greater than 10%, it is assumed that there is a moving target in the 10th frame image and that there is a moving target in the output image. When a moving target is detected in the monitoring image, the relevant monitoring information is recommended to the server for subsequent complex function testing, or recommended to the monitoring center screen for display.

[0053] A surveillance video recommendation system based on dynamic background removal includes:

[0054] The dynamic background removal module is configured to perform background occlusion processing on the surveillance video;

[0055] The moving target detection module is configured to detect moving targets in the monitoring video after background occlusion processing. The monitoring video containing moving targets is sent to the server for other function testing, or transmitted to the large screen in the monitoring center for display.

[0056] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a surveillance video recommendation method based on dynamic background removal.

[0057] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a surveillance video recommendation method based on dynamic background removal.

[0058] The beneficial effects of this invention are as follows:

[0059] 1. Design a low-complexity moving target detection algorithm based on background removal to overcome the influence of dynamic backgrounds such as leaves, vegetation, flags, and flashing lights in outdoor camera video streams on moving target detection, and achieve effective detection of moving targets;

[0060] 2. Reduce the number of videos recommended for display on the large screen, thereby reducing the workload of manual supervision and allowing staff to focus their attention on the monitoring screens of important locations.

[0061] 3. By filtering and recommending massive amounts of videos, the computing power requirements of deep learning-based business functions are reduced, and the average processing cost of each video stream is lowered.

[0062] 4. By conducting full-coverage analysis and processing of the video generated by the cameras, the probability of detecting campus emergencies can be improved, thereby enhancing the level of campus security. Attached Figure Description

[0063] Figure 1 This is a flowchart of the low-cost campus surveillance video recommendation method based on dynamic background removal according to the present invention.

[0064] Figure 2 This is a flowchart illustrating the image activity point statistics of the present invention.

[0065] Figure 3 This is a schematic diagram of the asymmetric expansion calculation process of the present invention;

[0066] Figure 4 This is a flowchart illustrating the dynamic background removal method using mask images according to the present invention.

[0067] Figure 5 This is a flowchart illustrating the method for moving target detection using statistical activity points according to the present invention. Detailed Implementation

[0068] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.

[0069] Example 1

[0070] A method for recommending surveillance videos based on dynamic background removal, such as Figure 1 As shown, it includes:

[0071] Dynamic background removal: This function removes background occlusion from surveillance videos.

[0072] Moving target detection: The monitoring video after background occlusion is processed is used to detect moving targets. The monitoring video with moving targets is sent to the server for other function testing, or transmitted to the large screen in the monitoring center for display.

[0073] Example 2

[0074] The difference between the surveillance video recommendation method based on dynamic background removal described in Example 1 and the following is:

[0075] Dynamic background removal, including:

[0076] Activity point statistics: Acquire surveillance video stream data, calculate the inter-frame difference image of grayscale images between two adjacent frames, and statistically analyze pixels with a value of 255 in more than 100 of the inter-frame difference images across 255 consecutive frames. A mask image is generated based on the statistical characteristics of these pixels and applied to the analysis of the subsequent 255 frames. New mask images are generated for each subsequent 255 frames, and so on. Activity point statistics include:

[0077] Observation of a large amount of outdoor surveillance video data on campus revealed that dynamic backgrounds maintain a relatively fixed motion state within a certain area for extended periods. This characteristic is reflected in the corresponding grayscale difference binarized image as the active area appearing as white. Pedestrians, vehicles, and other targets typically exhibit linear motion or remain stationary. In the processed difference image, a white area is observed moving linearly within the image, without lingering in one area for an extended period. Based on this characteristic, the statistical properties of activity points can be used to distinguish between dynamic backgrounds and moving targets, resulting in a preliminary mask image. The overall flowchart for activity point statistics is as follows. Figure 2 As shown.

[0078] 1) Obtaining the grayscale difference image of the surveillance video: The common secondary stream image of surveillance video has a height of 576, a width of 720, and a frame rate of 25. The initially obtained image is a three-channel image, meaning that each pixel in each image has three corresponding values, which correspond to the three colors: red (R), green (G), and blue (B). To facilitate image binarization calculation, the original three-channel image of the surveillance video needs to be converted into a grayscale image. The formula for calculating the grayscale value Gray of each pixel is shown in Equation (I):

[0079] Gray=R*0.299+G*0.587+B*0.144 (I)

[0080] The processed image is a grayscale image, and the data is stored as a two-dimensional array. The grayscale value of each pixel is represented by the value at the corresponding position in the array.

[0081] Calculate the grayscale image f of the current image. n , and the grayscale image f of the next frame. n+1 The absolute value difference between the two is calculated to obtain the current grayscale difference image f. dif That is, to calculate the absolute value of the corresponding positions of the two two-dimensional arrays corresponding to the two grayscale images, and save the result as a two-dimensional array of the same size;

[0082] 2) Binarize the grayscale difference image to obtain image f. dif-thr The threshold is set to 40. Data in the grayscale difference image that is greater than the threshold is set to 255 and displayed as white in the image, while data that is less than the threshold is set to 0 and displayed as black in the image. The area with a value of 255 is the part of the image where the change is more dramatic, and the area with a value of 0 is the part of the image where the change is more gradual.

[0083] Based on the different characteristics of dynamic backgrounds and the images of vehicles and people, the binarized grayscale difference image f is analyzed. dif-thr By statistically analyzing the pixels in the image, their statistical properties can be used for differentiation. A two-dimensional array f with the same size as the image is created. count This is used to record statistical information of 250 consecutive binarized grayscale difference images, f count Each data a in ij For the 250 images at the corresponding positions f dif-thr The corresponding position b ij The number of values ​​that are 255; a ij The calculation formula is shown in equation (II):

[0084]

[0085] From equation (II), we can see that a ij The value range of a is 0 to 250. ij When the value is greater than 65, this location in the image is considered to be dynamic background and should be removed. ij Set to 255; for a ij When the value is less than 65, the location in the image is considered to be either a stationary part or a part through which the target has passed, and a is set to... ij Set to 0 to obtain the initial mask image f mask .

[0086] Noise reduction: Noise reduction is performed on each mask image. The initial mask image noise reduction includes:

[0087] The initial mask image f obtained mask There is often a lot of data affected by noise in the initial mask image. The occluded parts are mostly individual pixels or very small areas, which need to be discarded in subsequent calculations. Therefore, the initial mask image needs to be denoised.

[0088] f mask The image is a binary image composed of 0 and 255 data points. Morphological dilation and erosion can be used for noise reduction: First, an opening operation is performed, then erosion is applied to eliminate individual pixels; second, a dilation operation is performed to restore the original area of ​​the normal portion, resulting in the denoised mask image f'. mask Opening operations are mainly used to eliminate small objects, separate objects at thin points, and smooth the boundaries of larger objects without significantly changing their area.

[0089] The erosion operation involves the following: the effective data of the initial mask image consists of the portion with a value of 255. Erosion reduces the size of the image portion corresponding to this effective data. The erosion operation uses a 3x3 two-dimensional array as the structure data, with all data within the array being 255. The array is slid across the image using its center position. At each position, a bitwise AND operation is performed between the structure data and the corresponding data in the image. When all the results are 255, the corresponding pixel data in the image is set to 255; otherwise, it is set to 0.

[0090] Dilation, or expansion, is the opposite of erosion in its effect on the effective data portion; it increases the area of ​​the effective data. The dilation operation uses a 3x3 two-dimensional array as its structural data, with all data points set to 255. The array is slid across the image using its center position, and a bitwise AND operation is performed between the structural data and the corresponding image data at each position. If any result contains a value of 255, the corresponding pixel in the image is set to 255; otherwise, it is set to 0.

[0091] Asymmetric dilation: Asymmetric dilation is applied to the denoised mask image to increase its area, thereby occluding dynamic backgrounds in the video and removing their influence on moving target detection. Asymmetric dilation includes:

[0092] After noise reduction, the mask image needs further targeted enlargement of the mask area to effectively remove dynamic backgrounds. Preliminary processing of on-campus and outdoor surveillance videos revealed that while setting the activity point statistics value too low effectively masks dynamic backgrounds, it also makes it difficult to remove noise from the initial mask image. Therefore, to ensure noise removal while simultaneously increasing the mask area, asymmetric dilation is required. The flowchart for asymmetric dilation is shown below. Figure 3 As shown.

[0093] 1) Calculate the centroid coordinates of the mask image.

[0094] For digital images, the centroid coordinates are calculated based on the portion of the image occupied by the target. The denoised mask image is a two-dimensional array, with the same size as the surveillance image, 720*576. The calculation process for the centroid's horizontal or vertical coordinates is as follows:

[0095] First, count the number of data points with a value of 255 in all vertical or horizontal directions to obtain an array of length 720. That is, each data point in the array is the number of data points with the same x or y coordinate that are 255.

[0096] Then, multiply the data in the obtained array by the corresponding x-coordinate or y-coordinate, and then calculate the average of all the data in the array, which is the x-coordinate or y-coordinate of the centroid.

[0097] Similarly, by transforming the x and y coordinates in the above calculation process, the y coordinate of the centroid can also be obtained. According to the calculation process, the x-coordinate of the centroid ranges from 1 to 720, and the y-coordinate ranges from 1 to 576. Since the data within the mask image only contains 255 and 0, the x-coordinate of the centroid can be calculated using equations (III) and (IV). i , centroid ordinate x j :

[0098]

[0099]

[0100] The calculated centroid coordinates are used in the next step of calculating the enlarged mask area.

[0101] 2) Mask area expansion; The mask portion in the mask image needs to be extended in two perpendicular directions towards the centroid to expand the mask area and effectively occlude the dynamic background. The specific calculation process is as follows:

[0102] ① Read the image data from the leftmost column of the image array, with a length of 576. Iterate through each column of the image data from left to right. When the data of the mask image is 255, set all subsequent image data corresponding to the y-coordinate to 255. Stop iterating when the centroid x-coordinate is reached. This gives you the mask expansion image f, which was traversed from the left. ml ;

[0103] ② Read the image data from the rightmost column of the image array, which has a length of 576. Iterate through each column of the image data from right to left. When the data of the mask image is 255, set all subsequent image data corresponding to the vertical coordinate to 255. Stop iterating when the horizontal coordinate of the centroid is reached. This gives the mask expansion image f, which was traversed from the left. mr ;

[0104] ③ Read the image data from the top row of the image array, which has a length of 720. Iterate through each row of the image data from top to bottom. When the data of the mask image is 255, set all subsequent image data with the corresponding horizontal coordinate to 255. Stop iterating when the horizontal coordinate of the centroid is reached. This will give you the mask expansion image f, which is traversed from the left. mu ;

[0105] ④ Read the image data from the bottom row of the image array, which has a length of 720. Iterate through each row of the image data from bottom to top. When the data of the mask image is 255, set all subsequent image data with the corresponding horizontal coordinate to 255. Stop iterating when the horizontal coordinate of the centroid is reached. This will give you the mask expansion image f, which is traversed from the left. md ;

[0106] Traverse the image data from four directions to the centroid center, and perform a union operation on the portions of the data with a value of 255 in the four images, as shown in equation (V):

[0107] f = f ml ∪f mr ∪f mu ∪f md (V)

[0108] The final image f is the mask image after asymmetric dilation, used for dynamic background removal in subsequent videos. Specifically, when the centroid coordinate is 0, indicating no occlusion in the image (i.e., all data is 0), no further mask area expansion calculation is performed.

[0109] Example 3

[0110] The difference between the surveillance video recommendation method based on dynamic background removal described in Embodiment 1 or 2 is that:

[0111] Moving target detection involves detecting moving targets in surveillance video that has been obscured by a dynamic background. This includes detecting the presence of moving targets in each frame of the monitored video, such as:

[0112] For each frame of image after background occlusion processing, activity points are counted. If the area of ​​active points in 10 consecutive frames exceeds 10% of the area of ​​the unoccluded image, then a moving target is considered to exist in the last frame. When a moving target is detected, the relevant monitoring information is recommended to the server for subsequent complex function testing, or recommended to the monitoring center's large screen for display.

[0113] Moving target detection, including:

[0114] When processing the calculated mask image, first set the part of image f with data of 255 to 0, and set the part with data of 0 to 255 to obtain the final mask image f1; this step is to facilitate subsequent calculations.

[0115] The video to be detected is converted into a grayscale image. The mask image f1 obtained through asymmetric dilation is then ANDed with the video image to obtain video image data with dynamic background removed. Post-processing only targets this part of the data to detect whether there are moving targets. The logical AND operation is performed by ANDing the value of each pixel in the image with the value of the corresponding position in the mask, such as: 1·1=1, 1·0=0, 0·0=0.

[0116] In an occluded image, the dynamic background appears black. When detecting moving targets, only the unoccluded portion of the image is counted. In the dynamic background removal algorithm, the binarized grayscale difference image f is used in the activity point counting step. dif-thr The calculation method involves performing activity point statistics on video images with dynamic backgrounds removed, that is, calculating the proportion of data with values ​​greater than 40 in the two-dimensional array corresponding to each frame of the image to the total data corresponding to all unoccluded areas.

[0117] Let f be the grayscale image of the surveillance camera after occlusion. c f is obtained after performing a difference operation on two adjacent frames. c-dif Values ​​greater than 40 are designated as active points. The proportion r of all data points with values ​​greater than 40 to all unoccluded image data is calculated. x Calculate the proportion of active points in 10 consecutive frames, r1, r2, ..., r 10 When all proportions are greater than 10%, it is assumed that there is a moving target in the 10th frame image and that there is a moving target in the output image. When a moving target is detected in the monitoring image, the relevant monitoring information is recommended to the server for subsequent complex function testing, or recommended to the monitoring center screen for display.

[0118] Example 4

[0119] The difference between the surveillance video recommendation method based on dynamic background removal described in any of Examples 1-3 is as follows:

[0120] This embodiment provides a dynamic background removal method using a mask image, the specific process of which is as follows: Figure 4 As shown.

[0121] The surveillance video stream data is obtained through the configuration file of the surveillance camera. The configuration file contains 22 variables related to the camera. The variable information in the surveillance camera's configuration file is as follows:

[0122]

[0123] The video stream is a stable transmission of video data, received as a continuous series of color images. The surveillance camera's image transmission rate is 25 frames per second. Upon receiving the data, the images are first converted to grayscale, transforming each frame into a grayscale image for easier subsequent activity point statistics. 251 consecutive grayscale image data frames f1, f2, ..., f... are used. 251 250 difference images f are obtained by performing difference calculations on two adjacent grayscale images. d1 f d2 , ..., f d250 The difference images are binarized by setting a threshold, resulting in images where pixel data consists of only two values: 0 and 255. The data for each pixel in 250 difference images is statistically analyzed, and pixels with more than 65 values ​​of 255 are designated as active points. This yields the initial mask image f. mask Because of the noise in the surveillance video data, the initial mask image cannot effectively occlude the dynamic background. Therefore, it is necessary to asymmetrically dilate the initial mask image to increase the mask area.

[0124] Asymmetric expansion of the initial mask requires calculation using the centroid of the initial mask. The coordinates x of the initial mask centroid can be obtained using formulas (III) and (IV). i x j The portion of the initial mask that is blocked extends horizontally and vertically toward the centroid to x, respectively. i x j That is, the portion of the mask occlusion below the centroid's vertical coordinate extends upwards to the x-axis. j The masking portion above extends downwards to x. j The masking portion on the left extends entirely to the right to x. i The masking portion on the right extends entirely to the left (x). i Finally, the mask images extended in the four directions are merged to obtain the mask image corresponding to the current 251 frames.

[0125] The surveillance video has a frame rate of 25 frames per second, so the mask image requires 10 seconds of normal surveillance video to generate. The video images after the 251st frame use the inflated mask image for dynamic background occlusion. The mask image is constantly updated as the video stream is transmitted, and a new mask image can be generated from the subsequent 251 frames. Therefore, the mask image is updated every 10 seconds.

[0126] Example 5

[0127] The difference between the surveillance video recommendation method based on dynamic background removal described in Example 4 and the following is:

[0128] This embodiment provides a method for moving target detection using statistical activity points, the specific process of which is as follows: Figure 5 As shown.

[0129] All pixels with a value of 255 in the calculated mask image are set to 0, and pixels with a value of 0 are set to 255 for easier subsequent calculations. A logical AND operation is performed between the inverted mask image and the grayscale image of the surveillance video to be detected. The parts of the mask image with a value of 0 represent the parts that need to be occluded. After processing, the corresponding data in the surveillance video image also becomes 0, appearing as black on the image. This part of the pixel data is not included in subsequent calculations.

[0130] Using 11 consecutive frames of grayscale images after masking c1 f c2 , ..., f c11 Ten difference images f are obtained by performing difference calculations on two adjacent grayscale images. cd1 f cd2 , ..., f cd10 The difference images are binarized by setting a threshold, resulting in images where pixel data consists of only two values: 0 and 255. The proportion of pixels with a value of 255 in each of the 10 difference images is then calculated, and the ratio f of the proportions in the 10 difference images is considered the optimal value. rate1 f rate2 , ..., f rate10 When all values ​​are greater than 0.1, the original image f of the 11th frame is considered to be... 11 If a moving target is found in a given image, the process is repeated to determine if a moving target is found in any subsequent image.

[0131] Example 6

[0132] A surveillance video recommendation system based on dynamic background removal includes:

[0133] The dynamic background removal module is configured to perform background occlusion processing on the surveillance video;

[0134] The moving target detection module is configured to detect moving targets in the monitoring video after background occlusion processing. The monitoring video containing moving targets is sent to the server for other function testing, or transmitted to the large screen in the monitoring center for display.

[0135] Example 7

[0136] A computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of a surveillance video recommendation method based on dynamic background removal as described in any of embodiments 1-5.

[0137] Example 8

[0138] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a surveillance video recommendation method based on dynamic background removal as described in any of Embodiments 1-5.

Claims

1. A method for recommending surveillance videos based on dynamic background removal, characterized in that, include: Dynamic background removal: This function removes background occlusion from surveillance videos. Moving target detection: The monitoring video after background occlusion is processed is used to detect moving targets. The monitoring video with moving targets is sent to the server for other function testing, or transmitted to the large screen in the monitoring center for display. Dynamic background removal, including: Activity point statistics: including: 1) Obtain the grayscale difference image of the surveillance video: Convert the three-channel image of the original surveillance image into a grayscale image. The formula for calculating the grayscale value (Gray) of each pixel is as follows ( As shown in the image: （ ） The processed image is a grayscale image, and the data is stored as a two-dimensional array. The grayscale value of each pixel is represented by the value at the corresponding position in the array. Calculate the grayscale image of the current image. and the grayscale image of the next frame. The absolute value difference between the two is calculated to obtain the current grayscale difference image. That is, to calculate the absolute value of the values ​​at corresponding positions in two two-dimensional arrays corresponding to two grayscale images, and save the result as a two-dimensional array of the same size; 2) Binarize the grayscale difference image to obtain the image. Create a two-dimensional array with the same dimensions as the image. It is used to record statistical information of 250 consecutive binarized grayscale difference images. Each data in 250 images at the corresponding locations Corresponding position The number of values ​​that are 255; The calculation formula is as follows ( As shown in the image: （ ） From the formula ( It can be seen that, The value range is 0-250, when When the value is greater than 65, this location in the image is considered to be dynamic background and should be removed. Set to 255; When the value is less than 65, the location in the image is considered to be either a stationary part or a part through which the target has passed. Set to 0 to obtain the initial mask image. ; Noise reduction processing: Noise reduction processing is performed on each mask image; Asymmetric dilation: Asymmetric dilation is performed on the denoised mask image to increase the area of ​​the mask image and achieve occlusion of dynamic backgrounds in the video. Moving target detection, including: For each frame of image after background occlusion processing, activity point statistics are performed. If the area of ​​activity points in 10 consecutive frames exceeds 10% of the area of ​​the unoccluded image, it is considered that there is a moving target in the last frame. When a moving target is detected, the relevant monitoring information is recommended to the server for subsequent complex function testing, or recommended to the monitoring center's large screen for display. Moving target detection, including: In subsequent processing, the calculated mask image is first processed by setting the portions of the image with a value of 255 to 0, and the portions with a value of 0 to 255, resulting in the final mask image. ; The video to be detected is converted into a grayscale image, and the mask image obtained after asymmetric dilation is... Perform a logical AND operation with the video image to obtain video image data with dynamic background removed; Active point statistics are performed on video images with dynamic backgrounds removed, that is, the proportion of data with a value greater than 40 in the two-dimensional array corresponding to each frame of the image to the total data corresponding to all unoccluded areas. Let the grayscale image of the monitoring system after occlusion be... After performing a difference operation on two adjacent frames, the result is obtained. Values ​​greater than 40 are designated as active points, and the proportion of all data points with values ​​greater than 40 to all unoccluded image data is calculated. Calculate the proportion of active points in 10 consecutive frames. When all proportions are greater than 10%, it is assumed that there is a moving target in the 10th frame image and that there is a moving target in the output image. When a moving target is detected in the monitoring image, the relevant monitoring information is recommended to the server for subsequent complex function testing, or recommended to the monitoring center screen for display.

2. The method for recommending surveillance videos based on dynamic background removal according to claim 1, characterized in that, Denoising processing, initial mask image denoising, includes: Denoising is achieved using morphological dilation and erosion: First, an opening operation is performed, followed by erosion to eliminate individual pixels. Then, a dilation operation is performed to restore the original area of ​​the normal portion, resulting in the denoised mask image. .

3. The method for recommending surveillance videos based on dynamic background removal according to claim 1, characterized in that, Asymmetric expansion includes: 1) Calculate the centroid coordinates of the mask image. The calculation process for the centroid's x-coordinate or y-coordinate is as follows: First, count the number of data points with a value of 255 in all vertical or horizontal directions to obtain an array of length 720. That is, each data point in the array is the number of data points with the same x or y coordinate that are 255. Then, multiply the data in the obtained array by the corresponding x-coordinate or y-coordinate, and then calculate the average of all the data in the array, which is the x-coordinate or y-coordinate of the centroid. According to the calculation process, the x-coordinate of the centroid ranges from 1 to 720, and the y-coordinate ranges from 1 to 576. Since the data within the mask image only includes 255 and 0, therefore, through formula (… ),Mode( Calculate the x-coordinate of the centroid. , centroid ordinate : （ ） （ ） 2) Mask area expansion; the specific calculation process is as follows: Read the image data from the leftmost column of the image array, which has a length of 576. Iterate through each column of the image data from left to right. When the data of the mask image is 255, set all subsequent image data with the corresponding y-coordinate to 255. Stop iterating when the centroid x-coordinate is reached. This gives you the mask expansion image obtained from the left-hand iteration. ; Read the image data from the rightmost column of the image array, which has a length of 576. Iterate through each column of the image data from right to left. When the data of the mask image is 255, set all subsequent image data with the corresponding y-coordinate to 255. Stop iterating when the x-coordinate of the centroid is reached. This will give you the mask expansion image obtained from the left side. ; Read the image data from the top row of the image array, which has a length of 720. Iterate through each row of the image data from top to bottom. When the data of the mask image is 255, set all subsequent image data with the corresponding horizontal coordinate to 255. Stop iterating at the horizontal coordinate of the centroid to obtain the mask expansion image traversed from the left. ; Read the image data from the bottom row of the image array, which has a length of 720. Iterate through each row of the image data from bottom to top. When the data of the mask image is 255, set all subsequent image data with the corresponding horizontal coordinate to 255. Stop iterating at the horizontal coordinate of the centroid to obtain the mask expansion image traversed from the left. ; Traverse the image data from four directions to the centroid, and perform a union operation on the portions of data with a value of 255 in the four images obtained, as shown in equation (). As shown in the image: （ ） The final image This is a mask image after asymmetric dilation, used for dynamic background removal in subsequent videos.

4. A surveillance video recommendation system based on dynamic background removal, used to implement the surveillance video recommendation method based on dynamic background removal as described in any one of claims 1-3, characterized in that, include: The dynamic background removal module is configured to perform background occlusion processing on the surveillance video; The moving target detection module is configured to detect moving targets in the monitoring video after background occlusion processing. The monitoring video containing moving targets is sent to the server for other function testing, or transmitted to the large screen in the monitoring center for display.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the surveillance video recommendation method based on dynamic background removal as described in any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the surveillance video recommendation method based on dynamic background removal as described in any one of claims 1-3.

Images