Safety video monitoring method for smart building and related device

Abstract

The invention discloses a safety video monitoring method and related device for an intelligent building, and the method comprises the steps: deploying a plurality of cameras in a key region of the intelligent building, and connecting the cameras to a central monitoring center; the key area comprises each intersection, each building unit door, each floor stairway, each floor window and a dustbin area; performing preliminary analysis on the video stream information of each camera in recent preset duration to obtain a plurality of pieces of first video stream information; analyzing the plurality of pieces of first video stream information according to a video stream optical stream analysis model to obtain an abnormal behavior of each piece of first video stream information; wherein the video stream optical flow analysis model comprises an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module; and the abnormal behavior is alarmed in real time through a monitoring center screen, and is automatically pushed to handheld monitoring equipment of security personnel. According to the invention, abnormal behaviors are detected in real time by using a video stream optical flow analysis model, and rapid response to building safety monitoring is realized.

Description

Technical Field

[0001] The present invention relates to the field of security monitoring technology, and in particular to a security video monitoring method and device, and a computing device for smart buildings. Background Art

[0002] In the current construction of smart cities, security video surveillance systems for smart buildings are increasingly valued. Traditional building security video surveillance methods mainly rely on manual monitoring and simple video recording and playback mechanisms. Although they have improved the level of building security management to a certain extent, they still have many shortcomings:

[0003] For example, there are many blind spots in monitoring. Traditional monitoring systems often only deploy cameras in a few areas such as the main entrances and exits of buildings and corridors, which cannot fully cover all potential risk points in the building. The response speed is slow. Manual monitoring requires security personnel to stare at the screen at all times. Not only is the labor intensity high, but it is also easy to fail to respond to emergencies in a timely manner due to fatigue or negligence. Even if you look for clues by replaying the video, it often takes a long time and cannot respond to emergencies quickly. The degree of intelligence is low, and it is impossible to automatically identify and analyze abnormal behaviors in the video. Even if abnormal scenes are monitored, manual judgment is required, which is inefficient and prone to errors.

[0004] To solve the above problems, the present invention uses a video stream optical flow analysis model to detect abnormal behaviors in real time to achieve a rapid response to building security monitoring. Summary of the invention

[0005] In view of the above problems, the present invention provides a security video monitoring method and device, and a computing device for smart buildings.

[0006] According to one aspect of the present invention, there is provided a security video monitoring method for a smart building, comprising:

[0007] Deploy multiple cameras in key areas of smart buildings and connect them to the central monitoring center; the key areas include intersections, unit doors, staircases, windows and window sills, trash bin areas, central squares, parking spaces, elevators, and corridors;

[0008] Performing a preliminary analysis on the video stream information of each camera with a recent preset time length to obtain a plurality of first video stream information;

[0009] Analyze the multiple first video stream information according to the video stream optical flow analysis model to obtain abnormal behaviors of each of the first video stream information; wherein the video stream optical flow analysis model includes an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module;

[0010] The abnormal behavior is reported in real time through the monitoring center screen and automatically pushed to the security personnel's handheld monitoring device.

[0011] In an optional manner, the optical flow calculation module is used to calculate the motion vector of each pixel in the video between consecutive frames; the motion is estimated by comparing the pixel intensity between adjacent frames to obtain the motion field information;

[0012] The feature extraction module is used to extract motion feature information describing motion features from the motion field information; the motion feature information includes motion speed, direction and acceleration; and optimize the extracted target contour and shape features through the motion feature information;

[0013] The abnormal behavior analysis module is used to compare the target contour and shape features with the features of the normal behavior model. If the target contour and shape features exceed the feature space of normal behavior or the set threshold range, it is determined to be an abnormal behavior.

[0014] In an optional manner, the abnormal behaviors include electric vehicles and batteries entering buildings, dropping objects from high places, crowds of people, falls, noise disturbing others, flammable and explosive items and fires, large dogs without leashes, and random parking.

[0015] In an optional manner, calculating the motion vector of each pixel in the video between consecutive frames further includes:

[0016] Convert the video frame of the video into a grayscale image, and select the center pixel of the ball as the tracking object; wherein the center pixel of the ball is obtained by a color segmentation algorithm;

[0017] A preset size window is defined around the center pixel of the ball, and the optical flow equation is solved by minimizing the square sum of the gray value changes of the pixels in the window to obtain the motion vector of the center pixel of the ball, wherein the pixels in the window have similar optical flows.

[0018] In an optional manner, the expression of the optical flow equation is:

[0019]

[0020] Where W is the preset size window; Δt is the time interval between two frames of images; are the motion vector components of the center pixel of the ball in the x and y directions; (x', y') is a pixel position in a preset size window; I is the image intensity function; I(x', y', t) is the image intensity at position (x', y') and time t.

[0021] In an optional manner, the motion vector of the center pixel of the ball is:

[0022]

[0023] Where A = ∑ (x',y')∈W [I x' (x',y',t)I y' (x',y',t)] T [I x' (x',y',t)I y' (x',y',t)], which is used to measure the gradient distribution of pixels within the window; B is the sum of the product of the temporal gradient and the spatial gradient of all pixels in the window; I x' ,I y' ,I t are the partial derivatives of the image intensity function with respect to x', y', and t respectively.

[0024] In an optional manner, the optimizing the extracted target contour and shape features through the motion feature information further includes:

[0025] Weighting the contour points of the target contour and shape features according to the movement speed and direction;

[0026] The weighted calculation formula of the contour points is:

[0027]

[0028] Among them, P i is the i-th point in the original contour; P' i is the i-th point after velocity weighting; w v is the weight factor of speed; v is the moving speed of the target; δ i =proj d (P i -p t ), δ i is the projection offset of the i-th contour point along the motion direction d, p t is the center position of the target in the current frame, proj d is the projection function along direction d; max(δ) is the value of all δ i The maximum value in ;

[0029] The weighted calculation formula of the contour point direction is:

[0030] P” i =P' i +w d .δ i

[0031] Among them, P i is the i-th point after direction weighting; w dis the weight factor of the direction; d is the moving direction of the target.

[0032] In an optional manner, comparing the target contour and shape features with features of a normal behavior model further comprises:

[0033] Comparing the differences between the target contour and shape features and the normal behavior model by Euclidean distance; wherein the contour and shape features of the target object include the positions of contour points, the aspect ratio of the shape, the circularity and the curvature distribution; the normal behavior model includes the standard deviation of the contour and shape features of the normal behavior;

[0034] If the difference between the target contour and shape features and the normal behavior model exceeds a preset threshold, it is determined to be abnormal behavior.

[0035] According to another aspect of the present invention, there is provided a security video monitoring device for a smart building, comprising:

[0036] A deployment module is used to deploy multiple cameras in key areas of the smart building and connect the cameras to the central monitoring center; the key areas include various intersections, unit doors of each building, staircases of each floor, windows and window sills of each floor, garbage bin areas, central squares, parking spaces, elevator rooms and corridors;

[0037] A preliminary analysis module, used for performing preliminary analysis on the video stream information of each camera for a recent preset time period to obtain a plurality of first video stream information;

[0038] An optical flow anomaly detection module, used to analyze the multiple first video flow information according to the video flow optical flow analysis model to obtain abnormal behaviors of each of the first video flow information; wherein the video flow optical flow analysis model includes an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module;

[0039] The alarm module is used to issue a real-time alarm for the abnormal behavior through the monitoring center screen and automatically push it to the security personnel's handheld monitoring device.

[0040] According to another aspect of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus.

[0041] According to the solution provided by the present invention, multiple cameras are deployed in key areas of smart buildings, and the cameras are connected to the central monitoring center; the key areas include each intersection, each building unit door, each floor staircase, each floor window and window sill, trash bin area, central square, parking space, elevator room and corridor; the video stream information of each camera with the latest preset time is preliminarily analyzed to obtain multiple first video stream information; the multiple first video stream information is analyzed according to the video stream optical flow analysis model to obtain the abnormal behavior of each first video stream information; wherein the video stream optical flow analysis model includes an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module; the abnormal behavior is alarmed in real time through the monitoring center screen and automatically pushed to the handheld monitoring device of the security personnel. The present invention uses the video stream optical flow analysis model to detect abnormal behavior in real time, realizes a rapid response to building safety monitoring, and at the same time reduces the pressure of manual monitoring, effectively preventing the occurrence of safety accidents.

[0042] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0044] Figure 1 A schematic diagram of a process of a security video monitoring method for a smart building according to an embodiment of the present invention is shown;

[0045] Figure 2 A schematic diagram of a security video monitoring method for a smart building according to an embodiment of the present invention is shown;

[0046] Figure 3a to Figure 3b A schematic diagram of abnormal behavior according to an embodiment of the present invention is shown;

[0047] Figure 4 A module schematic diagram of a security video monitoring device for a smart building according to an embodiment of the present invention is shown;

[0048] Figure 5 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0049] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0050] Figure 1 FIG. 1 is a flow chart of a security video monitoring method for a smart building according to an embodiment of the present invention. Specifically, Figure 1 As shown, the following steps are included:

[0051] Step S101, deploy multiple cameras in key areas of the smart building and connect the cameras to the central monitoring center; the key areas include various intersections, unit doors of each building, staircases of each floor, windows and window sills of each floor, trash bin areas, central squares, parking spaces, elevator rooms and corridors.

[0052] In this embodiment, by deploying cameras in various key areas (such as various intersections, building unit doors, stairways, windows and window sills, garbage bin areas, central squares, parking spaces, elevator rooms, and corridors), comprehensive coverage inside and outside the building is achieved to ensure that there are no blind spots in monitoring. Cameras such as high-definition network cameras, infrared night vision cameras, PTZ cameras, etc. can be selected according to the environmental characteristics and functional requirements of the monitoring area, and deployed in accordance with the principle of maximizing coverage and minimizing blind spots. The cameras are connected to the central monitoring center using wired or wireless networks to ensure stable transmission of video signals.

[0053] Step S102, performing a preliminary analysis on the video stream information of each camera with a recent preset time length to obtain a plurality of first video stream information.

[0054] By performing a preliminary analysis of the video stream information of each camera for the most recent preset time, a large amount of irrelevant video content can be filtered out, and only video clips containing potentially important information can be retained, thereby improving the overall analysis efficiency. Optionally, the video stream is subjected to quality inspection (such as clarity, noise level, etc.), and based on the results of the quality inspection, instructions are sent to each camera to automatically adjust the camera's monitoring range, focal length or recording parameters, etc., to better capture key information.

[0055] Specifically, according to actual needs and security levels, a preset duration (such as the last 5 minutes, 10 minutes, etc.) is set for each camera and stored in a temporary database to ensure that the latest and effective video stream information is captured in a timely manner.

[0056] Step S103, analyzing the multiple first video stream information according to the video stream optical flow analysis model to obtain abnormal behaviors of the respective first video stream information; wherein the video stream optical flow analysis model includes an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module.

[0057] In this embodiment, Figure 2 As shown, the optical flow calculation module is used to calculate the motion vector of each pixel in the video between consecutive frames; the motion is estimated by comparing the pixel intensity between adjacent frames to obtain the motion field information. For example, the Lucas-Kanade optical flow estimation method is used to track the motion trajectory of the pixel points in the video to generate the motion field information. The motion field information is a two-dimensional vector field, and each vector represents the displacement of the corresponding pixel point between consecutive frames.

[0058] The feature extraction module is used to extract motion feature information describing motion features from the motion field information; the motion feature information includes motion speed, direction and acceleration; and the extracted target contour and shape features are optimized by the motion feature information. The motion speed is the speed of the pixel point movement, the motion direction is the direction of the pixel point movement, and the acceleration is the rate of change of the pixel point speed. Before optimizing the extracted target contour and shape features by the motion feature information, the target contour and shape features are first extracted by edge detection and contour extraction algorithms.

[0059] The abnormal behavior analysis module is used to compare the target contour and shape features with the features of the normal behavior model. If the target contour and shape features exceed the feature space of normal behavior or the set threshold range, it is determined to be abnormal behavior. The normal behavior model can be a model trained based on a large amount of normal behavior data, including the distribution range or feature threshold of normal behavior in the feature space. By comparing the target contour and shape features with the features of the normal behavior model, it is determined whether the current behavior belongs to the category of normal behavior. If it exceeds the feature space of normal behavior or the set threshold range, it is determined to be abnormal behavior.

[0060] In an optional manner, calculating the motion vector of each pixel in the video between consecutive frames further includes:

[0061] Convert the video frame of the video into a grayscale image, and select the center pixel of the ball as the tracking object; wherein the center pixel of the ball is obtained by a color segmentation algorithm;

[0062] A preset size window is defined around the center pixel of the ball, and the optical flow equation is solved by minimizing the square sum of the gray value changes of the pixels in the window to obtain the motion vector of the center pixel of the ball, wherein the pixels in the window have similar optical flows.

[0063] In this embodiment, by converting the video frame into a grayscale image and using a color segmentation algorithm to determine the center pixel of the ball, a specific object in the video can be accurately selected and tracked, thereby achieving accurate calculation of the motion vector of the object. A preset size window is defined around the selected center pixel of the ball, and the optical flow equation is solved in the local area, thereby improving the accuracy of the optical flow calculation. Since the pixels in the window have similar optical flows, the motion characteristics of the target object can be more effectively captured through local calculation. In addition, by adjusting the size and shape of the window, it can adapt to target objects of different sizes, shapes and speeds. By calculating the local optical flow, the motion vector of the target object in the video can be calculated in real time.

[0064] Specifically, the video frame of the video is converted into a grayscale image. Since the grayscale image removes the color information and only retains the brightness information, the amount of calculation is reduced. On the grayscale image, a grayscale segmentation algorithm is used to distinguish the ball from other backgrounds or objects according to the grayscale features of the ball (such as a specific grayscale range), identify the ball and determine its center pixel. A window of a preset size is defined around the center pixel of the ball. The size of the window should be determined according to the size of the target object and the expected range of motion to ensure that sufficient motion information is captured without introducing too much background noise. The optical flow equation is solved by minimizing the square sum of the grayscale value changes of the pixels in the window in the local window. For example, the Lucas-Kanade algorithm is used to estimate the optical flow using the similarity of pixels in the local window, and the solved optical flow equation provides the motion vector of the center pixel of the ball between consecutive frames. Among them, the Lucas-Kanade algorithm assumes that the brightness (or grayscale value) of the same object remains unchanged between two consecutive frames; and the movement of pixels in a small neighborhood is consistent. The calculation steps of the Lucas-Kanade algorithm are:

[0065] Select a local window containing the center of the ball (e.g., 5×5 or 7×7 pixels), set the number of iterations and the convergence criteria.

[0066] For each pixel (x, y) within the window, its brightness remains unchanged between frames t and t+1, that is:

[0067] I(x,y,t)=I(x+dx,y+dy,t+1)

[0068] Where (dx, dy) is the motion vector of the pixel from frame t to frame t+1.

[0069] Use Taylor series expansion to linearize the above equation:

[0070] I(x+dx,y+dy,t+1)≈I(x,y,t)+I x dx+I y dy+I t

[0071] Among them, I x , I y , I t are the partial derivatives of the image in the x, y, and t directions respectively.

[0072] Since the movement of all pixels in the window is consistent, the above equation can be written in matrix form and the linear equation system can be solved (least squares method) to obtain (dx, dy).

[0073] In an optional manner, the expression of the optical flow equation is:

[0074]

[0075] Where W is the preset size window; Δt is the time interval between two frames of images; are the motion vector components of the center pixel of the ball in the x and y directions; (x', y') is a pixel position in a preset size window; I is the image intensity function; I(x', y', t) is the image intensity at position (x', y') and time t.

[0076] In this embodiment, the optical flow equation minimizes the square sum of the grayscale value changes of pixels in the window and utilizes the consistency assumption of pixel motion in the local area, which is more robust when processing areas with similar motion (such as the surface of a ball). By combining the image pyramid (in the calcOpticalFlowPyrLK function), the Lucas-Kanade algorithm can handle larger movements. Since the image pyramid reduces the image resolution layer by layer, the rough optical flow is first estimated, and then refined at a finer level, thereby achieving accurate tracking of fast motion. Compared to the global optical flow estimation method, the Lucas-Kanade algorithm operates in a local window, greatly reducing the amount of calculation and enabling real-time monitoring.

[0077] For example, suppose a ball moves from left to right on the screen, and the goal is to estimate the motion vector of the ball. Select the time interval Δt = 0.1 seconds, and set the preset size window W to 10×10 pixels. For each pixel (x', y') in the window W, construct the optical flow equation:

[0078]

[0079] The above equation can be solved using gradient descent or other optimization methods to obtain The optimal value of Pixels / second; pixels / second, the ball's speed in the x direction is 2 pixels / second, and there is no noticeable movement in the y direction.

[0080] In this embodiment, the motion vector of the center pixel of the ball is:

[0081]

[0082] Where A = ∑ (x',y')∈W [I x' (x',y',t)I y' (x',y',t)] T [I x' (x',y',t)I y' (x',y',t)], which is used to measure the gradient distribution of pixels within the window; B is the sum of the product of the temporal gradient and the spatial gradient of all pixels in the window; I x' ,I y' ,I t are the partial derivatives of the image intensity function with respect to x', y', and t respectively.

[0083] In an optional manner, the optimizing the extracted target contour and shape features through the motion feature information further includes:

[0084] Weighting the contour points of the target contour and shape features according to the movement speed and direction;

[0085] The weighted calculation formula of the contour points is:

[0086]

[0087] Among them, P i is the i-th point in the original contour; P'i is the i-th point after velocity weighting; w v is the weight factor of speed; v is the moving speed of the target; δ i =proj d (P i -p t ), δ i is the projection offset of the i-th contour point along the motion direction d, p t is the center position of the target in the current frame, proj d is the projection function along direction d; max(δ) is the value of all δ i The maximum value in ;

[0088] The weighted calculation formula of the contour point direction is:

[0089] P” i =P' i +w d .δ i

[0090] Among them, Pi is the i-th point after direction weighting; w d is the weight factor of the direction; d is the moving direction of the target.

[0091] In this embodiment, the actual contour and shape of the target can be estimated more accurately by the target's moving speed and direction. The positions of the contour points can be dynamically adjusted according to the target's moving state through weighted processing, thereby improving tracking accuracy.

[0092] For example, the moving speed of the ball v=2 pixels / frame and the direction d pointing to the right are obtained.

[0093] Set the speed weight factor w v =0.5, direction weight factor w d =0.5.

[0094] For each contour point P i , calculate its projection offset δ along the motion direction d i (Assume its maximum value is 10 pixels).

[0095] According to the above formula, for each contour point P i Perform velocity weighting to obtain P' i =P i +0.5(assuming δ i =0.5), that is, the contour point moves 0.5 pixels along the motion direction.

[0096] Then, for the weighted contour point P' i Perform direction weighting to obtain P" i =P' i +0.5δ i =P i +3, that is, the contour point moves 3 pixels along the direction of motion.

[0097] In an optional manner, comparing the target contour and shape features with features of a normal behavior model further comprises:

[0098] Comparing the differences between the target contour and shape features and the normal behavior model by Euclidean distance; wherein the contour and shape features of the target object include the positions of contour points, the aspect ratio of the shape, the circularity and the curvature distribution; the normal behavior model includes the standard deviation of the contour and shape features of the normal behavior;

[0099] If the difference between the target contour and shape features and the normal behavior model exceeds a preset threshold, it is determined to be abnormal behavior.

[0100] For example, the contour and shape features of the target population are obtained, including the location of contour points, aspect ratio of the shape, circularity, and curvature distribution. The preset threshold is set to 1.5 times the standard deviation through a pre-established normal behavior model (including the standard deviation of the contour and shape features of normal behavior).

[0101] For each contour point P i , calculate the standard deviation σ between its position and the corresponding position in the normal behavior model i The Euclidean distance d between i .

[0102] For the aspect ratio, circularity, and curvature distribution of the shape, the Euclidean distance from the normal behavior model is also calculated.

[0103] Then calculate the total difference D:

[0104]

[0105] Where N is the number of contour points. If the total difference D is greater than the preset threshold (1.5 times the standard deviation), it is determined to be abnormal behavior.

[0106] In this embodiment, Figure 3a to Figure 3b As shown, the abnormal behaviors include electric vehicles and batteries entering buildings, objects thrown from high places, crowds of people, falls, noise nuisance, explosive objects and fires, large dogs not tied to leashes, and random parking. For example, a model of electric vehicles and batteries entering buildings can be established based on the size and shape characteristics of electric vehicles, a model of objects thrown from high places can be established based on the characteristics of the falling trajectory of objects, a model of crowds can be established based on the density and distribution pattern of the crowd, and a model of noise nuisance can be established based on the spectral characteristics of the sound.

[0107] Taking the sound nuisance model as an example, multiple microphones (or cameras with audio) are installed in the community to capture sound signals, and the sound signals are converted into spectrograms using Fourier transform to analyze the main frequency components, energy distribution and other characteristics in the spectrogram.

[0108] Collect normal sound samples (such as daily conversations, vehicle driving sounds) and disturbing sound samples (such as playing music loudly, honking the horn for a long time, etc.).

[0109] A normal sound model and a disturbing sound model are established based on the above samples. The normal sound model includes the standard deviation of the spectrum characteristics of the normal sound. The disturbing sound model includes the standard deviation of the spectrum characteristics of the disturbing sound.

[0110] Then, by comparing the difference between the spectral characteristics of the sound signal and the sound nuisance model, it can be determined whether there is nuisance behavior. For example, if there is an obvious peak in the sound signal and the frequency of the peak is similar to the common frequency in the nuisance sound model, then there is nuisance behavior.

[0111] Step S104: a real-time alarm is issued for the abnormal behavior through the monitoring center screen, and the alarm is automatically pushed to the handheld monitoring device of the security personnel.

[0112] For example, an alarm message is displayed on the large screen of the monitoring center: "Warning! Electric vehicles and batteries enter the building, location: entrance of Building 1, time: 2024-08-29 10:02". The alarm message is automatically pushed to the handheld monitoring device of the security personnel. After receiving the alarm message, the security personnel quickly go to the entrance of Building 1 to deal with the situation of electric vehicles and batteries entering the building.

[0113] According to the solution provided by the present invention, multiple cameras are deployed in key areas of smart buildings, and the cameras are connected to the central monitoring center; the key areas include each intersection, each building unit door, each floor staircase, each floor window and window sill, trash bin area, central square, parking space, elevator room and corridor; the video stream information of each camera with the latest preset time is preliminarily analyzed to obtain multiple first video stream information; the multiple first video stream information is analyzed according to the video stream optical flow analysis model to obtain the abnormal behavior of each first video stream information; wherein the video stream optical flow analysis model includes an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module; the abnormal behavior is alarmed in real time through the monitoring center screen and automatically pushed to the handheld monitoring device of the security personnel. The present invention uses the video stream optical flow analysis model to detect abnormal behavior in real time, realizes a rapid response to building safety monitoring, and at the same time reduces the pressure of manual monitoring, effectively preventing the occurrence of safety accidents.

[0114] Figure 4 The schematic diagram of the structure of the security video monitoring device for smart buildings according to an embodiment of the present invention is shown. The security video monitoring device for smart buildings includes:

[0115] A deployment module 410 is used to deploy multiple cameras in key areas of the smart building and connect the cameras to the central monitoring center; the key areas include each intersection, each building unit door, each floor staircase, each floor window and window sill, trash bin area, central square, parking space, elevator room and corridor;

[0116] A preliminary analysis module 420 is used to perform preliminary analysis on the video stream information of each camera for a recent preset time period to obtain a plurality of first video stream information;

[0117] An optical flow anomaly detection module 430 is used to analyze the multiple first video flow information according to the video flow optical flow analysis model to obtain abnormal behaviors of each of the first video flow information; wherein the video flow optical flow analysis model includes an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module;

[0118] The alarm module 440 is used to issue a real-time alarm for the abnormal behavior through the monitoring center screen and automatically push it to the handheld monitoring device of the security personnel.

[0119] Figure 5 The schematic diagram of the structure of the computing device embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0120] like Figure 5 As shown, the computing device may include: a processor (processor) 502 , a communications interface (Communications Interface) 504 , a memory (memory) 506 , and a communication bus 508 .

[0121] The processor 502, the communication interface 504, and the memory 506 communicate with each other via the communication bus 508. The communication interface 504 is used to communicate with other devices such as a client or other server network elements. The processor 502 is used to execute the program 510, which can specifically execute the relevant steps in the above-mentioned embodiment of the security video monitoring method for smart buildings.

[0122] Specifically, the program 510 may include program codes, which include computer operation instructions.

[0123] The processor 502 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computing device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0124] The memory 506 is used to store the program 510. The memory 506 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0125] According to the solution provided by the present invention, multiple cameras are deployed in key areas of smart buildings, and the cameras are connected to the central monitoring center; the key areas include each intersection, each building unit door, each floor staircase, each floor window and window sill, trash bin area, central square, parking space, elevator room and corridor; the video stream information of each camera with the latest preset time is preliminarily analyzed to obtain multiple first video stream information; the multiple first video stream information is analyzed according to the video stream optical flow analysis model to obtain the abnormal behavior of each first video stream information; wherein the video stream optical flow analysis model includes an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module; the abnormal behavior is alarmed in real time through the monitoring center screen and automatically pushed to the handheld monitoring device of the security personnel. The present invention uses the video stream optical flow analysis model to detect abnormal behavior in real time, realizes a rapid response to building safety monitoring, and at the same time reduces the pressure of manual monitoring, effectively preventing the occurrence of safety accidents.

[0126] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition they can be divided into multiple submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner can be combined in any combination. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose. In addition, those skilled in the art will appreciate that although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments means being within the scope of the present invention and forming different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination. The present invention can be implemented by hardware including several different elements and by appropriately programmed computers. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The steps in the above embodiments should not be understood as limiting the execution order unless otherwise specified.

Claims

1. A security video monitoring method for smart buildings, characterized in that: include: Deploy multiple cameras in key areas of smart buildings and connect them to the central monitoring center; the key areas include intersections, unit doors, staircases, windows and window sills, trash bin areas, central squares, parking spaces, elevators, and corridors; Performing a preliminary analysis on the video stream information of each camera with a recent preset time length to obtain a plurality of first video stream information; Analyze the multiple first video stream information according to the video stream optical flow analysis model to obtain abnormal behaviors of each of the first video stream information; wherein the video stream optical flow analysis model includes an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module; The abnormal behavior is reported in real time through the monitoring center screen and automatically pushed to the security personnel's handheld monitoring device.

2. The security video monitoring method for smart buildings according to claim 1, characterized in that: The optical flow calculation module is used to calculate the motion vector of each pixel in the video between consecutive frames; the motion is estimated by comparing the pixel intensity between adjacent frames to obtain the motion field information; The feature extraction module is used to extract motion feature information describing motion features from the motion field information; the motion feature information includes motion speed, direction and acceleration; and optimize the extracted target contour and shape features through the motion feature information; The abnormal behavior analysis module is used to compare the target contour and shape features with the features of the normal behavior model. If the target contour and shape features exceed the feature space of normal behavior or the set threshold range, it is determined to be an abnormal behavior.

3. The security video monitoring method for smart buildings according to claim 1 or 2, characterized in that: The abnormal behaviors include electric vehicles and batteries entering buildings, objects being thrown from high places, crowds of people, falls, noise disturbing others, flammable and explosive items and fires, large dogs not being tied up, and random parking.

4. The security video monitoring method for smart buildings according to claim 3, characterized in that: The calculating of the motion vector of each pixel in the video between consecutive frames further comprises: Convert the video frame of the video into a grayscale image, and select the center pixel of the ball as the tracking object; wherein the center pixel of the ball is obtained by a color segmentation algorithm; A preset size window is defined around the center pixel of the ball, and the optical flow equation is solved by minimizing the square sum of the gray value changes of the pixels in the window to obtain the motion vector of the center pixel of the ball, wherein the pixels in the window have similar optical flows.

5. The security video monitoring method for smart buildings according to claim 4, characterized in that: The expression of the optical flow equation is: Where W is the preset size window; Δt is the time interval between two frames of images; are the motion vector components of the center pixel of the ball in the x and y directions; (x', y') is a pixel position in a preset size window; I is the image intensity function; I(x', y', t) is the image intensity at position (x', y') and time t.

6. The security video monitoring method for smart buildings according to claim 5, characterized in that: The motion vector of the center pixel of the ball is: Where A = Σ (x',y')∈W [I x' (x',y',t) I y' (x',y',t)] T [I x' (x',y',t) I y' (x',y',t)], which is used to measure the gradient distribution of pixels within the window; B is the sum of the product of the temporal gradient and the spatial gradient of all pixels in the window; I x' ,I y' ,I t are the partial derivatives of the image intensity function with respect to x', y', and t respectively.

7. The security video monitoring method for smart buildings according to claim 2, characterized in that: The object contour and shape features extracted by optimizing the motion feature information further include: Weighting the contour points of the target contour and shape features according to the movement speed and direction; The weighted calculation formula of the contour points is: Where P is the i-th point in the original contour; P' i is the i-th point after velocity weighting; w v is the weight factor of speed; v is the moving speed of the target; δ i =proj d (P i -p t ), δ i is the projection offset of the i-th contour point along the motion direction d, p t is the center position of the target in the current frame, proj d is the projection function along direction d; max(δ) is the value of all δ i The maximum value in ; The weighted calculation formula of the contour point direction is: P” i =P’ i +w d .δ i Among them, P i is the i-th point after direction weighting; w d is the weight factor of the direction; d is the moving direction of the target.

8. The security video monitoring method for smart buildings according to claim 2, characterized in that: The comparing the target contour and shape features with the features of the normal behavior model further comprises: Comparing the differences between the target contour and shape features and the normal behavior model by Euclidean distance; wherein the contour and shape features of the target object include the positions of contour points, the aspect ratio of the shape, the circularity and the curvature distribution; the normal behavior model includes the standard deviation of the contour and shape features of the normal behavior; If the difference between the target contour and shape features and the normal behavior model exceeds a preset threshold, it is determined to be abnormal behavior.

9. A security video monitoring device for smart buildings, characterized in that: include: A deployment module is used to deploy multiple cameras in key areas of the smart building and connect the cameras to the central monitoring center; the key areas include various intersections, unit doors of each building, staircases of each floor, windows and window sills of each floor, trash bin areas, central squares, parking spaces, elevator rooms and corridors; A preliminary analysis module, used for performing preliminary analysis on the video stream information of each camera for a recent preset time period to obtain a plurality of first video stream information; An optical flow anomaly detection module, used to analyze the multiple first video flow information according to the video flow optical flow analysis model to obtain abnormal behaviors of each of the first video flow information; wherein the video flow optical flow analysis model includes an optical flow calculation module, a feature extraction module and an abnormal behavior analysis module; The alarm module is used to issue a real-time alarm for the abnormal behavior through the monitoring center screen and automatically push it to the security personnel's handheld monitoring device.

10. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned security video surveillance method for smart buildings.

Images