Emergency event early warning method and device, electronic equipment and storage medium

Abstract

The application relates to an artificial intelligence technology and discloses an emergency early warning method, which comprises the following steps: dividing a monitoring image into multiple image subblocks, respectively extracting motion features, size features and texture features in the image subblocks, constructing feature probabilities corresponding to the motion features and the size features, calculating a matching correlation coefficient between the texture features and a preset feature codebook, performing initial abnormal behavior detection and secondary abnormal behavior detection according to the matching correlation coefficient and the feature probabilities, comparing a standard abnormal behavior detection result with an emergency early warning library to determine whether an emergency has occurred, and performing early warning. The application further discloses an emergency early warning device, an electronic device and a computer readable storage medium. The application can solve the problem of low accuracy of emergency early warning.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to an emergency early warning method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] With the continuous advancement of urbanization in China, the population of cities is growing year by year. Consequently, there are more and more large public buildings providing various social life services to meet the increasing demand for material and cultural life. However, as the number of public buildings increases, they also become a place where people gather. In order to ensure that accidents can be timely warned when people gather, an emergency early warning method is needed. SUMMARY

[0003] The present application provides an emergency early warning method, device and computer readable storage medium, which mainly aims to solve the problem of low accuracy of emergency early warning.

[0004] To achieve the above purpose, the present application provides an emergency early warning method, which comprises:

[0005] Obtaining a monitoring image from video data recorded by a plurality of monitoring devices based on a preset distributed framework;

[0006] Segmenting the monitoring image into a plurality of image sub-blocks meeting a preset segmentation condition, and extracting motion features, size features and texture features in the image sub-blocks respectively;

[0007] Respectively constructing feature probabilities corresponding to the motion features and the size features, and calculating a matching correlation coefficient between the texture features and a preset feature codebook;

[0008] Constructing an abnormal behavior detection system according to a first classifier and a second classifier constructed in advance, performing initial abnormal behavior detection on the feature probabilities and the matching correlation coefficient based on the abnormal behavior detection system, and obtaining an initial abnormal behavior detection result;

[0009] Obtaining an abnormal behavior image corresponding to the initial abnormal behavior detection result, detecting a behavior body and body position information corresponding to the behavior body in the abnormal behavior image using a target detection algorithm, inputting the body position information into a symmetric space transformation network, and obtaining a plurality of body target generated skeleton graphs;

[0010] Extracting body skeleton points in the plurality of body target generated skeleton graphs, calculating skeleton transformation characteristic values corresponding to the body skeleton points, performing re-abnormal behavior detection according to the skeleton transformation characteristic values, and obtaining a standard abnormal behavior detection result;

[0011] The standard abnormal behavior detection result is compared with a pre-constructed emergency early warning library to determine whether an emergency has occurred and to provide an early warning.

[0012] Optionally, the extracting the motion feature in the image sub-block comprises:

[0013] Obtaining a foreground part in the image sub-block, and extracting a horizontal optical flow corresponding to any pixel point in the foreground part in a horizontal coordinate axis direction and a vertical optical flow corresponding to the pixel point in a vertical coordinate axis direction;

[0014] Substituting the horizontal optical flow and the vertical optical flow into a preset motion feature calculation formula to obtain the motion feature of the image sub-block.

[0015] Optionally, the preset motion feature calculation formula is:

[0016]

[0017] wherein M is the motion feature in the image sub-block, N f is a total number of pixel points in the foreground part in the image sub-block, represents the horizontal optical flow corresponding to any pixel point in the foreground part in the horizontal coordinate axis direction, represents the vertical optical flow corresponding to the pixel point in the vertical coordinate axis direction, and n is the nth pixel in the foreground part.

[0018] Optionally, the extracting the size feature in the image sub-block comprises:

[0019] Obtaining a preset Gaussian template and preset first and second parameters, and calculating an initial feature of the image sub-block according to the Gaussian template;

[0020] Substituting the initial feature and a foreground proportion calculated according to the first and second parameters into a preset size feature calculation formula to obtain the size feature in the image sub-block.

[0021] Optionally, the extracting the texture feature in the image sub-block comprises:

[0022] Obtaining a plurality of target directions preset in advance, and extracting a plurality of amplitude values in the image sub-block from the plurality of target directions by using a two-dimensional filter;

[0023] Processing the plurality of amplitude values to obtain the texture feature in the image sub-block.

[0024] Optionally, the initial abnormal behavior detection is performed on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result, including:

[0025] The first classifier in the abnormal behavior detection system is used to perform speed judgment on the feature probability corresponding to the motion feature to obtain a speed judgment result.

[0026] When the speed judgment result is speed abnormality, the target monitoring image is determined as an abnormal image.

[0027] When the speed judgment result is speed normality, the second classifier in the abnormal behavior detection system is used to perform feature judgment on the feature probability corresponding to the size feature and the matching correlation coefficient, and when the feature judgment fails, the target monitoring image is determined as an abnormal image and as the initial abnormal behavior detection result.

[0028] Optionally, before the determination of whether a sudden event occurs by comparing the standard abnormal behavior detection result with a pre-constructed sudden event early warning library, the method further includes:

[0029] The historical abnormal behavior in the pre-acquired historical reference image is recognized, and an abnormal behavior category corresponding to the historical abnormal behavior is analyzed.

[0030] A corresponding sudden event type is obtained according to the abnormal behavior category and the performance analysis corresponding to the abnormal behavior category.

[0031] The sudden event types are summarized to obtain a sudden event early warning library.

[0032] To solve the above problems, the application further provides a sudden event early warning device, which includes:

[0033] The feature extraction module is configured to acquire monitoring images from video data recorded by a plurality of monitoring devices based on a pre-set distributed framework, segment the monitoring images into a plurality of image sub-blocks meeting a pre-set segmentation condition, and extract motion features, size features and texture features in the image sub-blocks, respectively.

[0034] The initial abnormal detection module is configured to construct feature probabilities corresponding to the motion features and the size features, respectively, calculate a matching correlation coefficient between the texture features and a pre-set feature codebook, construct an abnormal behavior detection system according to a pre-constructed first classifier and a second classifier, perform initial abnormal behavior detection on the feature probabilities and the matching correlation coefficient based on the abnormal behavior detection system, and obtain an initial abnormal behavior detection result.

[0035] The standard abnormal behavior detection module is configured to acquire an abnormal behavior image corresponding to the initial abnormal behavior detection result, detect a behavior body and body position information corresponding to the behavior body in the abnormal behavior image by using a target detection algorithm, input the body position information into a symmetric space transformation network to obtain a plurality of body target generated skeleton graphs, extract body skeleton points in the plurality of body target generated skeleton graphs, calculate skeleton transformation characteristic values corresponding to the body skeleton points, perform re-abnormal behavior detection according to the skeleton transformation characteristic values, and obtain a standard abnormal behavior detection result.

[0036] The event comparison module is configured to compare the standard abnormal behavior detection result with a pre-constructed emergency warning library to determine whether an emergency event occurs and perform warning.

[0037] To solve the above problems, the present application further provides an electronic device, which comprises:

[0038] a memory configured to store at least one instruction; and

[0039] a processor configured to execute the instruction stored in the memory to implement the emergency warning method.

[0040] To solve the above problems, the present application further provides a computer readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the emergency warning method.

[0041] In the embodiment of the present application, the monitoring image is divided into a plurality of image subblocks meeting a preset segmentation condition, and motion features, size features and texture features in the image subblocks are extracted respectively, different image-related features are acquired from the perspective of local features for initial abnormal behavior detection and standard abnormal behavior detection, so that the detection accuracy is higher. The emergency warning library and the abnormal behavior detection result are compared, the emergency warning library contains a plurality of emergency events, and when the comparison is consistent, the event to which the target monitoring image belongs is determined as an emergency event and warning is given. Therefore, the emergency warning method, device, electronic device and computer readable storage medium provided by the present application can solve the problem of low accuracy of emergency warning. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A flowchart of an emergency warning method provided by an embodiment of the present application is shown in the figure;

[0043] Figure 2 A functional module diagram of an emergency warning device provided by an embodiment of the present application is shown in the figure;

[0044] Figure 3A structural schematic diagram of an electronic device for implementing the emergency early warning method is provided in an embodiment of the present application.

[0045] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

[0047] An emergency early warning method is provided in the embodiments of the present application. The execution subject of the emergency early warning method includes but is not limited to at least one of the electronic devices such as a server and a terminal, which can be configured to execute the method provided in the embodiments of the present application. In other words, the emergency early warning method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0048] Reference Figure 1 FIG. 1 shows a flowchart of an emergency early warning method provided in an embodiment of the present application. In the embodiment, the emergency early warning method includes:

[0049] S1, obtaining a monitoring image from video data recorded by a plurality of monitoring devices based on a preset distributed framework.

[0050] In the embodiments of the present application, the monitoring device can refer to a plurality of cameras at different positions or other devices with a photography function. It is assumed that a target square is monitored. In order to achieve full coverage monitoring of the target square without dead angles, a plurality of monitoring cameras can be deployed at different viewing angles of the target square, and a plurality of monitoring cameras can also be deployed at a plurality of viewing angles of a plurality of intersections leading to the target square. A large number of monitoring cameras form a video monitoring network based on the Internet of Things, and a large amount of video data recorded by the monitoring devices is obtained.

[0051] Further, since the positions of the monitoring cameras are different, in order to alleviate the network transmission pressure, the video data recorded by the plurality of monitoring devices can be transmitted to a distributed system formed by a local area network, and the distributed framework in the distributed system is used for analysis. The distributed framework refers to a MapReduce framework, which is a distributed computing framework with the characteristics of effectiveness, fault tolerance and large-scale parallel capability.

[0052] In detail, the MapReduce framework can implement parallel processing of video data recorded by different monitoring devices, and output the processed effective video image as a standard monitoring image.

[0053] Further, after obtaining the monitoring image, the monitoring image can be further subjected to denoising processing. The denoising processing refers to a process of reducing noise in the image. Noise is an important cause of image interference. A piece of image may have various noises in actual application. The noise may be generated in transmission or in quantization processing, and thus needs to be subjected to denoising processing, so that the standard monitoring image after denoising processing is more accurate.

[0054] Preferably, the denoising processing can be implemented by a filter-based method, a model-based method and a learning-based method.

[0055] The filter-based method refers to removing image noise by using a designed low-pass filter. For example, the low-pass filter can be a median filter or an adaptive Wiener filter. The median filter is a commonly used nonlinear smoothing filter, and the basic principle is to replace the value of a point in a digital image with the median value of the points in the field of the point. The adaptive Wiener filter adjusts the output of the filter according to the local variance of the image. The model-based method defines the denoising task as a maximum posteriori optimization problem for solving. The learning-based method is implemented according to a deep network.

[0056] S2, the standard monitoring image is segmented into a plurality of image subblocks meeting a preset segmentation condition, and motion features, size features and texture features in the image subblocks are extracted respectively.

[0057] In the embodiment of the application, the preset segmentation condition refers to an image size set in advance and fixed images that are not overlapped, and the standard monitoring image is segmented into a plurality of image subblocks that are not overlapped. The image size set in advance can be adjusted according to application situations, and the segmentation into a plurality of image subblocks that are not overlapped can extract features of each local area, so as to obtain detection results of each subblock, which can filter and exclude irrelevant backgrounds and make the feature aggregation mode more flexible. In addition, the detection results of each image subblock can be used to judge the global situation, so that the global early warning detection is more accurate.

[0058] Further, the motion features in the image subblock can be extracted by the following method:

[0059] Obtaining a foreground part in the image subblock, and extracting a horizontal optical flow corresponding to an arbitrary pixel point in the foreground part in a horizontal coordinate axis direction and a vertical optical flow corresponding to the arbitrary pixel point in a vertical coordinate axis direction;

[0060] The lateral optical flow and the longitudinal optical flow are substituted into a preset motion feature calculation formula to obtain the motion feature of the image sub-block.

[0061] In detail, the optical flow refers to a concept in object motion detection in a visual field, and is used to describe the motion of an observed target, surface or edge caused by relative motion of an observer. Any pixel point in the foreground part corresponds to a lateral optical flow in the horizontal coordinate axis direction, which is Any pixel point in the foreground part corresponds to a longitudinal optical flow in the vertical coordinate axis direction, which is The preset motion feature calculation formula is:

[0062]

[0063] Wherein, M is the motion feature in the image sub-block, N f is the total number of pixel points in the foreground part of the image sub-block, represents the lateral optical flow corresponding to any pixel point in the foreground part in the horizontal coordinate axis direction, represents the longitudinal optical flow corresponding to any pixel point in the foreground part in the vertical coordinate axis direction, and n is the nth pixel in the foreground part.

[0064] Further, the size feature in the image sub-block is extracted by the following method:

[0065] A preset Gaussian template and preset first and second parameters are obtained, and the initial feature of the image sub-block is calculated according to the Gaussian template;

[0066] The initial feature and the foreground proportion calculated according to the first and second parameters are substituted into a preset size feature calculation formula to obtain the size feature in the image sub-block.

[0067] In detail, the preset Gaussian template is G, which can be a 3x3 Gaussian template in the present scheme, the first parameter is a, and the second parameter is b.

[0068] Preferably, the motion feature cannot fully express the information in the image, for example, it cannot distinguish between people and objects with the same motion speed, and in order to improve the detection effect, the size of the target in the foreground needs to be considered.

[0069] Specifically, the preset size feature calculation formula is:

[0070] S = ∑∑G(a-i+1,b-j+1)o(a,b)

[0071] Wherein, S is the size feature, G is the Gaussian template, a is the first parameter, b is the second parameter, o(a,b) is the foreground proportion, G(a-i+1,b-j+1) is the initial feature, i and j are the image positions of the image sub-block.

[0072] Further, the texture feature in the image sub-block is extracted by the following method:

[0073] A plurality of target directions are acquired, and a two-dimensional filter is used to extract a plurality of amplitude values in the image sub-block from the plurality of target directions.

[0074] The plurality of amplitude values are summarized to obtain the texture feature in the image sub-block.

[0075] In detail, the plurality of target directions can be 0, 45, 90 and 135 degrees, and a two-dimensional Gabor filter can be used to filter the picture in the four directions of 0, 45, 90 and 135 degrees to obtain a plurality of amplitude values m0, m 45 , m 90 and m 135 . The plurality of amplitude values are summarized to obtain the texture feature in the image sub-block as T=[m0m 45 m 90 m 135 ].

[0076] Preferably, the texture feature can use the texture inside the sub-block to screen the target.

[0077] S3, the feature probability corresponding to the motion feature and the size feature is respectively constructed, and a matching correlation coefficient between the texture feature and a preset feature codebook is calculated.

[0078] In the embodiment of the application, for the two basic features of the motion feature and the size feature, since the motion and the size of the target will change in different scenes, the semi-parametric can adapt to such changes, and therefore a relatively smooth model needs to be established.

[0079] Specifically, in the embodiment of the application, the feature probability corresponding to the motion feature is:

[0080]

[0081] Wherein, f(sΔx) is the feature probability corresponding to the motion feature, h is the bandwidth of the Gaussian kernel, sΔx represents the effective upper limit of the motion feature, N is the number of the image sub-blocks, Δx is the motion gap, and x n represents the motion parameter of the nth image sub-block.

[0082] Furthermore, the feature probability corresponding to the size feature is consistent with the feature probability corresponding to the motion feature, but the parameters are different, which will not be described here.

[0083] Specifically, the calculating the matching correlation coefficient between the texture feature and the preset feature codebook includes:

[0084]

[0085] Among them, p(T,c k ) is the matching correlation coefficient between the texture feature and the preset feature codebook, T is the texture feature, c k is an entry in the preset feature codebook, μ T is the element mean corresponding to the texture feature, is the element mean corresponding to the entry.

[0086] S4. Construct an abnormal behavior detection system based on the pre-constructed first classifier and the second classifier, and perform initial abnormal behavior detection on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result.

[0087] In this embodiment of the present invention, abnormal behavior is determined based on the acquired first and second classifiers. Different classifiers are used for abnormality detection in different dimensions. The first classifier is used for speed detection. A speed slower or faster than normal can be considered abnormal. Multiple normal speeds can be set in advance for reference. The second classifier is used to consider target size and texture features.

[0088] Specifically, using only target size cannot distinguish large objects or small groups of objects, leading to false detections. Furthermore, extracting textures with few foreground pixels primarily extracts background texture, so using texture features alone to identify anomalies can also result in high false detections. Therefore, it is necessary to consider both features in a complementary manner to improve detection performance.

[0089] Specifically, the performing initial abnormal behavior detection on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result includes:

[0090] Using the first classifier in the abnormal behavior detection system to perform speed judgment on the feature probability corresponding to the motion feature to obtain a speed judgment result;

[0091] When the speed judgment result is that the speed is abnormal, the target monitoring image is judged as an abnormal image;

[0092] When the speed judgment result is normal speed, the feature probability corresponding to the size feature and the matching correlation coefficient are subjected to feature judgment by a second classifier in the abnormal behavior detection system, and when the feature judgment fails, the target monitoring image is determined as an abnormal image and as an initial abnormal behavior detection result.

[0093] In detail, the abnormal behavior judgment in combination with different features can improve the accuracy of abnormal behavior detection.

[0094] S5, obtain an abnormal behavior image corresponding to the initial abnormal behavior detection result, detect a behavior human body in the abnormal behavior image and human body position information corresponding to the behavior human body by using a target detection algorithm, input the human body position information into a symmetric space transformation network to obtain a plurality of human target generated skeleton graphs.

[0095] In the embodiment of the application, the initial abnormal behavior detection result is to initially detect a standard monitoring image, judge whether an abnormal behavior appears in the standard monitoring image from multiple angles such as motion feature, size feature and texture feature, locate the abnormal behavior image corresponding to the initial abnormal behavior detection result, and perform re-detection on the abnormal behavior image to further determine whether there is an abnormality, thereby improving the accuracy of abnormal behavior judgment.

[0096] Specifically, the behavior human body in the abnormal behavior image is detected by using a target detection algorithm, wherein the target detection algorithm can be a sampling region-based convolutional neural network, which mainly realizes image recognition through window selection, generates a candidate frame, classifies the candidate frame, judges whether there is a target in each candidate frame, obtains the behavior human body in the abnormal behavior image, and extracts human body position information corresponding to the behavior human body, inputs the human body position information in the image into a symmetric space transformation network to generate a plurality of human target skeleton graphs.

[0097] The symmetric space transformation network is composed of a space transformation network, a single person posture detector and a reverse space transformation network, the space transformation network extracts human region information in the candidate frame as input of the single person posture detector, the single person posture detector realizes single person posture estimation in the candidate frame, and the reverse space transformation network remaps the human posture detected by the single person posture detector back to the image to obtain a plurality of human target generated skeleton graphs.

[0098] S6, extract human skeleton points in a plurality of the human target generated skeleton graphs, calculate skeleton transformation characteristic values corresponding to the human skeleton points, perform re-abnormal behavior detection according to the skeleton transformation characteristic values to obtain a standard abnormal behavior detection result.

[0099] In the embodiment of the present application, the human body skeleton points in the human body skeleton graph generated by extracting a plurality of human body targets are obtained by using the strategy of non-maximum suppression (NMS), wherein the non-maximum suppression is to suppress elements that are not maximum values, which can be understood as local maximum search. The local represents a neighborhood, and the neighborhood has two variable parameters, namely the dimension of the neighborhood and the size of the neighborhood.

[0100] Specifically, the skeleton transformation characteristic value corresponding to the human body skeleton point is calculated, including:

[0101] The human body skeleton point is mapped to a preset two-dimensional rectangular coordinate system, and the first frame coordinate and the tail frame coordinate corresponding to the human body skeleton point are obtained on the preset two-dimensional rectangular coordinate system.

[0102] The motion speed corresponding to the human body skeleton point is calculated according to the first frame coordinate, the tail frame coordinate and a preset motion speed calculation formula.

[0103] The motion speed is divided into horizontal speed and vertical speed, and the motion acceleration corresponding to the human body skeleton point is calculated according to the horizontal speed, the vertical speed and a preset acceleration calculation formula.

[0104] The motion speed and the motion acceleration are summarized as the skeleton transformation characteristic value corresponding to the human body skeleton point.

[0105] Further, the preset motion speed calculation formula is:

[0106]

[0107]

[0108]

[0109] wherein, V p is the motion speed corresponding to the human body skeleton point, is the horizontal speed, is the vertical speed, f represents the acquisition frequency frame, i is the first frame value, x i，p is the horizontal coordinate value in the first frame coordinate, y i，p is the vertical coordinate value in the first frame coordinate, x i+f，p is the horizontal coordinate value in the tail frame coordinate, y i+f，p is the vertical coordinate value in the tail frame coordinate.

[0110] Specifically, the preset acceleration calculation formula is:

[0111]

[0112]

[0113]

[0114] wherein a p is the motion acceleration corresponding to the human skeleton point, is the horizontal acceleration, is the vertical acceleration, and f represents the acquisition frequency frame, is the horizontal velocity of the i+f frame, is the horizontal velocity of the i frame, is the vertical velocity of the i+f frame, is the vertical velocity of the i frame.

[0115] Further, the re-abnormal behavior detection is performed according to the skeleton transformation characteristic value, and a standard abnormal behavior detection result is obtained, that is, whether an abnormal situation occurs is determined according to the size between the motion velocity and the motion acceleration in the skeleton transformation characteristic value and the velocity threshold and the acceleration threshold. This is because the human body has a close relationship with common abnormal behaviors when abnormal behaviors occur, such as the hand joint, the elbow joint, the foot joint, the knee joint, and other skeleton points.

[0116] S7, determining whether an emergency occurs by comparing the standard abnormal behavior detection result with a pre-constructed emergency warning library, and performing a warning.

[0117] In the embodiment of the application, before determining whether an emergency occurs by comparing the standard abnormal behavior detection result with a pre-constructed emergency warning library, the method further comprises:

[0118] identifying a historical abnormal behavior in a pre-acquired historical reference image, and analyzing an abnormal behavior category corresponding to the historical abnormal behavior;

[0119] obtaining a corresponding emergency type according to the abnormal behavior category and a performance analysis corresponding to the abnormal behavior category;

[0120] summarizing a plurality of emergency types to obtain an emergency warning library.

[0121] In detail, the historical reference image refers to a monitoring image corresponding to a period in which an emergency occurred in the past. The abnormal behavior category can be a change in motion velocity, a change in crowd density, a change in human body center of gravity, a change in motion amplitude, a change in motion direction, etc. The performance corresponding to the abnormal behavior category refers to a specific behavior of a human being.

[0122] For example, the emergency generally includes natural disasters, accident disasters, public health events and social security events, and the present scheme focuses on processing the social security events, which can be the emergency in an indoor environment such as a shopping mall or an office building.

[0123] Further, the emergency warning library contains a plurality of different types of emergencies, and the abnormal behavior detection result is compared with the emergency warning library, and when the comparison is consistent, the event to which the target monitoring image belongs is determined as an emergency and is warned.

[0124] In the embodiment of the present application, the monitoring image is segmented into a plurality of image sub-blocks meeting the preset segmentation condition, the motion feature, the size feature and the texture feature in the image sub-blocks are extracted respectively, different image-related features are obtained from the perspective of local features for initial abnormal behavior detection and standard abnormal behavior detection, so that the detection accuracy is higher. The abnormal behavior detection result is compared with the emergency warning library, the emergency warning library contains a plurality of emergencies, and when the comparison is consistent, the event to which the target monitoring image belongs is determined as an emergency and is warned. Therefore, the emergency warning method provided by the present application can solve the problem of low accuracy of emergency warning.

[0125] As shown in Figure 2 is a functional module diagram of an emergency warning device provided by an embodiment of the present application.

[0126] The emergency warning device 100 can be installed in an electronic device. According to the implemented functions, the emergency warning device 100 can include a feature extraction module 101, an initial abnormal detection module 102, a standard abnormal detection module 103 and an event comparison module 104. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, and are stored in the memory of the electronic device.

[0127] In the present embodiment, the functions of each module / unit are as follows:

[0128] The feature extraction module 101 is configured to obtain monitoring images from video data recorded by a plurality of monitoring devices based on a preset distributed framework, segment the monitoring images into a plurality of image sub-blocks meeting a preset segmentation condition, and extract motion features, size features and texture features in the image sub-blocks respectively.

[0129] The initial anomaly detection module 102 is configured to construct feature probabilities corresponding to the motion features and the size features respectively, calculate a matching correlation coefficient between the texture features and a preset feature codebook, construct an anomaly behavior detection system based on a first classifier and a second classifier constructed in advance, perform initial anomaly behavior detection on the feature probabilities and the matching correlation coefficient based on the anomaly behavior detection system, and obtain an initial anomaly behavior detection result.

[0130] The standard anomaly detection module 103 is configured to acquire an anomaly behavior image corresponding to the initial anomaly behavior detection result, detect a behavior human body and human body position information corresponding to the behavior human body in the anomaly behavior image by using a target detection algorithm, input the human body position information into a symmetric space transformation network to obtain a plurality of human body target generated skeleton graphs, extract human skeleton points in the plurality of human body target generated skeleton graphs, calculate a skeleton transformation characteristic value corresponding to the human skeleton points, perform re-anomaly behavior detection based on the skeleton transformation characteristic value, and obtain a standard anomaly behavior detection result.

[0131] The event comparison module 104 is configured to compare the standard anomaly behavior detection result with a pre-constructed emergency event early warning library to determine whether an emergency event occurs and perform early warning.

[0132] In detail, the specific implementation of each module of the emergency event early warning device 100 is as follows:

[0133] Step one, based on a preset distributed framework, acquire monitoring images from video data recorded by a plurality of monitoring devices.

[0134] In the embodiment of the application, the monitoring device can refer to a plurality of cameras at different positions or other devices with photography functions. Assuming that a target square is monitored, in order to realize full coverage monitoring of the target square without dead angles, a plurality of monitoring cameras can be deployed at different viewing angles of the target square, and a plurality of monitoring cameras can also be deployed at a plurality of viewing angles of a plurality of intersections leading to the target square. A large number of monitoring cameras form a video monitoring network based on the Internet of Things, and a large amount of video data recorded by the monitoring devices is obtained.

[0135] Further, since the positions of the monitoring cameras are different, in order to alleviate the network transmission pressure, the video data recorded by the plurality of monitoring devices can be transmitted to a distributed system formed by a local area network, and the distributed system is analyzed by using a distributed framework in the distributed system. The distributed framework refers to a MapReduce framework, and the MapReduce framework is a distributed computing framework with the characteristics of effectiveness, fault tolerance and large-scale parallel capability.

[0136] In detail, the MapReduce framework can realize parallel processing of video data recorded by different monitoring devices, and output the processed effective video image as a standard monitoring image.

[0137] Further, after obtaining the monitoring image, the monitoring image can be further subjected to denoising processing. The denoising processing refers to a process of reducing noise in the image. Noise is an important cause of image interference. A piece of image may have various noises in actual application. The noise may be generated in transmission or in quantization processing, and thus needs to be subjected to denoising processing, so that the standard monitoring image after denoising processing is more accurate.

[0138] Preferably, the denoising processing can be realized by a filter-based method, a model-based method and a learning-based method.

[0139] The filter-based method refers to removing image noise by using a designed low-pass filter. For example, the low-pass filter can be a median filter or an adaptive Wiener filter. The median filter is a commonly used nonlinear smoothing filter, and the basic principle is to replace the value of a point in a digital image with the median value of the points in the field of the point. The adaptive Wiener filter adjusts the output of the filter according to the local variance of the image. The model-based method defines the denoising task as a maximum posteriori optimization problem for solving. The learning-based method is realized according to a deep network.

[0140] Step two, the standard monitoring image is segmented into a plurality of image sub-blocks meeting a preset segmentation condition, and motion features, size features and texture features in the image sub-blocks are extracted respectively.

[0141] In the embodiment of the application, the preset segmentation condition refers to an image size set in advance and fixed images without overlapping, and the standard monitoring image is segmented into a plurality of image sub-blocks without overlapping. The image size set in advance can be adjusted according to application situations, and the segmentation into a plurality of image sub-blocks without overlapping can extract features of each local area, so as to obtain detection results of each sub-block, which can screen and exclude irrelevant backgrounds and make the feature aggregation mode more flexible. In addition, the global situation can be judged according to the detection results of each image sub-block, so that the global early warning detection is more accurate.

[0142] Further, the motion features in the image sub-blocks can be extracted by the following method:

[0143] The foreground part in the image sub-block is obtained, and the horizontal optical flow corresponding to the horizontal coordinate axis and the vertical optical flow corresponding to the vertical coordinate axis of any pixel point in the foreground part are extracted;

[0144] The lateral optical flow and the longitudinal optical flow are substituted into a preset motion feature calculation formula to obtain the motion feature of the image sub-block.

[0145] In detail, the optical flow refers to a concept in object motion detection in a visual field, and is used to describe the motion of an observed target, surface or edge caused by relative motion of an observer. Any pixel point in the foreground part corresponds to a lateral optical flow in the horizontal coordinate axis direction, which is Any pixel point in the foreground part corresponds to a longitudinal optical flow in the vertical coordinate axis direction, which is The preset motion feature calculation formula is:

[0146]

[0147] Wherein, M is the motion feature in the image sub-block, N f is the total number of pixel points in the foreground part of the image sub-block, represents the lateral optical flow corresponding to any pixel point in the foreground part in the horizontal coordinate axis direction, represents the longitudinal optical flow corresponding to any pixel point in the foreground part in the vertical coordinate axis direction, and n is the nth pixel in the foreground part.

[0148] Further, the size feature in the image sub-block is extracted by the following method:

[0149] A preset Gaussian template and preset first and second parameters are obtained, and the initial feature of the image sub-block is calculated according to the Gaussian template;

[0150] The initial feature and the foreground proportion calculated according to the first and second parameters are substituted into a preset size feature calculation formula to obtain the size feature in the image sub-block.

[0151] In detail, the preset Gaussian template is G, which can be a 3x3 Gaussian template in the present scheme, the first parameter is a, and the second parameter is b.

[0152] Preferably, the motion feature cannot fully express the information in the image, for example, it cannot distinguish between people and objects with the same motion speed, and in order to improve the detection effect, the size of the target in the foreground needs to be considered.

[0153] Specifically, the preset size feature calculation formula is:

[0154] S = ∑∑G(a-i+1,b-j+1)o(a,b)

[0155] Wherein, S is the size feature, G is the Gaussian template, a is the first parameter, b is the second parameter, o(a,b) is the foreground proportion, G(a-i+1,b-j+1) is the initial feature, i and j are the image positions of the image sub-block.

[0156] Further, the texture feature in the image sub-block is extracted by the following method:

[0157] A plurality of target directions are acquired, and a two-dimensional filter is used to extract a plurality of amplitude values in the image sub-block from the plurality of target directions.

[0158] The plurality of amplitude values are summarized to obtain the texture feature in the image sub-block.

[0159] In detail, the plurality of target directions can be 0, 45, 90 and 135 degrees, and a two-dimensional Gabor filter can be used to filter the picture in the four directions of 0, 45, 90 and 135 degrees to obtain a plurality of amplitude values m0, m 45 , m 90 and m 135 . The plurality of amplitude values are summarized to obtain the texture feature in the image sub-block as T=[m0m 45 m 90 m 135 ].

[0160] Preferably, the introduction of the texture feature can use the texture inside the sub-block to screen the target.

[0161] Step three, respectively constructing the feature probability corresponding to the motion feature and the size feature, and calculating the matching correlation coefficient between the texture feature and the preset feature codebook.

[0162] In the embodiment of the application, for the two basic features of the motion feature and the size feature, since the motion and the size of the target will change in different scenes, the semi-parametric can adapt to such changes, and therefore a relatively smooth model needs to be established.

[0163] Specifically, in the embodiment of the application, the feature probability corresponding to the motion feature is:

[0164]

[0165] Wherein, f(sDelta x) is the feature probability corresponding to the motion feature, h is the bandwidth of the Gaussian kernel, sDelta x represents the effective upper limit of the motion feature, N is the number of the image sub-blocks, Delta x is the motion gap, and x n represents the motion parameter of the nth image sub-block.

[0166] Further, the feature probability corresponding to the size feature is consistent with the feature probability corresponding to the motion feature, only the parameters are different, which will not be described here.

[0167] Specifically, the matching correlation coefficient between the texture feature and the preset feature codebook is calculated, including:

[0168]

[0169] wherein p(T,c k ) is the matching correlation coefficient between the texture feature and the preset feature codebook, T is the texture feature, c k is an entry in the preset feature codebook, μ T is the element mean corresponding to the texture feature, is the element mean corresponding to the entry.

[0170] Step four, constructing an abnormal behavior detection system according to the first classifier and the second classifier, and performing initial abnormal behavior detection on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result.

[0171] In the embodiment of the application, the first classifier and the second classifier are used for abnormal behavior judgment, wherein different classifiers are used for abnormal detection in different dimensions, the first classifier is used for speed detection, and slower or faster than the normal speed can be judged as abnormal, and multiple normal speeds can be set in advance for reference. The second classifier is used to consider the size and texture features of the target.

[0172] In detail, only the target size cannot distinguish large target objects and small target groups, which leads to false detection, and when the texture containing few foreground pixels is extracted, the texture of the background is mainly extracted, so using the texture feature alone to judge the abnormality will also lead to high false detection. Therefore, both need to be considered to form a complement, thereby improving the detection performance.

[0173] Specifically, the matching correlation coefficient between the texture feature and the preset feature codebook is calculated, including:

[0174] The first classifier in the abnormal behavior detection system is used to judge the speed of the feature probability corresponding to the motion feature to obtain a speed judgment result;

[0175] When the speed judgment result is speed abnormality, the target monitoring image is determined as an abnormal image;

[0176] When the speed judgment result is normal speed, the feature probability corresponding to the size feature and the matching correlation coefficient are subjected to feature judgment by a second classifier in the abnormal behavior detection system, and when the feature judgment fails, the target monitoring image is determined as an abnormal image and as an initial abnormal behavior detection result.

[0177] In detail, abnormal behavior judgment in combination with different features can improve the accuracy of abnormal behavior detection.

[0178] Step five, an abnormal behavior image corresponding to the initial abnormal behavior detection result is obtained, a behavior human body in the abnormal behavior image is detected by a target detection algorithm, and human body position information corresponding to the behavior human body is input into a symmetric space transformation network to obtain multiple human body target generated skeleton graphs.

[0179] In the embodiment of the application, the initial abnormal behavior detection result is used to initially detect a standard monitoring image, judge whether abnormal behavior appears in the standard monitoring image from multiple angles such as motion features, size features and texture features, locate the abnormal behavior image corresponding to the initial abnormal behavior detection result, and perform re-detection on the abnormal behavior image to further determine whether there is abnormality, thereby improving the accuracy of abnormal behavior judgment.

[0180] Specifically, the behavior human body in the abnormal behavior image is detected by a target detection algorithm, wherein the target detection algorithm can be a sampling region-based convolutional neural network, which mainly realizes image recognition through window selection, generates a candidate frame, classifies the candidate frame, judges whether there is a target in each candidate frame, obtains the target, that is, the behavior human body in the abnormal behavior image, and extracts human body position information corresponding to the behavior human body, and inputs the human body position information in the image into a symmetric space transformation network to generate multiple human body target skeleton graphs.

[0181] The symmetric space transformation network is composed of a space transformation network, a single person posture detector and a reverse space transformation network, the space transformation network extracts human region information in the candidate frame as input of the single person posture detector, the single person posture detector realizes single person posture estimation in the candidate frame, and the reverse space transformation network remaps the human posture detected by the single person posture detector back to the image to obtain the multiple human body target generated skeleton graphs.

[0182] Step six, human skeleton points in the multiple human body target generated skeleton graphs are extracted, skeleton transformation characteristic values corresponding to the human skeleton points are calculated, re-abnormal behavior detection is performed according to the skeleton transformation characteristic values, and a standard abnormal behavior detection result is obtained.

[0183] In the embodiment of the present application, the human body skeleton points in the human body skeleton graph generated by extracting a plurality of human body targets are obtained by using the strategy of non-maximum suppression (NMS), wherein the non-maximum suppression is to suppress elements that are not maximum values, which can be understood as local maximum search. The local represents a neighborhood, and the neighborhood has two variable parameters, namely the dimension of the neighborhood and the size of the neighborhood.

[0184] Specifically, the skeleton transformation characteristic value corresponding to the human body skeleton point is calculated, including:

[0185] The human body skeleton point is mapped to a preset two-dimensional rectangular coordinate system, and the first frame coordinate and the tail frame coordinate corresponding to the human body skeleton point are obtained on the preset two-dimensional rectangular coordinate system.

[0186] The motion speed corresponding to the human body skeleton point is calculated according to the first frame coordinate, the tail frame coordinate and a preset motion speed calculation formula.

[0187] The motion speed is divided into horizontal speed and vertical speed, and the motion acceleration corresponding to the human body skeleton point is calculated according to the horizontal speed, the vertical speed and a preset acceleration calculation formula.

[0188] The motion speed and the motion acceleration are summarized as the skeleton transformation characteristic value corresponding to the human body skeleton point.

[0189] Further, the preset motion speed calculation formula is:

[0190]

[0191]

[0192]

[0193] V i = (x i - x i-1 ) 2 + (y i - y i-1 ) 2 f p is the motion speed corresponding to the human body skeleton point, is the horizontal speed, is the vertical speed, f represents the acquisition frequency frame, i is the first frame value, x i，p is the horizontal coordinate value in the first frame coordinate, y i，p is the vertical coordinate value in the first frame coordinate, x i+f，p is the horizontal coordinate value in the tail frame coordinate, y i+f，p is the vertical coordinate value in the tail frame coordinate.

[0194] Specifically, the preset acceleration calculation formula is:

[0195]

[0196]

[0197]

[0198] wherein, a p is the motion acceleration of the human skeleton point, is the horizontal acceleration, is the vertical acceleration, and f represents the acquisition frequency frame, is the horizontal velocity of the i+f frame, is the horizontal velocity of the i frame, is the vertical velocity of the i+f frame, is the vertical velocity of the i frame.

[0199] Further, the re-abnormal behavior detection is performed according to the skeleton transformation characteristic value, and a standard abnormal behavior detection result is obtained, that is, whether an abnormal situation occurs is determined according to the size between the motion velocity and the motion acceleration in the skeleton transformation characteristic value and the velocity threshold and the acceleration threshold. This is because the human body has a close relationship with the common abnormal behavior when the abnormal behavior occurs, and the hand joint, the elbow joint, the foot joint, the knee joint or other skeleton points.

[0200] Step seven, whether an emergency occurs is determined by comparing the standard abnormal behavior detection result with a pre-constructed emergency early warning library, and early warning is performed.

[0201] In the embodiment of the application, before determining whether an emergency occurs by comparing the standard abnormal behavior detection result with the pre-constructed emergency early warning library, the method further comprises:

[0202] identifying a historical abnormal behavior in a pre-acquired historical reference image, and analyzing an abnormal behavior category corresponding to the historical abnormal behavior;

[0203] obtaining a corresponding emergency type according to the abnormal behavior category and a performance analysis corresponding to the abnormal behavior category;

[0204] summarizing a plurality of emergency types to obtain an emergency early warning library.

[0205] In detail, the historical reference image refers to a monitoring image corresponding to a period in which an emergency occurred in the past. The abnormal behavior category can be a change in motion velocity, a change in crowd density, a change in human body gravity center, a change in motion amplitude, a change in motion direction, etc. The performance corresponding to the abnormal behavior category refers to a specific behavior of a human being.

[0206] For example, the emergency generally includes natural disasters, accident disasters, public health events and social security events, and the present scheme focuses on processing the social security events, which can be the emergency in an indoor environment such as a shopping mall or an office building.

[0207] Further, the emergency warning library contains a plurality of different types of emergencies, and the emergency warning library is compared with the abnormal behavior detection result, when the comparison is consistent, the event to which the target monitoring image belongs is determined as an emergency and is warned.

[0208] In the embodiment of the present application, the monitoring image is segmented into a plurality of image sub-blocks meeting the preset segmentation condition, the motion feature, the size feature and the texture feature in the image sub-blocks are extracted respectively, different image-related features are obtained from the perspective of local features to perform initial abnormal behavior detection and standard abnormal behavior detection, so that the accuracy of detection is higher. And the emergency warning library is compared with the abnormal behavior detection result, the emergency warning library contains a plurality of emergencies, when the comparison is consistent, the event to which the target monitoring image belongs is determined as an emergency and is warned. Therefore, the emergency warning device provided by the present application can solve the problem of low accuracy of emergency warning.

[0209] As shown in Figure 3 Fig. 1 is a structural schematic diagram of an electronic device for implementing an emergency warning method according to an embodiment of the present application.

[0210] The electronic device can include a processor 10, a memory 11, a communication interface 12 and a bus 13, and can further include a computer program stored in the memory 11 and executable on the processor 10, such as an emergency warning program.

[0211] The memory 11 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of the electronic device, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 11 can include both an internal storage unit and an external storage device of the electronic device. The memory 11 can be used to store application software installed in the electronic device and various data, such as the code of the emergency alert program, and can also be used to temporarily store data that has been output or will be output.

[0212] The processor 10 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more combinations of a central processing unit (CPU), a microprocessor, a digital processing chip, a graphics processor, and various control chips, etc. The processor 10 is the control unit of the electronic device, and connects various components of the electronic device through various interfaces and lines, and executes various functions and processes data by running or executing programs or modules stored in the memory 11 (such as the emergency alert program, etc.) and calling data stored in the memory 11.

[0213] The communication interface 12 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface can include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used to establish a communication connection between the electronic device and other electronic devices. The user interface can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface can also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch screen, etc. The display can also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visualized user interface.

[0214] The bus 13 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 13 can be divided into an address bus, a data bus, a control bus, etc. The bus 13 is configured to enable connection and communication between the memory 11, the at least one processor 10, etc.

[0215] Figure 3 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 3 The structure shown does not constitute a limitation on the electronic device, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0216] For example, although not shown, the electronic device can further include a power supply (such as a battery) for powering the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so that the power management device can implement functions such as charge management, discharge management, and power consumption management, etc. The power supply can also include one or more direct current or alternating current power supplies, recharging devices, power supply fault detection circuits, power supply converters or inverters, power supply status indicators, etc. Any components. The electronic device can also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which are not described here.

[0217] Further, the electronic device can further include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), and is typically used to establish a communication connection between the electronic device and other electronic devices.

[0218] Optionally, the electronic device can further include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface can also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch, etc. Among them, the display can also be appropriately called a display screen or a display unit, which is used to display information processed in the electronic device and to display a visualized user interface.

[0219] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by this structure.

[0220] The emergency warning program stored in the memory 11 in the electronic device is a combination of multiple instructions, which, when running in the processor 10, can realize:

[0221] obtaining a monitoring image from video data recorded by multiple monitoring devices based on a preset distributed framework;

[0222] segmenting the monitoring image into multiple image sub-blocks meeting a preset segmentation condition, and extracting motion features, size features and texture features in the image sub-blocks respectively;

[0223] respectively constructing feature probabilities corresponding to the motion features and the size features, and calculating a matching correlation coefficient between the texture features and a preset feature codebook;

[0224] constructing an abnormal behavior detection system according to a first classifier and a second classifier constructed in advance, performing initial abnormal behavior detection on the feature probabilities and the matching correlation coefficient based on the abnormal behavior detection system, and obtaining an initial abnormal behavior detection result;

[0225] obtaining an abnormal behavior image corresponding to the initial abnormal behavior detection result, detecting a behavior human body and human body position information corresponding to the behavior human body in the abnormal behavior image by using a target detection algorithm, inputting the human body position information into a symmetric space transformation network, and obtaining multiple human target generated skeleton graphs;

[0226] extracting human skeleton points in the multiple human target generated skeleton graphs, calculating skeleton transformation characteristic values corresponding to the human skeleton points, performing re-abnormal behavior detection according to the skeleton transformation characteristic values, and obtaining a standard abnormal behavior detection result;

[0227] determining whether an emergency has occurred by comparing the standard abnormal behavior detection result with a pre-constructed emergency warning library, and performing warning.

[0228] Specifically, the specific implementation method of the processor 10 on the above instructions can refer to the description of related steps in the corresponding embodiments, which will not be repeated here. Figure 1

[0229] ​Further, the modules / units integrated in the electronic device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0230] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can realize the following when executed by a processor of an electronic device:

[0231] obtain a monitoring image from video data recorded by a plurality of monitoring devices based on a preset distributed framework;

[0232] segment the monitoring image into a plurality of image sub-blocks meeting a preset segmentation condition, and extract motion features, size features and texture features in the image sub-blocks respectively;

[0233] construct feature probabilities corresponding to the motion features and the size features respectively, and calculate a matching correlation coefficient between the texture features and a preset feature codebook;

[0234] construct an abnormal behavior detection system according to a first classifier and a second classifier constructed in advance, perform initial abnormal behavior detection on the feature probabilities and the matching correlation coefficient based on the abnormal behavior detection system, and obtain an initial abnormal behavior detection result;

[0235] obtain an abnormal behavior image corresponding to the initial abnormal behavior detection result, detect a behavior human body in the abnormal behavior image and human body position information corresponding to the behavior human body by using a target detection algorithm, input the human body position information into a symmetric space transformation network, and obtain a plurality of human body target generated skeleton graphs;

[0236] extract human body skeleton points in the plurality of human body target generated skeleton graphs, calculate skeleton transformation characteristic values corresponding to the human body skeleton points, perform re-abnormal behavior detection according to the skeleton transformation characteristic values, and obtain a standard abnormal behavior detection result;

[0237] determine whether an emergency event occurs by comparing the standard abnormal behavior detection result with a pre-constructed emergency event early warning library, and perform early warning.

[0238] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other manners. For example, the described apparatus embodiments are merely illustrative. For example, the division of the modules is merely logical function division. In actual implementation, other division manners can be used.

[0239] The modules illustrated as separated components can or can not be physically separated, and the components illustrated as modules can or can not be physical units. They can be located in one place, or can be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purposes of the embodiments.

[0240] In addition, each function module in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of hardware plus software function modules.

[0241] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0242] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be considered as limiting the claims involved.

[0243] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain, in essence, is a decentralized database, a series of data blocks associated using cryptography. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-fake) and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0244] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. The second word is used to indicate the name, and does not indicate any specific order.

[0245] Finally, it should be noted that the above examples are merely intended to illustrate the technical solutions of the present application and not to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method for early warning of an emergency, characterized in that: The method comprises: Obtain surveillance images from video data recorded by multiple surveillance devices based on a preset distributed framework; Segmenting the surveillance image into a plurality of image sub-blocks that meet preset segmentation conditions, and extracting motion features, size features, and texture features from the image sub-blocks respectively; Constructing feature probabilities corresponding to the motion feature and the size feature respectively, and calculating a matching correlation coefficient between the texture feature and a preset feature codebook; constructing an abnormal behavior detection system according to the pre-constructed first classifier and the second classifier, and performing initial abnormal behavior detection on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result; Acquire an abnormal behavior image corresponding to the initial abnormal behavior detection result, detect a human body in the abnormal behavior image and human body position information corresponding to the human body using a target detection algorithm, input the human body position information into a symmetric space transformation network, and obtain a skeleton graph of multiple human targets; Extracting a plurality of human skeleton points from the human target generated skeleton graph, calculating skeleton transformation characteristic values ​​corresponding to the human skeleton points, and performing abnormal behavior detection again based on the skeleton transformation characteristic values ​​to obtain a standard abnormal behavior detection result; By comparing the standard abnormal behavior detection results with the pre-built emergency warning library, it is determined whether an emergency has occurred and an early warning is issued.

2. The emergency warning method according to claim 1, wherein: The extracting the motion features from the image sub-blocks includes: Obtaining a foreground portion of the image sub-block, and extracting a horizontal optical flow corresponding to any pixel point in the foreground portion in the direction of the horizontal coordinate axis and a vertical optical flow corresponding to any pixel point in the foreground portion in the direction of the vertical coordinate axis; The horizontal optical flow and the vertical optical flow are substituted into a preset motion feature calculation formula to obtain the motion feature of the image sub-block.

3. The emergency warning method according to claim 2, wherein: The preset motion feature calculation formula is: Wherein, M is the motion feature in the image sub-block, N f is the total number of pixels in the foreground part of the image sub-block, represents the horizontal optical flow corresponding to any pixel point in the foreground part in the direction of the horizontal axis, It represents the longitudinal optical flow corresponding to any pixel point in the foreground part in the direction of the vertical coordinate axis, and n is the nth pixel in the foreground part.

4. The emergency warning method according to claim 1, wherein: The extracting the size feature from the image sub-block includes: Obtaining a preset Gaussian template and a preset first parameter and a second parameter, and calculating initial features of the image sub-block according to the Gaussian template; The initial feature and the foreground proportion calculated according to the first parameter and the second parameter are substituted into a preset size feature calculation formula to obtain the size feature in the image sub-block.

5. The emergency warning method according to claim 1, wherein: The extracting of texture features from the image sub-blocks includes: Acquire a plurality of preset target directions, and extract a plurality of amplitude values ​​in the image sub-block from the plurality of target directions using a two-dimensional filter; The multiple amplitude values ​​are aggregated to obtain the texture features in the image sub-block.

6. The emergency warning method according to claim 1, wherein: The performing initial abnormal behavior detection on the feature probability and the matching correlation coefficient based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result includes: Using the first classifier in the abnormal behavior detection system to perform speed judgment on the feature probability corresponding to the motion feature to obtain a speed judgment result; When the speed judgment result is that the speed is abnormal, the target monitoring image is judged as an abnormal image; When the speed judgment result is that the speed is normal, the second classifier in the abnormal behavior detection system is used to perform feature judgment on the feature probability corresponding to the size feature and the matching correlation coefficient. When the feature judgment fails, the target monitoring image is judged as an abnormal image and used as the initial abnormal behavior detection result.

7. The emergency warning method according to claim 1, wherein: Before determining whether an emergency has occurred by comparing the standard abnormal behavior detection result with a pre-built emergency warning library, the method includes: Identifying historical abnormal behaviors in pre-acquired historical reference images, and analyzing abnormal behavior categories corresponding to the historical abnormal behaviors; Obtaining a corresponding emergency event type based on the abnormal behavior category and the performance analysis corresponding to the abnormal behavior category; Multiple emergency event types are aggregated to obtain an emergency event warning database.

8. An emergency warning device, characterized in that: The device comprises: A feature extraction module is used to obtain surveillance images from video data recorded by multiple surveillance devices based on a preset distributed framework, segment the surveillance images into multiple image sub-blocks that meet preset segmentation conditions, and extract motion features, size features, and texture features from the image sub-blocks respectively; an initial anomaly detection module, configured to construct feature probabilities corresponding to the motion feature and the size feature, respectively, and calculate a matching correlation coefficient between the texture feature and a preset feature codebook; construct an abnormal behavior detection system based on a pre-constructed first classifier and a second classifier; and perform initial abnormal behavior detection on the feature probabilities and the matching correlation coefficients based on the abnormal behavior detection system to obtain an initial abnormal behavior detection result; A standard anomaly detection module is configured to obtain an abnormal behavior image corresponding to the initial abnormal behavior detection result, detect the human body in the abnormal behavior image and the human position information corresponding to the human body using a target detection algorithm, input the human position information into a symmetric space transformation network to obtain a skeleton graph generated by multiple human targets, extract human skeleton points from the skeleton graph generated by the multiple human targets, calculate the skeleton transformation characteristic values ​​corresponding to the human skeleton points, perform abnormal behavior detection again based on the skeleton transformation characteristic values, and obtain a standard abnormal behavior detection result; The event comparison module is used to determine whether an emergency has occurred by comparing the standard abnormal behavior detection results with a pre-built emergency warning library and to issue an early warning.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the emergency warning method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the emergency warning method according to any one of claims 1 to 7 is implemented.

Images