Video alarm aggregation method and system based on machine learning

Abstract

The invention discloses a video alarm aggregation method and system based on machine learning, and relates to the technical field of video monitoring analysis. The method comprises the following steps of: obtaining a video alarm feature vector group of each piece of data according to all historically received video alarm data, clustering by using a clustering algorithm to obtain a plurality of clustering centers, then dividing each piece of data into a corresponding clustering category according to an Euler distance calculation result, and finally, according to the vector group and a category division result, obtaining a video alarm feature vector group of each piece of data. A machine learning algorithm is used for training to obtain an alarm data clustering category division model, the model is used for dividing newly received video alarm data into a certain clustering category with the maximum division probability, and finally, only when the duration from the latest alarm action execution timestamp to the new alarm timestamp exceeds a preset duration threshold value, the new alarm data is divided into the new alarm data. In this way, similar video alarms can be intelligently combined, repeated alarms are effectively reduced, and the utilization efficiency of system resources and the judgment capability of a user on alarm information are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of video surveillance analysis, and in particular relates to a video alarm aggregation method and system based on machine learning. Background Art

[0002] In today's digital age, video AI (Artificial Intelligence) analysis technology has been widely used in industrial safety production supervision scenarios. However, a common problem currently faced is the high false alarm rate. One of the key influencing factors is that the same alarm is frequently triggered, resulting in a large number of duplicate alarms. This phenomenon of duplicate alarms will bring many negative effects: on the one hand, it will seriously affect the efficiency of the alarm system, that is, a large number of duplicate alarm information will occupy system resources, including storage resources, computing resources, and network transmission resources, etc., which may cause the system to experience delays or insufficient resources when processing truly valuable alarm information; on the other hand, from the perspective of user experience, too many duplicate alarms will interfere with the user's judgment of the alarm information, making it difficult for users to quickly and accurately identify the alarms that really need attention and processing, and may even cause users to distrust the entire alarm system, reducing the practicality of the system.

[0003] Therefore, how to provide a video alarm aggregation solution to intelligently merge similar video alarms, thereby effectively reducing duplicate alarms, improving system resource utilization efficiency and user judgment ability on alarm information, and thereby improving the overall system response speed and user satisfaction, is a topic that technical personnel in this field urgently need to study. Summary of the invention

[0004] The purpose of the present invention is to provide a video alarm aggregation method and system based on machine learning, so as to solve the problem that the existing video alarm scheme has system performance limitation and affects the user experience due to repeated alarms.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: In a first aspect, a video alarm aggregation method based on machine learning is provided, comprising: Acquire all video alarm data received historically, wherein the video alarm data contains an alarm timestamp, an alarm type, an alarm location, and an alarm video clip; For each piece of video alarm data in all the video alarm data, feature extraction processing is performed on the alarm type, alarm position and alarm video segment in the corresponding data to obtain a corresponding video alarm feature vector group including an alarm type feature vector, an alarm position feature vector and an alarm video feature vector; According to the video alarm feature vector group of each piece of video alarm data, clustering is performed using a clustering algorithm to obtain a plurality of cluster centers; For each of the video alarm data, according to the multiple cluster centers and the corresponding video alarm feature vector groups, the corresponding Euler distances to the multiple cluster centers are calculated respectively, and then a cluster center corresponding to the shortest Euler distance is determined from the multiple cluster centers, and the corresponding data is divided into a cluster category corresponding to the cluster center; The video alarm feature vector group of each video alarm data is used as a model input item, and the clustering category division result of each video alarm data is used as a model output item, and an artificial intelligence model based on a machine learning algorithm is imported for model training to obtain an alarm data clustering category division model; Performing the feature extraction process on the alarm type, alarm position and alarm video segment in the newly received video alarm data respectively to obtain a new video alarm feature vector group; Importing the new video alarm feature vector group into the alarm data clustering category classification model to obtain the probability of classifying the newly received video alarm data into each cluster category; If it is found that the probability of classifying the newly received video alarm data into a certain cluster category exceeds a preset probability threshold, the newly received video alarm data is classified into the certain cluster category; Only when the duration from the most recent execution timestamp of the alarm action corresponding to the certain cluster category to the alarm timestamp in the newly received video alarm data exceeds a preset duration threshold, an alarm action corresponding to the certain cluster category is executed once according to the newly received video alarm data.

[0006] Based on the above invention content, a new scheme for video alarm aggregation based on clustering algorithm and machine learning algorithm is provided, that is, first, according to all the video alarm data received in history, a video alarm feature vector group containing an alarm type feature vector, an alarm position feature vector and an alarm video feature vector is obtained for each piece of data, and a clustering algorithm is applied to cluster to obtain multiple cluster centers, and then each piece of data is divided into a corresponding cluster category according to the Euler distance calculation result, and then according to the vector group and the category division result, a machine learning algorithm is applied to train to obtain an alarm data cluster category division model, and the model is applied to divide the newly received video alarm data into a cluster category with a maximum division probability, and finally, the alarm action is executed only when the time from the most recent execution timestamp of the alarm action to the new alarm timestamp exceeds a preset time threshold, so that similar video alarms can be intelligently merged, duplicate alarms can be effectively reduced, and the system resource utilization efficiency and the user's judgment ability on alarm information can be improved, thereby improving the overall response speed of the system and the user's satisfaction, which is convenient for practical application and promotion.

[0007] In a possible design, for each piece of video alarm data in all the video alarm data, feature extraction processing is performed on the alarm type, alarm position and alarm video segment in the corresponding data, respectively, to obtain a corresponding video alarm feature vector group including an alarm type feature vector, an alarm position feature vector and an alarm video feature vector, including: For a certain video alarm data among all the video alarm data, a single hot encoding method and a layered encoding method are used to encode the alarm type in the corresponding data to obtain a corresponding alarm type feature vector; Normalizing the alarm position in the certain video alarm data and being the coordinate information in the video screen to obtain normalized coordinates, and then combining the normalized coordinates with the alarm position in the certain video alarm data and being the monitoring area number and / or the geometric shape field of view partition number in the monitoring area to form an alarm position feature vector of the certain video alarm data; Importing the alarm video clip in the certain video alarm data into the pre-trained convolutional neural network model, and then using the result output by the intermediate layer in the convolutional neural network model as the alarm video feature vector of the certain video alarm data; The alarm type feature vector, the alarm position feature vector and the alarm video feature vector of the certain video alarm data are combined into a video alarm feature vector group of the certain video alarm data.

[0008] In a possible design, the result output by the intermediate layer in the convolutional neural network model is used as the alarm video feature vector of the certain video alarm data, including: Using the result output by the intermediate layer in the convolutional neural network model as a video segment feature vector; Performing PCA dimensionality reduction processing on the feature vector of the video clip to obtain a feature vector after dimensionality reduction; The feature vector after dimension reduction is used as the alarm video feature vector of the certain video alarm data.

[0009] In a possible design, a clustering algorithm is applied to obtain a plurality of cluster centers based on the video alarm feature vector groups of the respective video alarm data, including: For each of the video alarm data, according to the corresponding video alarm feature vector group and the video alarm feature vector group of each other video alarm data, calculate the corresponding Euler distance with each of the other video alarm data, and count the number of distances whose corresponding Euler distance is less than a preset distance threshold; According to the number of distances of each piece of video alarm data and the distribution position of each piece of video alarm data in the multidimensional space, a plurality of pieces of video alarm data having a maximum number of distances among all the video alarm data are determined, wherein each dimension in the multidimensional space corresponds one-to-one to each dimensional feature value in the video alarm feature vector group; Arrange the plurality of video alarm data in descending order according to the maximum values ​​of the number of distances to obtain a video alarm data sequence; Select the previous one from the video alarm data sequence A copy of the video alarm data, and the The video alarm feature vector groups of the video alarm data are used one by one as Initial cluster centers, where Indicates the preset number of clustering categories; Based on the Initial cluster centers are selected, and multiple cluster centers are obtained by clustering using the K-Means clustering algorithm and other steps after the initial cluster center selection step.

[0010] In a possible design, after obtaining the probability of classifying the newly received video alarm data into each cluster category, the method further includes: If it is found that the probability of classifying the newly received video alarm data into each cluster category does not exceed the preset probability threshold, then according to the probability of classifying the newly received video alarm data into each cluster category, determine a first cluster category corresponding to the highest probability and a second cluster category corresponding to the second highest probability, wherein the second highest probability refers to a probability that is only less than the highest probability; Determine whether the sum of the highest probability and the second highest probability exceeds the preset probability threshold and whether the difference between the highest probability and the second highest probability is less than a preset difference threshold; If both are true, merging the first cluster category and the second cluster category, and classifying the newly received video alarm data into the merged cluster category; Only when the duration from the most recent execution timestamp of the alarm action corresponding to the merged cluster category to the alarm timestamp in the newly received video alarm data exceeds the preset duration threshold, the alarm action corresponding to the merged cluster category is executed once according to the newly received video alarm data.

[0011] In one possible design, after determining whether the sum of the highest probability and the second highest probability exceeds the preset probability threshold and whether the difference between the highest probability and the second highest probability is less than a preset difference threshold, the method further includes: If it is determined that the sum of the highest probability and the second highest probability does not exceed the preset probability threshold and / or the difference between the highest probability and the second highest probability is greater than or equal to the preset difference threshold, a new cluster category is created, and the newly received video alarm data is classified into the new cluster category; An alarm action corresponding to the new clustering category is executed according to the newly received video alarm data.

[0012] In one possible design, the method further includes: Periodically counting the total number of the video alarm data classified into the corresponding category in the most recent period for each currently existing cluster category; If it is found that the total number of the video alarm data classified into any cluster category in the most recent cycle is less than the preset number threshold, then for each other cluster category currently existing, based on the video alarm feature vector group of all the video alarm data classified into the corresponding category in the most recent cycle and the video alarm feature vector group of all the video alarm data classified into any cluster category in the most recent cycle, calculate the similarity between the corresponding category and any cluster category; According to the similarities between the other cluster categories and the any cluster category, determining another cluster category corresponding to the maximum similarity; The other cluster category is merged with any cluster category to obtain a merged cluster category.

[0013] In one possible design, after the clustering category is changed, the method further includes: The video alarm feature vector group of all the video alarm data classified into the currently existing cluster categories in the most recent cycle is used as a model input item, and the cluster category classification result of all the video alarm data classified into the currently existing cluster categories in the most recent cycle is used as a model output item, and the artificial intelligence model is imported to re-train the model to obtain a new alarm data cluster category classification model.

[0014] In a second aspect, a video alarm aggregation system based on machine learning is provided, comprising an alarm data acquisition unit, a feature vector extraction unit, a clustering algorithm application unit, a data category division unit, a division model training unit, a division model application unit, a new data category division unit and an alarm action execution unit; The alarm data acquisition unit is used to acquire all video alarm data received in the past, wherein the video alarm data contains an alarm timestamp, an alarm type, an alarm location, and an alarm video clip; The feature vector extraction unit is communicatively connected to the alarm data acquisition unit, and is used to perform feature extraction processing on the alarm type, alarm position and alarm video segment in the corresponding data for each piece of video alarm data in all the video alarm data, so as to obtain a corresponding video alarm feature vector group including the alarm type feature vector, the alarm position feature vector and the alarm video feature vector; The clustering algorithm application unit is communicatively connected to the feature vector extraction unit, and is used to apply a clustering algorithm to cluster the video alarm feature vector groups of the respective video alarm data to obtain a plurality of cluster centers; The data category division unit is respectively connected to the feature vector extraction unit and the clustering algorithm application unit in communication, and is used to calculate the corresponding Euler distances to the multiple cluster centers according to the multiple cluster centers and the corresponding video alarm feature vector groups for each piece of video alarm data, and then determine a cluster center corresponding to the shortest Euler distance from the multiple cluster centers, and divide the corresponding data into a cluster category corresponding to the cluster center; The partition model training unit is respectively connected to the feature vector extraction unit and the data category partition unit for communication, and is used to use the video alarm feature vector group of each video alarm data as a model input item, and use the cluster category partition result of each video alarm data as a model output item, import the artificial intelligence model based on the machine learning algorithm for model training, and obtain the alarm data cluster category partition model; The feature vector extraction unit is further used to perform the feature extraction processing on the alarm type, alarm position and alarm video segment in the newly received video alarm data respectively to obtain a new video alarm feature vector group; The partition model application unit is respectively connected to the feature vector extraction unit and the partition model training unit for communication, and is used to import the new video alarm feature vector group into the alarm data cluster category partition model to obtain the probability of partitioning the newly received video alarm data into each cluster category; The new data category classification unit is communicatively connected to the classification model application unit, and is used to classify the newly received video alarm data into a certain cluster category if it is found that the probability of classifying the newly received video alarm data into the certain cluster category exceeds a preset probability threshold; The alarm action execution unit is communicatively connected to the new data category division unit, and is used to execute an alarm action corresponding to the certain cluster category according to the newly received video alarm data only when the duration from the most recent execution timestamp of the alarm action corresponding to the certain cluster category to the alarm timestamp in the newly received video alarm data exceeds a preset duration threshold.

[0015] In a third aspect, the present invention provides a computer system comprising a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the video alarm aggregation method as described in the first aspect or any possible design of the first aspect.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the video alarm aggregation method as described in the first aspect or any possible design of the first aspect is executed.

[0017] In a fifth aspect, the present invention provides a computer program product, comprising a computer program or instructions, which, when executed by a computer, implements the video alarm aggregation method as described in the first aspect or any possible design of the first aspect.

[0018] Beneficial effects of the above scheme: (1) The present invention creatively provides a new scheme for video alarm aggregation based on clustering algorithm and machine learning algorithm, that is, firstly, based on all the video alarm data received in the past, a video alarm feature vector group containing an alarm type feature vector, an alarm position feature vector and an alarm video feature vector is obtained for each piece of data, and a clustering algorithm is applied to cluster to obtain multiple cluster centers, and then each piece of data is divided into corresponding cluster categories according to the Euler distance calculation result, and then, based on the vector group and the category division result, a machine learning algorithm is applied to train to obtain an alarm data clustering category division model, and the model is applied to divide the newly received video alarm data into a cluster category with the maximum division probability, and finally, the alarm action is executed only when the time from the most recent execution timestamp of the alarm action to the new alarm timestamp exceeds a preset time threshold, so that similar video alarms can be intelligently merged, effectively reducing duplicate alarms, improving the efficiency of system resource utilization and the user's ability to judge alarm information, and thus improving the overall response speed of the system and user satisfaction; (2) Cluster categories can also be dynamically adjusted, such as merging or creating new ones, based on the latest actual division situation, so that the entire solution can continuously adapt to new alarm patterns; (3) By timely updating the partitioning model after the clustering categories are dynamically adjusted, the sustainability of the entire video alarm aggregation can be ensured; (4) It can solve the problem of repeated alarms caused by frequent triggering of similar alarms in video analysis. It is particularly suitable for industrial safety production supervision scenarios and is easy to apply and promote in practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A flowchart of a video alarm aggregation method based on machine learning provided in an embodiment of the present application.

[0021] Figure 2 A schematic diagram of a process for dynamically adjusting the first clustering category in a video alarm aggregation method provided in an embodiment of the present application.

[0022] Figure 3 A schematic diagram of a process for dynamically adjusting the second clustering category in a video alarm aggregation method provided in an embodiment of the present application.

[0023] Figure 4A schematic diagram of the structure of a video alarm aggregation system based on machine learning provided in an embodiment of the present application.

[0024] Figure 5 A schematic diagram of the structure of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structures of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these embodiments without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0026] It should be understood that although the terms first and second, etc. may be used herein to describe various objects, these objects should not be limited by these terms. These terms are only used to distinguish one object from another object. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object without departing from the scope of the exemplary embodiments of the present invention.

[0027] It should be understood that the term "and / or" that may appear in this document is merely a description of the association relationship between associated objects, indicating that there may be three relationships. For example, A and / or B can indicate three situations: A exists alone, B exists alone, or A and B exist at the same time. For another example, A, B and / or C can indicate the existence of any one of A, B and C or any combination of them. The term " / and" that may appear in this document describes another type of association object relationship, indicating that there may be two relationships. For example, A / and B can indicate two situations: A exists alone or A and B exist at the same time. In addition, the character " / " that may appear in this document generally indicates that the previous and next associated objects are in an "or" relationship.

[0028] Example like Figure 1As shown, the video alarm aggregation method based on machine learning provided in the first aspect of this embodiment can be executed by, but is not limited to, a computer system with certain computing resources, such as an edge device, cloud server, personal computer (PC, a multi-purpose computer with a size, price and performance suitable for personal use; desktops, laptops, small laptops, tablets and ultrabooks are all personal computers), smart phones, personal digital assistants (PDA) or wearable devices and other electronic devices that are connected to a video AI analysis device (such as an AI camera) with video alarm phenomenon recognition capabilities. Figure 1 As shown, the video alarm aggregation method may include but is not limited to the following steps S1 to S9.

[0029] S1. Obtain all video alarm data received historically, wherein the video alarm data includes but is not limited to alarm timestamp, alarm type, alarm location, and alarm video clip, etc.

[0030] In step S1, the video alarm data is the output result of the existing video AI analysis equipment and can be obtained conventionally. In order to ensure the data quality of subsequent processing, it is also necessary to perform conventional data cleaning processing on all the video alarm data to remove abnormal data or erroneous data, for example, check whether the alarm timestamp is within a reasonable range; check whether the alarm type meets the predefined type specifications, etc.; data that does not meet the requirements can be corrected or deleted to ensure the data quality of subsequent processing. In addition, in order to ensure the timeliness of subsequent clustering results, all video alarm data historically received in at least one recent unit period can be obtained preferentially, where the aforementioned unit period can be but not limited to days, weeks or months, etc.

[0031] S2. For each piece of video alarm data among all the video alarm data, feature extraction processing is performed on the alarm type, alarm position and alarm video segment in the corresponding data respectively, and a corresponding video alarm feature vector group is obtained which includes an alarm type feature vector, an alarm position feature vector and an alarm video feature vector.

[0032] In the step S2, the video alarm feature vector group is used as a feature extraction result based on the video alarm data. Specifically, for each video alarm data in all the video alarm data, feature extraction processing is performed on the alarm type, alarm position and alarm video segment in the corresponding data to obtain a corresponding video alarm feature vector group containing an alarm type feature vector, an alarm position feature vector and an alarm video feature vector, including but not limited to the following steps S21 to S24.

[0033] S21. For a certain video alarm data among all the video alarm data, the alarm type in the corresponding data is encoded using a one-hot encoding method / and a layered encoding method (that is, a one-hot encoding method, or a one-hot encoding method and a layered encoding method) to obtain a corresponding alarm type feature vector.

[0034] In the step S21, all alarm types in all the video alarm data may be comprehensively sorted out and classified. For example, in a security monitoring scenario, the alarm types may include human intrusion alarm types, object movement alarm types, and abnormal behavior alarm types, etc.; then, one-hot encoding (One-Hot Encoding, which is an existing method of converting categorical variables into numerical variables that can be understood by a machine learning model) is used to encode different alarm types respectively: assuming that there are n different alarm types, a vector of length n is created, and for each alarm type, the value of the corresponding vector position is configured to be 1, and the remaining positions are 0; for example, if the alarm type is a human intrusion alarm type, and the vector position corresponding to the human intrusion alarm type is the third in sequence, then the alarm type feature vector obtained after encoding the human intrusion alarm type is [0,0,1,0,…,0]. In addition, for some alarm types with a hierarchical structure, they can be encoded first using a hierarchical encoding method: for example, the abnormal behavior alarm type can be further subdivided into running alarm subtypes and fighting alarm subtypes, etc. In this case, a multi-layer encoding structure can be used to first encode the high-level abnormal behavior alarm type (specifically, it can be but not limited to using a one-hot encoding method), and then additionally encode the specific alarm subtypes at the next layer (specifically, it can be but not limited to using a one-hot encoding method).

[0035] S22. Normalize the alarm position in the certain video alarm data which is the coordinate information in the video screen to obtain normalized coordinates, and then combine the normalized coordinates with the alarm position in the certain video alarm data which is the monitoring area number and / or the geometric shape field of view partition number in the monitoring area (that is, the monitoring area number, or the geometric shape field of view partition number in the monitoring area, or the monitoring area number and the geometric shape field of view partition number in the monitoring area) to form an alarm position feature vector of the certain video alarm data.

[0036] In step S22, since the alarm position includes the coordinate information in the video screen and the monitoring area number / and the geometric shape field of view partition number in the monitoring area, etc., it is necessary to first normalize the coordinate information in the video screen, and then combine the processing result with the monitoring area number and / or the geometric shape field of view partition number in the monitoring area to form the alarm position feature vector. Specifically, assuming that the coordinate information in the video screen is a two-dimensional coordinate , the corresponding normalized interval can be calculated according to the following formula Normalized coordinates within :

[0037] In the formula, Indicates the screen width. Indicates the screen height; assuming that the monitoring area is divided into m different areas, each area has a unique number, then a vector of length m+2 can be created, the first two positions are normalized coordinates, and the following m positions correspond to the m different areas one by one, and the corresponding monitoring area position is configured with a value of 1, and the remaining positions are configured with a value of 0 (that is, the monitoring area number is similar to the one-hot encoding method), so as to obtain the alarm position feature vector. In addition, for some irregular specific monitoring areas, shape-based feature representation can also be used: for example, the monitoring area is divided into a plurality of small geometric shapes (such as triangles or rectangles, etc.) of visual field partitions, and a binary vector representation is generated by judging which geometric shape visual field partitions the alarm position falls within, that is, each geometric shape visual field partition corresponds to a position in the vector, and if the alarm position is within the shape visual field partition, the corresponding position is configured with a value of 1, and the remaining positions are 0 (that is, the geometric shape visual field partition number in the monitoring area is also similar to the one-hot encoding method).

[0038] S23. Import the alarm video clip in the certain video alarm data into the pre-trained convolutional neural network model, and then use the result output by the middle layer in the convolutional neural network model as the alarm video feature vector of the certain video alarm data.

[0039] In the step S23, the Convolutional Neural Network (CNN) model is a deep learning model, which is mainly used for classification and recognition tasks of signal data such as images, videos or voices. It can automatically extract local features of data and combine complex features by simulating the visual perception mechanism of organisms. Therefore, the warning video feature vector can be conventionally extracted through the intermediate layer in the convolutional neural network model (for example, the penultimate layer, whose output is usually a high-dimensional feature vector so as to capture high-level information in the video clip such as texture, object shape, motion pattern, etc.). In order to further compress the feature dimension, a dimensionality reduction technique such as principal components analysis (PCA) can also be used to reduce the dimension of the result output by the intermediate layer in the convolutional neural network model, that is, further preferably, the result output by the intermediate layer in the convolutional neural network model is used as the alarm video feature vector of the certain video alarm data, including but not limited to the following steps: firstly, the result output by the intermediate layer in the convolutional neural network model is used as the video segment feature vector; then, the video segment feature vector is subjected to PCA dimensionality reduction processing to obtain the reduced dimension feature vector; finally, the reduced dimension feature vector is used as the alarm video feature vector of the certain video alarm data. For example, the original video segment feature vector is reduced from 1000 dimensions to 200 dimensions, which can not only reduce the amount of calculation and storage space, but also maintain the representativeness of the features. In addition, the convolutional neural network model can specifically include but is not limited to the existing VGG model (Visual Geometry Group, a deep convolutional neural network proposed by the Visual Geometry Group of the University of Oxford, mainly used for image classification and recognition tasks) or Resnet model (residual network, a deep learning model proposed by He Kaiming, Zhang Xiangyu, Ren Shaoqing and Sun Jian in 2015 to solve the degradation problem in deep neural networks), etc.

[0040] S24. Combine the alarm type feature vector, the alarm position feature vector and the alarm video feature vector of the certain video alarm data into a video alarm feature vector group of the certain video alarm data.

[0041] S3. Apply a clustering algorithm to cluster the video alarm feature vector groups of the respective video alarm data to obtain a plurality of cluster centers.

[0042] In step S3, the clustering algorithm is an unsupervised learning method, which aims to divide the data set into several groups (clusters) according to the similarity or distance between data points. Specifically, the clustering algorithm can be, but is not limited to, K-Means clustering or DBSCAN (Density-Based Spatial Clustering of Applications with Noise, a more representative density-based clustering algorithm) clustering, etc., wherein the K-Means clustering algorithm is an iterative clustering analysis algorithm, and its steps are generally: pre-dividing the data into K groups, and randomly selecting K objects as initial clustering centers, and then calculating the distance between each object and each seed clustering center, assigning each object to the clustering center closest to it, and continuously updating the iterative clustering center until the error sum of squares (i.e., the criterion function) converges to obtain the clustering result. Considering that the existing K-Means clustering algorithm may change the compactness and discreteness of the clusters due to the influence of extreme values ​​during the clustering process, thereby reducing the accuracy of the entire clustering result, preferably, a clustering algorithm is applied to obtain multiple clustering centers based on the video alarm feature vector group of each video alarm data, including but not limited to the following steps S31 to S35.

[0043] S31. For each of the video alarm data, based on the corresponding video alarm feature vector group and the video alarm feature vector group of each other video alarm data, calculate the corresponding Euler distance with each of the other video alarm data, and count the number of distances whose corresponding Euler distance is less than a preset distance threshold.

[0044] In step S31, the specific calculation formula of the Euler distance is an existing formula. For example, for a certain video alarm data, if there are 1023 other video alarm data, 1023 Euler distances will be calculated, and if 578 of the 1023 Euler distances are less than the preset distance threshold, then the number of distances of the certain video alarm data and the Euler distance less than the preset distance threshold is 578. The aforementioned number of distances positively reflects the total number of other video alarm data located around the certain video alarm data in the multidimensional space, that is, it positively reflects the size of the neighbor density corresponding to the certain video alarm data.

[0045] S32. According to the number of distances of each piece of video alarm data and the distribution position of each piece of video alarm data in the multi-dimensional space, determine multiple pieces of video alarm data with maximum number of distances among all the video alarm data, wherein each dimension in the multi-dimensional space corresponds one-to-one to each dimensional feature value in the video alarm feature vector group.

[0046] In the step S32, the number of distances of each of the multiple video alarm data is greater than the number of distances of adjacent video alarm data in each dimension, which can be similar to the method of finding extreme values ​​through partial derivatives of multivariate functions. According to the number of distances of each of the video alarm data and the distribution position of each of the video alarm data in the multidimensional space, the multiple video alarm data with the maximum number of distances are determined.

[0047] S33. Arrange the multiple video alarm data in order from large to small according to the maximum value of the number of distances to obtain a video alarm data sequence.

[0048] S34. Select the previous one from the video alarm data sequence A copy of the video alarm data, and the The video alarm feature vector groups of the video alarm data are used one by one as Initial cluster centers, where Indicates the preset number of cluster categories.

[0049] S35. Based on the Initial cluster centers are selected, and multiple cluster centers are obtained by clustering using the K-Means clustering algorithm and other steps after the initial cluster center selection step.

[0050] Therefore, based on the above steps S31 to S35, the sample distribution information can be used to select multiple points with the highest density as the initial clustering centers, so as to effectively solve the problem of clustering falling into local optimality due to human interference or extreme value influence, so that the final clustering result can meet the conditions that the similarity of the same cluster is the highest and the similarity of different clusters is the lowest, thereby ensuring the stability of the clustering result.

[0051] S4. For each of the video alarm data, based on the multiple cluster centers and the corresponding video alarm feature vector groups, the corresponding Euler distances to the multiple cluster centers are calculated respectively, and then a cluster center corresponding to the shortest Euler distance is determined from the multiple cluster centers, and the corresponding data is divided into a cluster category corresponding to the cluster center.

[0052] In step S4, each cluster center corresponds to a cluster category. For example, if the video alarm feature vector group of the plurality of cluster centers and a certain video alarm data is respectively calculated and the Euler distances to the plurality of cluster centers are: the Euler distance to cluster center A is 81 unit distances, the Euler distance to cluster center B is 99 unit distances; the Euler distance to cluster center C is 127 unit distances, the Euler distance to cluster center D is 56 unit distances, and the Euler distance to cluster center E is 79 unit distances, then the certain video alarm data can be classified into the cluster category corresponding to cluster center D.

[0053] S5. Use the video alarm feature vector group of each video alarm data as a model input item, and use the clustering category division result of each video alarm data as a model output item, import it into an artificial intelligence model based on a machine learning algorithm for model training, and obtain an alarm data clustering category division model.

[0054] In step S5, the machine learning algorithm is a core algorithm of artificial intelligence that specifically studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structures to continuously improve their own performance, and is the fundamental way to make computers intelligent; specifically, the machine learning algorithm preferably uses a linear regression algorithm based on the Python sklearn library to quickly and accurately find the rules in the data. Thus, the video alarm feature vector group of the video alarm data can be used as sample data, and the clustering category division result of the video alarm data (which is specifically the label value of the corresponding clustering category) can be used as label data, and through a conventional calibration verification modeling process (the specific process includes a calibration process and a verification process of the model, that is, first by comparing the model simulation results with the measured data, and then adjusting the model parameters according to the comparison results, so that the simulation results are consistent with the actual process), the alarm data clustering category division model is trained. Preferably, in the calibration verification modeling process of the alarm data clustering category division model, the model parameters are tuned using a tree-structured Bayesian optimization algorithm. In addition, the machine learning algorithm may also be, but is not limited to, an artificial intelligence algorithm based on support vector machine, K nearest neighbor method, stochastic gradient descent method, multivariate linear regression, multilayer perceptron, decision tree, back propagation neural network or radial basis function network.

[0055] S6. Perform the feature extraction process on the alarm type, alarm position and alarm video segment in the newly received video alarm data respectively to obtain a new video alarm feature vector group.

[0056] In step S6, the specific feature extraction process refers to the aforementioned step S2, which will not be repeated here.

[0057] S7. Import the new video alarm feature vector group into the alarm data clustering category classification model to obtain the probability of classifying the newly received video alarm data into each cluster category.

[0058] In step S6, the probability is the confidence level of classifying the newly received video alarm data into each cluster category.

[0059] S8. If it is found that the probability of classifying the newly received video alarm data into a certain cluster category exceeds a preset probability threshold, the newly received video alarm data is classified into the certain cluster category.

[0060] In step S8, the preset probability threshold may be pre-set based on historical experience, for example, set to 70%.

[0061] S9. Only when the duration from the most recent execution timestamp of the alarm action corresponding to the certain clustering category to the alarm timestamp in the newly received video alarm data exceeds a preset duration threshold, execute the alarm action corresponding to the certain clustering category according to the newly received video alarm data.

[0062] In step S9, it is considered that there may be a situation where the previously received video alarm data (i.e., its alarm timestamp is earlier than the alarm timestamp in the newly received video alarm data) is classified into the certain cluster category. Therefore, in order to avoid frequent execution of similar alarm actions, it is necessary to execute an alarm action corresponding to the certain cluster category according to the newly received video alarm data only when the difference between the two timestamps exceeds the preset time threshold. The preset time threshold can also be pre-set based on historical experience, for example, it is designed to be 5 minutes. The specific execution method of the alarm action is the existing conventional method, such as using an audible and visual alarm method. Considering that when the duration from the most recent execution timestamp of the alarm action corresponding to the certain clustering category to the alarm timestamp in the newly received video alarm data does not exceed the preset duration threshold, although the alarm action corresponding to the certain clustering category will not be executed once according to the newly received video alarm data, the alarm evidence corresponding to the most recently executed alarm action can be updated / enriched according to the newly received video alarm data, that is, preferably, when the duration from the most recent execution timestamp of the alarm action corresponding to the certain clustering category to the alarm timestamp in the newly received video alarm data does not exceed the preset duration threshold, the method also includes but is not limited to the following steps S901 to S904.

[0063] S901. Calculate the video similarity between the alarm video segment and the existing video segment corresponding to the most recently executed alarm action based on the alarm video segment in the newly received video alarm data.

[0064] In step S901, the specific calculation method of the video similarity may adopt a content-based video similarity algorithm, for example, calculating the histogram similarity of key frames of two video clips and using it as the video similarity.

[0065] S902. Determine whether the video similarity exceeds a preset similarity threshold.

[0066] In the step S902, the preset similarity threshold may also be preset based on historical experience, for example, set to 80%.

[0067] S903. If it is determined that the video similarity exceeds the preset similarity threshold, the video quality index values ​​of the alarm video segment in the newly received video alarm data and the existing video segment are calculated respectively, and the video segment corresponding to the highest video quality index value is selected from the alarm video segment and the existing video segment as the existing video segment corresponding to the most recently executed alarm action.

[0068] In step S903, the specific calculation method of the video quality index value is also the existing conventional method, for example, calculating the video definition as the video quality index value. In addition, the unselected video segments will be deleted in order to save data storage space.

[0069] S904. If it is determined that the video similarity does not exceed the preset similarity threshold, the alarm video segment in the newly received video alarm data and the existing video segment are spliced ​​into a new video segment, and the new video segment is used as the existing video segment corresponding to the most recently executed alarm action.

[0070] In step S9, while executing an alarm action corresponding to the certain cluster category based on the newly received video alarm data, the trigger number corresponding to the certain cluster category can be incremented by 1, or the trigger number corresponding to the certain cluster category and the most recently executed alarm action can be incremented by 1 (this trigger number will be initialized to 1 when the most recently executed alarm action is performed).

[0071] Therefore, based on the video alarm aggregation method described in the aforementioned steps S1 to S9, a new scheme for video alarm aggregation based on a clustering algorithm and a machine learning algorithm is provided, that is, first, according to all the video alarm data received historically, a video alarm feature vector group containing an alarm type feature vector, an alarm position feature vector and an alarm video feature vector is obtained for each piece of data, and a clustering algorithm is applied to cluster to obtain multiple cluster centers, and then each piece of data is divided into a corresponding cluster category according to the Euler distance calculation result, and then according to the vector group and the category division result, a machine learning algorithm is applied to train to obtain an alarm data cluster category division model, and the model is applied to divide the newly received video alarm data into a cluster category with a maximum division probability, and finally, the alarm action is executed only when the time from the most recent execution timestamp of the alarm action to the new alarm timestamp exceeds a preset time threshold, so that similar video alarms can be intelligently merged, duplicate alarms can be effectively reduced, the system resource utilization efficiency and the user's judgment ability on alarm information can be improved, and the overall response speed of the system and user satisfaction can be improved, which is convenient for practical application and promotion.

[0072] Based on the technical solution of the first aspect, this embodiment further provides a possible design of how to dynamically adjust the clustering category in the video alarm aggregation method, that is, Figure 2 As shown, after obtaining the probability of classifying the newly received video alarm data into each cluster category, the method further includes but is not limited to the following steps S701 to S704.

[0073] S701. If it is found that the probability of classifying the newly received video alarm data into each cluster category does not exceed the preset probability threshold, then according to the probability of classifying the newly received video alarm data into each cluster category, determine the first cluster category corresponding to the highest probability and the second cluster category corresponding to the second highest probability, wherein the second highest probability refers to a probability that is only smaller than the highest probability.

[0074] In step S701, for example, if there are four cluster categories: cluster category A, cluster category B, cluster category C and cluster category D, and the probability of classifying the newly received video alarm data into cluster category A is 20%, the probability of classifying the newly received video alarm data into cluster category B is 50%, the probability of classifying the newly received video alarm data into cluster category C is 10%, and the probability of classifying the newly received video alarm data into cluster category D is 40%, then the highest probability is 50%, the first cluster category is cluster category B, the second highest probability is 40%, and the second cluster category is cluster category D.

[0075] S702. Determine whether the sum of the highest probability and the second highest probability exceeds the preset probability threshold and whether the difference between the highest probability and the second highest probability is less than a preset difference threshold.

[0076] In the step S702, the preset phase difference threshold may also be preset based on historical experience, for example, set to 12%.

[0077] S703. If both are true, merge the first cluster category and the second cluster category, and classify the newly received video alarm data into the merged cluster category.

[0078] In step S703, based on the example of step S701, cluster category B and cluster category D may be merged to obtain a merged cluster category. In addition, when performing the merging process, it is also necessary to routinely reconfirm the corresponding most recently executed alarm action, and reconfirm the existing video clip corresponding to the most recently executed alarm action, and so on.

[0079] After step S702, specifically, after determining whether the sum of the highest probability and the second highest probability exceeds the preset probability threshold and whether the difference between the highest probability and the second highest probability is less than the preset difference threshold, the method further includes but is not limited to the following steps: if it is determined that the sum of the highest probability and the second highest probability does not exceed the preset probability threshold and / or the difference between the highest probability and the second highest probability is greater than or equal to the preset difference threshold, then create a new cluster category and classify the newly received video alarm data into the new cluster category; perform an alarm action corresponding to the new cluster category according to the newly received video alarm data. In addition, the alarm video segment in the newly received video alarm data can also be used as an existing video segment corresponding to the most recently executed alarm action of the new cluster category.

[0080] S704. Only when the duration from the most recent execution timestamp of the alarm action corresponding to the merged cluster category to the alarm timestamp in the newly received video alarm data exceeds the preset duration threshold, execute the alarm action corresponding to the merged cluster category according to the newly received video alarm data.

[0081] In the step S704, the specific details can be found in the aforementioned step S9, which will not be described in detail here.

[0082] Therefore, based on the above-mentioned possible design one, the cluster categories can also be dynamically adjusted such as merging or creating new ones according to the latest actual division situation, so that the entire solution can continuously adapt to new alarm modes.

[0083] Based on the technical solution of the first aspect or possible design one, this embodiment further provides another possible design two for dynamically adjusting clustering categories in the video alarm aggregation method, that is, Figure 3 As shown, the method also includes but is not limited to the following steps S101 to S104.

[0084] S101. Periodically, for each currently existing cluster category, count the total number of video alarm data classified into the corresponding category in the most recent period.

[0085] In the step S101, the period may be, but not limited to, an hour, a time of day, a half day, a day, or the like.

[0086] S102. If it is found that the total number of the video alarm data classified into any clustering category in the most recent cycle is less than the preset number threshold, then for each other clustering category currently existing, the similarity between the corresponding category and any clustering category is calculated based on the video alarm feature vector group of all the video alarm data classified into the corresponding category in the most recent cycle and the video alarm feature vector group of all the video alarm data classified into any clustering category in the most recent cycle.

[0087] In step S102, the preset number threshold can also be pre-set based on historical experience, for example, it is designed to be 5. The specific calculation process of the similarity is a prior art method, for example, firstly determining a multidimensional space center based on the video alarm feature vector group of all the video alarm data classified into any other clustering category in the most recent cycle, and determining another multidimensional space center based on the video alarm feature vector group of all the video alarm data classified into any clustering category in the most recent cycle, and then using the cosine similarity algorithm to calculate the cosine similarity of the two centers as the similarity between any other clustering category and any clustering category.

[0088] S103. Determine another clustering category corresponding to the maximum similarity according to the similarities between each other clustering category and any one of the clustering categories.

[0089] S104. Merge the other cluster category with any cluster category to obtain a merged cluster category.

[0090] Based on the aforementioned possible design 2, the clustering categories can be dynamically adjusted, such as merging, according to the latest actual division situation, so that the entire solution can continuously adapt to new alarm modes.

[0091] Based on the technical solutions of the possible design one or possible design two mentioned above, this embodiment also provides a possible design three for timely updating the partitioning model after the clustering category is dynamically adjusted, that is, after the clustering category changes (such as merging or creating a new clustering category, etc.), the method also includes but is not limited to: using the video alarm feature vector group of all the video alarm data that have been classified into the currently existing clustering categories in the most recent cycle as the model input item, and using the clustering category division results of all the video alarm data that have been classified into the currently existing clustering categories in the most recent cycle as the model output item, importing the artificial intelligence model to re-train the model, and obtaining a new alarm data clustering category division model. The specific details of the aforementioned model training can be found in the aforementioned step S5, which will not be repeated here.

[0092] Therefore, based on the aforementioned possible design three, the sustainability of the entire video alarm aggregation can be ensured by timely updating the partitioning model after the clustering categories are dynamically adjusted.

[0093] like Figure 4 As shown, the second aspect of this embodiment provides a virtual system for implementing the video alarm aggregation method described in the first aspect or any possible design of the first aspect, including an alarm data acquisition unit, a feature vector extraction unit, a clustering algorithm application unit, a data category division unit, a division model training unit, a division model application unit, a new data category division unit and an alarm action execution unit; The alarm data acquisition unit is used to acquire all video alarm data received in the past, wherein the video alarm data contains an alarm timestamp, an alarm type, an alarm location, and an alarm video clip; The feature vector extraction unit is communicatively connected to the alarm data acquisition unit, and is used to perform feature extraction processing on the alarm type, alarm position and alarm video segment in the corresponding data for each piece of video alarm data in all the video alarm data, so as to obtain a corresponding video alarm feature vector group including the alarm type feature vector, the alarm position feature vector and the alarm video feature vector; The clustering algorithm application unit is communicatively connected to the feature vector extraction unit, and is used to apply a clustering algorithm to cluster the video alarm feature vector groups of the respective video alarm data to obtain a plurality of cluster centers; The data category division unit is respectively connected to the feature vector extraction unit and the clustering algorithm application unit in communication, and is used to calculate the corresponding Euler distances to the multiple cluster centers according to the multiple cluster centers and the corresponding video alarm feature vector groups for each piece of video alarm data, and then determine a cluster center corresponding to the shortest Euler distance from the multiple cluster centers, and divide the corresponding data into a cluster category corresponding to the cluster center; The partition model training unit is respectively connected to the feature vector extraction unit and the data category partition unit for communication, and is used to use the video alarm feature vector group of each video alarm data as a model input item, and use the cluster category partition result of each video alarm data as a model output item, import the artificial intelligence model based on the machine learning algorithm for model training, and obtain the alarm data cluster category partition model; The feature vector extraction unit is further used to perform the feature extraction processing on the alarm type, alarm position and alarm video segment in the newly received video alarm data respectively to obtain a new video alarm feature vector group; The partition model application unit is respectively connected to the feature vector extraction unit and the partition model training unit for communication, and is used to import the new video alarm feature vector group into the alarm data cluster category partition model to obtain the probability of partitioning the newly received video alarm data into each cluster category; The new data category classification unit is communicatively connected to the classification model application unit, and is used to classify the newly received video alarm data into a certain cluster category if it is found that the probability of classifying the newly received video alarm data into the certain cluster category exceeds a preset probability threshold; The alarm action execution unit is communicatively connected to the new data category division unit, and is used to execute an alarm action corresponding to the certain cluster category according to the newly received video alarm data only when the duration from the most recent execution timestamp of the alarm action corresponding to the certain cluster category to the alarm timestamp in the newly received video alarm data exceeds a preset duration threshold.

[0094] The working process, working details and technical effects of the aforementioned system provided in the second aspect of this embodiment can be referred to the video alarm aggregation method described in the first aspect or any possible design of the first aspect, and will not be described in detail here.

[0095] like Figure 5As shown, the third aspect of this embodiment provides a computer system for executing the video alarm aggregation method as described in the first aspect or any possible design of the first aspect, including a memory, a processor and a transceiver that are sequentially connected in communication, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the video alarm aggregation method as described in the first aspect or any possible design of the first aspect. For example, the memory may include, but is not limited to, a random access memory (Random-Access Memory, RAM), a read-only memory (Read-Only Memory, ROM), a flash memory (Flash Memory), a first-in-first-out memory (First Input First Output, FIFO) and / or a first-in-last-out memory (First Input Last Output, FILO), etc.; the processor may include, but is not limited to, a microprocessor of the STM32F105 series. In addition, the computer system may also include, but is not limited to, a power module, a display screen and other necessary components.

[0096] The working process, working details and technical effects of the aforementioned computer system provided in the third aspect of this embodiment can be referred to the video alarm aggregation method described in the first aspect or any possible design of the first aspect, and will not be repeated here.

[0097] The fourth aspect of this embodiment provides a computer-readable storage medium storing instructions including the video alarm aggregation method as described in the first aspect or any possible design of the first aspect, that is, the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the video alarm aggregation method as described in the first aspect or any possible design of the first aspect is executed. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to computer-readable storage media such as floppy disks, optical disks, hard disks, flash memories, USB flash drives, and / or memory sticks, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0098] The working process, working details and technical effects of the aforementioned computer-readable storage medium provided in the fourth aspect of this embodiment can be referred to the video alarm aggregation method described in the first aspect or any possible design of the first aspect, and will not be repeated here.

[0099] A fifth aspect of the present embodiment provides a computer program product, including a computer program or an instruction, wherein the computer program or the instruction, when executed by a computer, implements the video alarm aggregation method as described in the first aspect or any possible design of the first aspect. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0100] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A video alarm aggregation method based on machine learning, characterized in that: include: Acquire all video alarm data received historically, wherein the video alarm data contains an alarm timestamp, an alarm type, an alarm location, and an alarm video clip; For each piece of video alarm data in all the video alarm data, feature extraction processing is performed on the alarm type, alarm position and alarm video segment in the corresponding data to obtain a corresponding video alarm feature vector group including an alarm type feature vector, an alarm position feature vector and an alarm video feature vector; According to the video alarm feature vector group of each piece of video alarm data, clustering is performed using a clustering algorithm to obtain a plurality of cluster centers; For each of the video alarm data, according to the multiple cluster centers and the corresponding video alarm feature vector groups, the corresponding Euler distances to the multiple cluster centers are calculated respectively, and then a cluster center corresponding to the shortest Euler distance is determined from the multiple cluster centers, and the corresponding data is divided into a cluster category corresponding to the cluster center; The video alarm feature vector group of each video alarm data is used as a model input item, and the clustering category division result of each video alarm data is used as a model output item, and an artificial intelligence model based on a machine learning algorithm is imported for model training to obtain an alarm data clustering category division model; Performing the feature extraction process on the alarm type, alarm position and alarm video segment in the newly received video alarm data respectively to obtain a new video alarm feature vector group; Importing the new video alarm feature vector group into the alarm data clustering category classification model to obtain the probability of classifying the newly received video alarm data into each cluster category; If it is found that the probability of classifying the newly received video alarm data into a certain cluster category exceeds a preset probability threshold, the newly received video alarm data is classified into the certain cluster category; Only when the duration from the most recent execution timestamp of the alarm action corresponding to the certain cluster category to the alarm timestamp in the newly received video alarm data exceeds a preset duration threshold, an alarm action corresponding to the certain cluster category is executed once according to the newly received video alarm data.

2. The video alarm aggregation method according to claim 1, characterized in that: For each piece of video alarm data in all the video alarm data, feature extraction processing is performed on the alarm type, alarm position and alarm video segment in the corresponding data, respectively, to obtain a corresponding video alarm feature vector group including an alarm type feature vector, an alarm position feature vector and an alarm video feature vector, including: For a certain video alarm data among all the video alarm data, a single hot encoding method and a layered encoding method are used to encode the alarm type in the corresponding data to obtain a corresponding alarm type feature vector; Normalizing the alarm position in the certain video alarm data and being the coordinate information in the video screen to obtain normalized coordinates, and then combining the normalized coordinates with the alarm position in the certain video alarm data and being the monitoring area number and / or the geometric shape field of view partition number in the monitoring area to form an alarm position feature vector of the certain video alarm data; Importing the alarm video clip in the certain video alarm data into the pre-trained convolutional neural network model, and then using the result output by the intermediate layer in the convolutional neural network model as the alarm video feature vector of the certain video alarm data; The alarm type feature vector, the alarm position feature vector and the alarm video feature vector of the certain video alarm data are combined into a video alarm feature vector group of the certain video alarm data.

3. The video alarm aggregation method according to claim 2, characterized in that: The result output by the middle layer in the convolutional neural network model is used as the alarm video feature vector of the certain video alarm data, including: Using the result output by the intermediate layer in the convolutional neural network model as a video segment feature vector; Performing PCA dimensionality reduction processing on the feature vector of the video clip to obtain a feature vector after dimensionality reduction; The feature vector after dimension reduction is used as the alarm video feature vector of the certain video alarm data.

4. The video alarm aggregation method according to claim 1, characterized in that: According to the video alarm feature vector group of each video alarm data, a clustering algorithm is applied to obtain a plurality of cluster centers, including: For each of the video alarm data, according to the corresponding video alarm feature vector group and the video alarm feature vector group of each other video alarm data, calculate the corresponding Euler distance with each of the other video alarm data, and count the number of distances whose corresponding Euler distance is less than a preset distance threshold; According to the number of distances of each piece of video alarm data and the distribution position of each piece of video alarm data in the multidimensional space, a plurality of pieces of video alarm data having a maximum number of distances among all the video alarm data are determined, wherein each dimension in the multidimensional space corresponds one-to-one to each dimensional feature value in the video alarm feature vector group; Arrange the plurality of video alarm data in descending order according to the maximum values ​​of the number of distances to obtain a video alarm data sequence; Select the previous one from the video alarm data sequence A copy of the video alarm data, and the The video alarm feature vector groups of the video alarm data are used one by one as Initial cluster centers, where Indicates the preset number of clustering categories; Based on the Initial cluster centers are selected, and multiple cluster centers are obtained by clustering using the K-Means clustering algorithm and other steps after the initial cluster center selection step.

5. The video alarm aggregation method according to claim 1, characterized in that: After obtaining the probability of classifying the newly received video alarm data into each cluster category, the method further includes: If it is found that the probability of classifying the newly received video alarm data into each cluster category does not exceed the preset probability threshold, then according to the probability of classifying the newly received video alarm data into each cluster category, determine a first cluster category corresponding to the highest probability and a second cluster category corresponding to the second highest probability, wherein the second highest probability refers to a probability that is only less than the highest probability; Determine whether the sum of the highest probability and the second highest probability exceeds the preset probability threshold and whether the difference between the highest probability and the second highest probability is less than a preset difference threshold; If both are true, merging the first cluster category and the second cluster category, and classifying the newly received video alarm data into the merged cluster category; Only when the duration from the most recent execution timestamp of the alarm action corresponding to the merged cluster category to the alarm timestamp in the newly received video alarm data exceeds the preset duration threshold, the alarm action corresponding to the merged cluster category is executed once according to the newly received video alarm data.

6. The video alarm aggregation method according to claim 5, characterized in that: After determining whether the sum of the highest probability and the second highest probability exceeds the preset probability threshold and whether the difference between the highest probability and the second highest probability is less than a preset difference threshold, the method further includes: If it is determined that the sum of the highest probability and the second highest probability does not exceed the preset probability threshold and / or the difference between the highest probability and the second highest probability is greater than or equal to the preset difference threshold, a new cluster category is created, and the newly received video alarm data is classified into the new cluster category; An alarm action corresponding to the new clustering category is executed according to the newly received video alarm data.

7. The video alarm aggregation method according to claim 1, characterized in that: The method further comprises: Periodically counting the total number of the video alarm data classified into the corresponding category in the most recent period for each currently existing cluster category; If it is found that the total number of the video alarm data classified into any cluster category in the most recent cycle is less than the preset number threshold, then for each other cluster category currently existing, based on the video alarm feature vector group of all the video alarm data classified into the corresponding category in the most recent cycle and the video alarm feature vector group of all the video alarm data classified into any cluster category in the most recent cycle, calculate the similarity between the corresponding category and any cluster category; According to the similarities between the other cluster categories and the any cluster category, determining another cluster category corresponding to the maximum similarity; The other cluster category is merged with any cluster category to obtain a merged cluster category.

8. The video alarm aggregation method according to claim 5, 6 or 7, characterized in that: After the clustering category is changed, the method further includes: The video alarm feature vector group of all the video alarm data classified into the currently existing cluster categories in the most recent cycle is used as a model input item, and the cluster category classification result of all the video alarm data classified into the currently existing cluster categories in the most recent cycle is used as a model output item, and the artificial intelligence model is imported to re-train the model to obtain a new alarm data cluster category classification model.

9. A video alarm aggregation system based on machine learning, characterized in that: It includes an alarm data acquisition unit, a feature vector extraction unit, a clustering algorithm application unit, a data category division unit, a division model training unit, a division model application unit, a new data category division unit and an alarm action execution unit; The alarm data acquisition unit is used to acquire all video alarm data received in the past, wherein the video alarm data contains an alarm timestamp, an alarm type, an alarm location, and an alarm video clip; The feature vector extraction unit is communicatively connected to the alarm data acquisition unit, and is used to perform feature extraction processing on the alarm type, alarm position and alarm video segment in the corresponding data for each piece of video alarm data in all the video alarm data, so as to obtain a corresponding video alarm feature vector group including the alarm type feature vector, the alarm position feature vector and the alarm video feature vector; The clustering algorithm application unit is communicatively connected to the feature vector extraction unit, and is used to apply a clustering algorithm to cluster the video alarm feature vector groups of the respective video alarm data to obtain a plurality of cluster centers; The data category division unit is respectively connected to the feature vector extraction unit and the clustering algorithm application unit in communication, and is used to calculate the corresponding Euler distances to the multiple cluster centers according to the multiple cluster centers and the corresponding video alarm feature vector groups for each piece of video alarm data, and then determine a cluster center corresponding to the shortest Euler distance from the multiple cluster centers, and divide the corresponding data into a cluster category corresponding to the cluster center; The partition model training unit is respectively connected to the feature vector extraction unit and the data category partition unit for communication, and is used to use the video alarm feature vector group of each video alarm data as a model input item, and use the cluster category partition result of each video alarm data as a model output item, import the artificial intelligence model based on the machine learning algorithm for model training, and obtain the alarm data cluster category partition model; The feature vector extraction unit is further used to perform the feature extraction processing on the alarm type, alarm position and alarm video segment in the newly received video alarm data respectively to obtain a new video alarm feature vector group; The partition model application unit is respectively connected to the feature vector extraction unit and the partition model training unit for communication, and is used to import the new video alarm feature vector group into the alarm data cluster category partition model to obtain the probability of partitioning the newly received video alarm data into each cluster category; The new data category classification unit is communicatively connected to the classification model application unit, and is used to classify the newly received video alarm data into a certain cluster category if it is found that the probability of classifying the newly received video alarm data into the certain cluster category exceeds a preset probability threshold; The alarm action execution unit is communicatively connected to the new data category division unit, and is used to execute an alarm action corresponding to the certain cluster category according to the newly received video alarm data only when the duration from the most recent execution timestamp of the alarm action corresponding to the certain cluster category to the alarm timestamp in the newly received video alarm data exceeds a preset duration threshold.

10. A computer system, characterized in that: It includes a memory, a processor and a transceiver which are communicatively connected in sequence, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the video alarm aggregation method as described in any one of claims 1 to 8.

Images