Abnormal behavior security early warning system based on multi-video image analysis

Abstract

The present invention discloses an abnormal behavior security early warning system based on multi-video image analysis, which relates to the field of security early warning, including: a path construction module, which constructs a security path based on the installation position of a video acquisition device and constructs an abnormal behavior comparison library; a feature comparison module, which uses an algorithm to construct a security comparison model, identifies abnormal behavior of personnel and obtains a comprehensive risk index through quantitative analysis, and determines abnormal security personnel; a personnel determination module, which divides the security path into grades and determines security risk personnel; a personnel analysis module, which determines the change in risk determination value of security risk personnel and determines the priority investigation path; an area analysis module, which determines dangerous areas and high-risk security personnel through cluster analysis; a model optimization module, which verifies high-risk and suspicious personnel, obtains mislabeled and missed personnel, and optimizes the security comparison model. The present invention is conducive to improving the security early warning capability and is suitable for a variety of security scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of security early warning, and in particular to an abnormal behavior security early warning system based on multi-video image analysis. Background Art

[0002] In the security field, as society's demand for security continues to increase, traditional security systems often rely on manual patrols or simple video surveillance, which have many defects.

[0003] In the existing technology, the security system can only perform preliminary action recognition in terms of personnel behavior analysis, and is unable to conduct in-depth analysis and quantitative analysis of personnel behavior. It is difficult to accurately determine whether personnel behavior is truly abnormal and its risk level, resulting in a large number of potential security risk personnel being ignored, increasing the possibility of safety accidents.

[0004] In the existing technology, there is a lack of a complete risk calculation mechanism. It is unable to comprehensively consider multiple factors such as security path level, abnormal behavior frequency, persistence index, diversity ratio, etc. to accurately determine the risk level of personnel, thereby failing to provide security personnel with a scientific and effective decision-making basis. There is blindness in resource allocation and key prevention and control. The existing technology has not established a feedback mechanism based on actual security verification results, which makes it impossible for the comparison model to learn and adapt to new behavior patterns in a timely manner when facing complex and changeable security scenarios. It is easy to accumulate errors due to the inherent defects of the algorithm, resulting in a gradual decline in security performance and an inability to ensure the safety of the security area in a long-term and stable manner.

[0005] To this end, the present invention provides an abnormal behavior security early warning system based on multi-video image analysis. Summary of the invention

[0006] The object of the present invention is to provide an abnormal behavior security warning system based on multi-video image analysis to solve at least one of the above-mentioned prior art problems.

[0007] In a first aspect, the present invention provides an abnormal behavior security early warning system based on multi-video image analysis, comprising:

[0008] Path construction module: Based on the installation location of the video acquisition device, build a security path, mark historical abnormal data, and build an abnormal behavior comparison library;

[0009] Feature comparison module: Based on the video image data obtained by the video acquisition device, a security comparison model is constructed to identify abnormal behaviors of personnel, quantitatively analyze the degree of abnormal behaviors of personnel, obtain a comprehensive risk index, determine abnormal security personnel within the security path, and store abnormal security personnel in the abnormal security personnel database;

[0010] Personnel determination module: divides the security path into multiple security levels, obtains abnormal security personnel from the abnormal security personnel database, and makes risk judgments on the abnormal behaviors of abnormal security personnel to determine whether the abnormal security personnel are security risk personnel;

[0011] Personnel analysis module: Based on security risk personnel, determine whether the risk judgment value of security risk personnel within the security level path is continuously increasing. If it is continuous, a security warning signal is generated, and the security personnel will check the security risk personnel. Conversely, the risk level of security risk personnel within each security level is quantitatively analyzed to determine the priority investigation path;

[0012] Area analysis module: Based on the priority screening path, the distribution of security risk personnel at each security level is analyzed for cluster analysis to obtain the dangerous areas of the priority screening path, and the security risk personnel in the dangerous areas are marked as high-risk security personnel;

[0013] Model optimization module: Security personnel verify high-risk personnel to determine whether they are mislabeled. If so, the security comparison model is adjusted and optimized.

[0014] In a second aspect, the present invention provides an abnormal behavior security early warning method based on multi-video image analysis, comprising:

[0015] Step 1: Based on the installation location of the video acquisition device, build a security path, mark historical abnormal data, and build an abnormal behavior comparison library;

[0016] Step 2: Based on the video image data obtained by the video acquisition device, a security comparison model is constructed to identify the abnormal behavior of personnel, quantify the degree of abnormality of personnel behavior, obtain a comprehensive risk index, determine the abnormal security personnel in the security path, and store the abnormal security personnel in the abnormal security personnel database;

[0017] Step 3: divide the security path into multiple security levels, obtain the security abnormal personnel from the security abnormal personnel database, and make risk judgments on the abnormal behaviors of the security abnormal personnel to determine whether the security abnormal personnel are security risk personnel;

[0018] Step 4: Based on the security risk personnel, determine whether the risk judgment value of the security risk personnel within the security level path is continuously increasing. If it is continuous, generate a security warning signal, and the security personnel will check the security risk personnel. Otherwise, conduct a quantitative analysis of the risk level of the security risk personnel within each security level to determine the priority investigation path;

[0019] Step 5: Based on the priority screening path, analyze the distribution of security risk personnel at each security level and perform cluster analysis to obtain the dangerous areas of the priority screening path, and mark the security risk personnel in the dangerous areas as high-risk security personnel;

[0020] Step 6: Security personnel verify high-risk personnel to determine whether they are mislabeled. If so, the security comparison model is adjusted and optimized.

[0021] Beneficial effects of the present invention:

[0022] 1. The present invention can identify abnormal security personnel and risk personnel and accurately judge their risk level through video image analysis technology and multi-module collaborative work. The operation of the personnel judgment module and the personnel analysis module can timely discover dangerous personnel with continuously increasing risk judgment values, quickly generate warning signals, and enable security personnel to intervene and check as soon as possible, thereby improving the timeliness and accuracy of security warnings and helping to ensure the safety of security areas.

[0023] 2. With the help of the model optimization module, the system can optimize and adjust the security comparison model according to the actual security situation. By marking and processing the data of mislabeled and missed persons, calculating the model training value and adjustment value, and increasing the number of model algorithm training times, the model can continuously learn new behavior patterns, continuously improve the security system's ability to identify abnormal behaviors and overall security performance, and adapt to complex and changing security environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 It is a module diagram of the abnormal behavior security early warning system based on multi-video image analysis of the present invention;

[0026] Figure 2 It is a flow chart of the abnormal behavior security early warning method based on multi-video image analysis of the present invention;

[0027] Figure 3 It is a structural diagram of a computer device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] Embodiment 1

[0030] Figure 1 This is a module diagram of an abnormal behavior security warning system based on multi-video image analysis in the present invention. The abnormal behavior security warning system based on multi-video image analysis can be implemented by software and / or hardware, and the abnormal behavior security warning system based on multi-video image analysis can be configured in a computer device. Optionally, the computer device can be an electronic device, and the electronic device can be a notebook, a desktop computer, a smart tablet, etc., which is not limited in the embodiment of the present invention.

[0031] like Figure 1 As shown, the abnormal behavior security early warning system based on multi-video image analysis of the present invention includes the following modules:

[0032] Path construction module: Based on the installation location of the video acquisition device, build a security path, mark historical abnormal data, and build an abnormal behavior comparison library;

[0033] In some embodiments, video capture equipment is installed in the target security area;

[0034] Construct security paths based on the installation paths of video acquisition devices within the target security area;

[0035] Obtain historical video image data from video acquisition devices and the Internet, mark abnormal behaviors in the video image data, use all abnormal behaviors as a data set, and build an abnormal behavior comparison library;

[0036] For example, historical video image data is obtained from video acquisition equipment and the Internet, video data of various security incidents that have occurred in the past are collected, and abnormal behavior features related to security are extracted, such as the modus operandi, escape route, appearance features, body movements, facial expressions, etc. of suspects in theft cases;

[0037] Feature comparison module: Based on the video image data obtained by the video acquisition device, a security comparison model is constructed to identify abnormal behaviors of personnel, quantitatively analyze the degree of abnormal behaviors of personnel, obtain a comprehensive risk index, determine abnormal security personnel within the security path, and store abnormal security personnel in the abnormal security personnel database;

[0038] Based on the video image data obtained by the video acquisition device, a security comparison model is built to identify abnormal behavior;

[0039] The specific security comparison model is constructed as follows:

[0040] A1. Use the YOLOv8 algorithm to identify the outline of people in each frame of the video image;

[0041] It should be noted that YOLOv8 is a target monitoring algorithm. The principle is to divide the input image into grids, each grid is responsible for monitoring the object in it, and through one forward propagation, the network simultaneously predicts the bounding box, category and confidence of the object.

[0042] Obtain historical security area video data from the video acquisition device, divide the video data into video frames, and mark the position of the personnel in each video frame;

[0043] The labeled data is preprocessed, and the labeled image frames are used as the data set, which is divided into a training set and a test set in a ratio of 7:3;

[0044] The training set is input into the YOLOv8 algorithm for model training. The learning rate of the algorithm is 0.001, the batch size used for each algorithm update is 16, and the number of training rounds is 100;

[0045] Input the real-time video image into the YOLOv8 algorithm to identify the outline of people in each frame of the video image;

[0046] A2. Use a two-stream convolutional neural network algorithm to identify abnormal behavior of personnel;

[0047] It should be noted that the two-stream convolutional neural network contains two branches: spatial stream and temporal stream. The spatial stream is used to process the spatial information of the video frame and extract the appearance features of the characters, such as posture and clothing. The temporal stream is used to process the time series information of the video and capture the dynamic features of the characters' movements, such as walking, running, fighting, etc. The features of the two branches are then merged to perform behavior classification. In security, it can accurately identify abnormal behaviors such as bending over and reaching for things during theft, or physical conflicts in fights.

[0048] Based on the YOLOv8 algorithm, the person outline of each frame of the video image is identified, and the area image of each frame of the video image where each person outline is located is cropped to obtain the person outline image;

[0049] The person outline image is scaled to a size of 222*224 pixels, and the cropped person outline image in each frame is used as the spatial stream input. For the time stream, the person outline images of 10 adjacent frames are arranged in chronological order to construct a time series data for extracting the dynamic features of the person's behavior changing over time.

[0050] The person profile image dataset containing spatiotemporal data is divided into a training set and a test set in a ratio of 8:2. The abnormal behavior comparison library is used to annotate the corresponding behavior category labels for the person profile images corresponding to each set of spatiotemporal data. The labels clearly indicate normal behavior and abnormal behavior.

[0051] Set appropriate training parameters, for example, the learning rate can be initially set to 0.0001, the batch size to 8, and the number of training rounds to 500;

[0052] Based on the security comparison model, identify abnormal behavior of people in video image data;

[0053] If a person shows abnormal behavior in the video image data, obtain the number of times the person is marked as having abnormal behavior by the video acquisition device during the monitoring period, marked as Cs;

[0054] By formula: Obtain the personnel's abnormal behavior frequency AF, where T is the length of the monitoring cycle;

[0055] By formula: Get the person's abnormal behavior duration index ADTI (Abnormal Behavior Duration Time Indicator), where Cx k is the duration of each abnormal behavior, k is the number of each abnormal behavior;

[0056] Obtain the number of types of abnormal behaviors of personnel and the number of abnormal behaviors in the abnormal behavior comparison library, perform ratio processing on the number of types of abnormal behaviors of personnel and the number of abnormal behaviors in the abnormal behavior comparison library to obtain an abnormal diversity ratio, and mark the abnormal diversity ratio as Yc;

[0057] The abnormal behavior frequency AF, the abnormal behavior persistence index ADTI of personnel, and the abnormal diversity ratio Yc are dimensionless processed;

[0058] The abnormal behavior frequency AF, the abnormal behavior persistence index ADTI of the personnel, and the abnormal diversity ratio Yc are weighted and summed to obtain a comprehensive risk index;

[0059] Compare the comprehensive risk index with the risk index threshold to identify security abnormalities;

[0060] If the comprehensive risk index of a person in the video image data is higher than the risk threshold, the person is marked as a security abnormal person and the security abnormal person is stored in the security abnormal person database;

[0061] If the comprehensive risk index of a person in the video image data is lower than the risk threshold, it is necessary to continuously monitor the changes in the comprehensive risk index of the person;

[0062] The technical solution of this embodiment is: based on the installation location of the video acquisition device, a security path is constructed, historical abnormal data is marked, an abnormal behavior comparison library is constructed, and a security comparison model is constructed based on the video image data obtained by the video acquisition device to identify abnormal behavior of personnel, and the abnormal degree of personnel behavior is quantitatively analyzed to obtain a comprehensive risk index, determine the abnormal security personnel within the security path, and store the abnormal security personnel in the abnormal security personnel database, which is conducive to the subsequent analysis of abnormal security personnel.

[0063] Embodiment 2

[0064] like Figure 1 As shown, the abnormal behavior security early warning system based on multi-video image analysis of the present invention also includes the following modules:

[0065] Personnel determination module: divides the security path into multiple security levels, obtains abnormal security personnel from the abnormal security personnel database, and makes risk judgments on the abnormal behaviors of abnormal security personnel to determine whether the abnormal security personnel are security risk personnel;

[0066] Obtain the security path, divide the security path into multiple paths of different levels according to the division method from outside to inside and from sparse personnel to dense personnel, and obtain the security level path;

[0067] For example, taking a hospital as an example, the security paths are divided into 1-4 levels, namely: core medical area, diagnosis and treatment area, public service area, and peripheral protection area;

[0068] From the security comparison model, obtain the number of abnormal behaviors and the number of all behaviors of security abnormal personnel in each security level path;

[0069] The number of abnormal behaviors in each security level is compared with the number of all behaviors to obtain the abnormal behavior ratio Pf;

[0070] Based on the first appearance probability Pf of security abnormal personnel, abnormal behavior frequency AF, abnormal behavior index ADTI, and abnormal diversity ratio Yc in each security level, a conditional probability table is constructed.

[0071] Based on the conditional probability table, using Bayesian theorem, calculate the posterior probability P of security abnormal personnel under different security level paths i , where i represents the security path level, and the value range of i is [1,n];

[0072] By Bayes' formula: Get the posterior probability of each security level path ;

[0073] It should be noted that P(i) is obtained from historical statistical data. For example, in the past period of time, the proportion of paths belonging to level i among all security paths is calculated as P(i);

[0074] It refers to the joint conditional probability of abnormal behavior ratio Pf, abnormal behavior frequency AF, abnormal behavior index ADTI and abnormal diversity ratio Yc appearing in the conditional probability table under the condition that the security path level is ;

[0075] By formula: Get , where n is the total number of security levels, and P(j) is another way of expressing the prior probability P(i);

[0076] The posterior probability in each security level path is multiplied by the comprehensive risk index to obtain the risk judgment value;

[0077] The risk determination value is compared with the risk determination threshold. If the risk determination value of the security abnormal person is higher than the risk determination threshold, the security abnormal person is marked as a security risk person, and the data of the security risk person is uploaded to the security abnormal person database;

[0078] If the risk assessment value of the security abnormal personnel is lower than the risk assessment threshold, the risk assessment value of the security abnormal personnel needs to be continuously monitored;

[0079] Personnel analysis module: Based on security risk personnel, determine whether the risk judgment value of security risk personnel within the security level path is continuously increasing. If it is continuous, a security warning signal is generated, and the security personnel will check the security risk personnel. Conversely, the risk level of security risk personnel within each security level is quantitatively analyzed to determine the priority investigation path;

[0080] Obtain historical data of security risk personnel from the security abnormal personnel database to determine whether the security risk personnel is marked as a security risk personnel for the first time;

[0081] If the security risk person is not marked as a security risk person for the first time, a security warning signal will be generated and security personnel will be required to check the security risk person;

[0082] If the security risk personnel is marked as a security risk personnel for the first time, it is determined whether the risk determination value of the security risk personnel is continuously increasing within the security level path;

[0083] Specifically, the risk determination value of the security risk personnel in each security level path is obtained, and the risk determination value in each security level path is used as a data element to construct a security risk personnel sequence;

[0084] Mark the security risk personnel sequence as Ri , i is the number of each security level path, and the value range of i is [1,n];

[0085] Calculate the first-order difference of adjacent risk judgment values ​​in the security risk personnel sequence R i ;

[0086] By formula: R i =R i+1 -R i Get the first-order difference of adjacent risk judgment values R i ;

[0087] like R i >0, marked as positive difference, if the security risk personnel sequence R1, R2, ..., R n All of them are positive differences, which means that the security risk personnel are within the security level path, and the risk judgment value is continuously increasing, then a security warning signal is generated, and security personnel are required to check the security risk personnel;

[0088] like R i ≤0, it is considered that the security risk personnel are within the security level path, the risk judgment value changes discontinuously, and the security risk personnel are marked as interval risk personnel;

[0089] It should be noted that if the risk assessment value of a security risk person is within the security level, it means that the risk assessment value is continuously increasing in the process of transitioning from a lower security level to a higher security level. This continuous increase is likely to indicate that the behavior of the risk person is becoming more and more threatening. By generating security warning signals in a timely manner, security personnel can intervene and check as early as possible to avoid potential safety accidents.

[0090] On the contrary, the risk assessment value of security risk personnel changes intermittently within the security level. Security risk personnel change their behavior due to security measures or external environmental factors, which reduces the risk assessment value. It is necessary to further analyze the behavior changes of security risk personnel.

[0091] Obtain the number of risk personnel and the total number of personnel within each security level;

[0092] The ratio of the number of interval risk personnel to the number of all personnel is processed to obtain the interval personnel ratio;

[0093] Obtain the risk determination values ​​of all interval risk personnel within each security level, sum and average the risk determination values ​​of all interval risk personnel, and obtain the interval risk mean;

[0094] Multiply the interval personnel ratio and the interval risk mean to obtain the interval risk value;

[0095] Obtain the interval risk value within each security level path, sort the security level paths in descending order of risk value, and determine the priority investigation path;

[0096] The technical solution of this embodiment is: divide the security path into multiple security levels, obtain abnormal security personnel from the abnormal security personnel database, and make risk judgments on the abnormal behaviors of abnormal security personnel to determine whether the abnormal security personnel are security risk personnel. Based on the security risk personnel, judge whether the risk judgment value of the security risk personnel in the security level path is continuously increasing. If it is continuous, generate a security warning signal, and the security personnel will check the security risk personnel. Otherwise, quantify the risk degree of the security risk personnel in each security level and determine the priority screening path. Based on the priority screening path, it is conducive to more efficient and targeted allocation of security resources, thereby improving the efficiency and effectiveness of security work.

[0097] Embodiment 3

[0098] like Figure 1 As shown, the abnormal behavior security warning system based on multi-video image analysis of the present invention also includes the following modules

[0099] Area analysis module: Based on the priority screening path, the distribution of security risk personnel at each security level is analyzed for cluster analysis to obtain the dangerous areas of the priority screening path, and the security risk personnel in the dangerous areas are marked as high-risk security personnel;

[0100] Obtain the risk assessment values ​​of all security risk personnel in the priority screening path, use the risk assessment values ​​of all security risk personnel as data elements, and construct a risk personnel data group;

[0101] In the risk personnel data group, the risk determination values ​​of the security risk personnel are sorted in descending order, and the security risk personnel corresponding to the maximum risk determination value is obtained as the central risk personnel;

[0102] Taking the coordinates of the central risk personnel as the origin, establish the security path coordinate system;

[0103] During the monitoring period, the security path coordinate system and the coordinates of all security risk personnel are obtained in real time, and the coordinates of each security risk personnel and the risk judgment value are stored as a risk group in the dynamic risk set;

[0104] For example, the dynamic risk set is: LR j ={[(3,5)、0.8],(-2,4)、0.6],......,(0,0)、0.9,(4,-2)、0.5]]}, where [(3,5)、0.8] represents the risk group, (3,5) and 0.8 represent the coordinates and risk judgment value of the security risk personnel in the security path coordinate system, respectively, j represents the number of each security risk personnel in the dynamic risk set, and the value range of j is [1,m];

[0105] Through the density clustering algorithm, a regional determination model is established to determine the dangerous areas within the priority inspection path;

[0106] Specifically, the regional determination model is established as follows:

[0107] S1. Based on the dynamic risk set, quantitative analysis is performed on the risk data of the dynamic risk set to determine the neighborhood radius ε;

[0108] Specifically, the variances σ1 and σ2 of the risk judgment values ​​of all security risk personnel in two adjacent monitoring periods in the dynamic risk set are obtained;

[0109] Calculate the variance change of the risk judgment value of two adjacent monitoring periods σ, σ=|σ1-σ2|;

[0110] Get the x-axis range of the security path coordinates in the dynamic risk set [x min ,x max ]、y-axis range [y min ,y max ];

[0111] Calculate the spatial range parameter S based on the x-axis range and y-axis range of the security path coordinates;

[0112] By formula: Get the spatial range parameter S;

[0113] Get the change of the spatial range parameter S between two adjacent monitoring periods, marked as S, S=|S1-S2|;

[0114] Get the dynamic risk set, the moving speed V of each security risk personnel in adjacent monitoring cycles;

[0115] It should be noted that the moving speed V of each security risk personnel j， It is calculated for the same security risk personnel; the moving speed V of each security risk personnel j, the displacement can be determined by the change in the coordinate position of the security risk personnel in the adjacent monitoring period, and the displacement is processed by the ratio of the time length of the adjacent monitoring period to obtain the moving speed V of each security risk personnel j ;

[0116] Obtain the change rate Bh of the number of security risk personnel in adjacent monitoring cycles of the dynamic risk set;

[0117] It should be noted that the change rate of security risk personnel is obtained by ratioing the change in the number of people in the dynamic risk set between two adjacent monitoring periods with the length of time of the adjacent monitoring periods to obtain the change rate of the number of security risk personnel Bh;

[0118] By formula: Get the neighborhood radius ε;

[0119] S2, determine the cluster center point within the neighborhood radius ε;

[0120] Get the distance d of each security risk person in the priority screening path. For each risk person in the path risk set, calculate the number of security risk persons within the neighborhood radius ε. If the number is greater than 3, the latest position coordinate point of the security risk person within the neighborhood radius ε is marked as the center point.

[0121] S3, generate clusters using density clustering algorithm;

[0122] Use density clustering algorithm to cluster the location data of security risk personnel and ordinary personnel;

[0123] Starting from any core point, clusters are constructed through density reachability relationships. If the point corresponding to a risk group is within the neighborhood of the core point, it is considered that the direct density between the core points is reachable;

[0124] If there is a point chain such that each point is directly density-reachable to the next point, then these points are density-reachable, and all density-reachable points are grouped into the same cluster;

[0125] For example, if the risk group [(0,0), 0.9] is the core point, and the risk group [(3,5), 0.6] is within the core point neighborhood radius ε, it is considered that [(0,0), 0.9] and [(3,5), 0.6] are directly density-reachable, and [(0,0), 0.9] and [(3,5), 0.6] are marked as the same cluster;

[0126] S4, obtaining all clusters generated by the density clustering algorithm, sorting the clusters according to the number of risk groups in the clusters, and determining the cluster area with the most risk groups as the dangerous area;

[0127] Based on the dangerous areas within the priority inspection path, the security risk personnel in the dangerous areas are marked as high-risk security personnel;

[0128] Model optimization module: security personnel verify high-risk personnel and determine whether they are mislabeled. If so, the security comparison model is adjusted and optimized;

[0129] Based on the high-risk security personnel, the security guards will check the high-risk security personnel to determine whether the high-risk security personnel have been mislabeled by the system;

[0130] If a security high-risk person is mistakenly marked by the system, the security personnel will check the mistakenly marked security risk personnel in the personnel analysis module and mark the mistakenly marked security risk personnel and security high-risk personnel as mistakenly marked personnel;

[0131] If a security officer or guard finds a person whose behavior is suspicious during a security route patrol, but the person is not marked as a security risk person or a security high-risk person by the system, the person will be marked as a missed person;

[0132] Obtain video images of mislabeled persons and missed persons in the security comparison model in the feature comparison module;

[0133] The abnormal behaviors in the video images of the omitted persons are stored in an abnormal behavior comparison library;

[0134] Mark the misjudged abnormal behavior of the video image of the mislabeled person as normal behavior;

[0135] It should be noted that the normal and abnormal behaviors of mislabeled and omitted persons are manually labeled to avoid the accumulation of errors and misjudgment of complex behavior patterns caused by relying solely on algorithms;

[0136] Obtain the number of mislabeled personnel and the number of personnel with security anomalies, and perform ratio processing on the number of mislabeled personnel and the number of personnel with security anomalies to obtain the mislabeled personnel ratio;

[0137] Obtain the number of personnel identified in the security comparison model, perform subtraction processing on the number of personnel identified in the security comparison model and the number of security abnormal personnel to obtain the number of normal personnel;

[0138] Obtain the number of missed persons, and perform ratio processing on the number of missed persons and the number of normal persons to obtain the missed person ratio;

[0139] The weighted sum of the mislabeled personnel ratio and the omitted personnel ratio is processed to obtain the model training value;

[0140] The number of training rounds of the model algorithm in the security comparison model is multiplied by the model training value to obtain the model adjustment value;

[0141] Based on the model adjustment value, the number of training times of the model algorithm of the security comparison model is increased to achieve optimization adjustment of the security comparison model;

[0142] It should be noted that the model algorithms in the security comparison model include: YOLOv8, two-stream convolutional neural network algorithm, and the number of training rounds of the model algorithm in the security comparison model is multiplied by the model training value, which is to calculate each model algorithm in the security comparison model in turn;

[0143] The technical solution of this embodiment is: based on the priority screening path, the distribution of security risk personnel at each security level is analyzed for cluster analysis to obtain the dangerous areas of the priority screening path, and the security risk personnel in the dangerous areas are marked as high-risk security personnel. The security personnel verify the high-risk personnel to determine whether the personnel are mislabeled. If there is a mislabeling, the security comparison model is adjusted and optimized.

[0144] Embodiment 4

[0145] like Figure 2 As shown, the present invention provides an abnormal behavior security early warning method based on multi-video image analysis, comprising the following steps:

[0146] Step 1: Based on the installation location of the video acquisition device, build a security path, mark historical abnormal data, and build an abnormal behavior comparison library;

[0147] Step 2: Based on the video image data obtained by the video acquisition device, a security comparison model is constructed to identify the abnormal behavior of personnel, quantify the degree of abnormality of personnel behavior, obtain a comprehensive risk index, determine the abnormal security personnel in the security path, and store the abnormal security personnel in the abnormal security personnel database;

[0148] Step 3: divide the security path into multiple security levels, obtain the security abnormal personnel from the security abnormal personnel database, and make risk judgments on the abnormal behaviors of the security abnormal personnel to determine whether the security abnormal personnel are security risk personnel;

[0149] Step 4: Based on the security risk personnel, determine whether the risk judgment value of the security risk personnel within the security level path is continuously increasing. If it is continuous, generate a security warning signal, and the security personnel will check the security risk personnel. Otherwise, conduct a quantitative analysis of the risk level of the security risk personnel within each security level to determine the priority investigation path;

[0150] Step 5: Based on the priority screening path, analyze the distribution of security risk personnel at each security level and perform cluster analysis to obtain the dangerous areas of the priority screening path, and mark the security risk personnel in the dangerous areas as high-risk security personnel;

[0151] Step 6: Security personnel verify high-risk personnel to determine whether they are mislabeled. If so, the security comparison model is adjusted and optimized.

[0152] Embodiment 5

[0153] Reference Figure 3 The embodiment of the present invention further provides a computer device 3, comprising: a memory 302 and a processor 301 and a computer program 303 stored in the memory 302. When the computer program 303 is executed on the processor 301, the abnormal behavior security warning method based on multi-video image analysis as described in any one of the above methods is implemented.

[0154] The computer device 3 may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art may understand that;

[0155] Figure 3 It is only an example of computer device 3 and does not constitute a limitation on computer device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, for example, it may also include input and output devices, network access devices, etc.

[0156] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0157] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may also be an external storage device of the computer device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 3. Further, the memory 302 may also include both an internal storage unit and an external storage device of the computer device 3. The memory 302 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 302 may also be used to temporarily store data that has been output or is to be output.

[0158] Embodiment 6

[0159] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the abnormal behavior security warning method based on multi-video image analysis as described in any one of the above methods is implemented.

[0160] In this embodiment, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, USB flash drive, mobile hard disk, disk or optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0161] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0162] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0163] In the embodiments disclosed in the present application, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the apparatus / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0164] One point, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, which may be electrical, mechanical or other forms.

[0165] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0166] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0167] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. Abnormal behavior security warning system based on multi-video image analysis, characterized in that: include: Path construction module: The path construction module constructs a security path based on video image data, marks abnormal data of personnel behavior in the security path, and builds an abnormal behavior comparison library; Feature matching module: The feature matching module is used to build a security matching model, conduct quantitative analysis on abnormal personnel behavior data to obtain a comprehensive risk index, and identify abnormal security personnel within the security path; Personnel determination module: The personnel determination module is used to divide the security path into multiple security levels, and to make risk judgments on the abnormal behaviors of abnormal security personnel to obtain risk judgment values, and determine security risk personnel; Personnel analysis module: The personnel analysis module is used to determine whether the risk assessment value of security risk personnel is continuously increasing. If it is not continuously increasing, the risk level of security risk personnel in each security level is quantitatively analyzed to determine the priority investigation path; Regional analysis module: Based on the priority screening path, the regional analysis module is used to perform cluster analysis on the distribution of security risk personnel at each security level, obtain the dangerous areas of the priority screening path, and mark the security risk personnel in the dangerous area as high-risk security personnel; Model optimization module: The model optimization module is used to verify high-risk security personnel and determine whether the personnel are mislabeled. If there is a mislabeling, the security comparison model is adjusted and optimized.

2. The abnormal behavior security warning system based on multi-video image analysis according to claim 1 is characterized in that: The method for determining the abnormal security personnel in the security path is as follows: Video image data obtained based on a video acquisition device; If a person shows abnormal behavior in the video image data, obtain the number of times the person is marked as having abnormal behavior by the video acquisition device during the monitoring period, marked as Cs; By formula: Obtain the personnel's abnormal behavior frequency AF, where T is the length of the monitoring cycle; By formula: Get the abnormal behavior persistence index ADTI of the personnel, where Cx k is the duration of each abnormal behavior, k is the number of each abnormal behavior; The number of types of abnormal behaviors of personnel and the number of abnormal behaviors in the abnormal behavior comparison database are processed by ratio to obtain the abnormal diversity ratio, which is recorded as Yc; And process the abnormal behavior frequency AF, the abnormal behavior persistence index ADTI of personnel, and the abnormal diversity ratio Yc to obtain a comprehensive risk index; If the comprehensive risk index of a person in the video image data is higher than the risk threshold, the person will be marked as a security abnormality person.

3. The abnormal behavior security early warning system based on multi-video image analysis according to claim 1 is characterized in that: The security risk personnel are determined as follows: And make risk assessment on abnormal behaviors of abnormal security personnel to obtain risk assessment value; If the risk determination value of the security abnormal person is higher than the risk determination threshold, the security abnormal person will be marked as a security risk person.

4. The abnormal behavior security early warning system based on multi-video image analysis according to claim 3 is characterized in that: The risk determination value is obtained in the following manner: Divide the security path into multiple paths of different levels to obtain a security level path; The number of abnormal behaviors in each security level is compared with the number of all behaviors to obtain the abnormal behavior ratio Pf; Based on the first appearance probability Pf of security abnormal personnel, abnormal behavior frequency AF, abnormal behavior index ADTI, and abnormal diversity ratio Yc within each security level, a conditional probability table is constructed; Based on the conditional probability table, using Bayesian theorem, calculate the posterior probability P of security abnormal personnel under different security level paths i , where i represents the security path level, and the value range of i is [1,n]; Where n is the total number of security levels; The posterior probability within each security level path is multiplied by the comprehensive risk index to obtain the risk judgment value.

5. The abnormal behavior security early warning system based on multi-video image analysis according to claim 1 is characterized in that: The method for obtaining the priority screening path is as follows: Obtain the number of risk personnel and the total number of personnel within each security level; The ratio of the number of interval risk personnel to the number of all personnel is processed to obtain the interval personnel ratio; Obtain the risk determination values ​​of all interval risk personnel within each security level, sum and average the risk determination values ​​of all interval risk personnel, and obtain the interval risk mean; Multiply the interval personnel ratio and the interval risk mean to obtain the interval risk value; Obtain the interval risk value within each security level path, sort the security level paths in descending order of risk value, and determine the priority inspection path.

6. The abnormal behavior security early warning system based on multi-video image analysis according to claim 5 is characterized in that: The determination method of the interval risk personnel is as follows: Obtain the risk determination value of the security risk personnel in each security level path, use the risk determination value in each security level path as a data element, and construct a security risk personnel sequence; Mark the security risk personnel sequence as R i , i is the number of each security level path, and the value range of i is [1,n]; Calculate the first-order difference of adjacent risk judgment values ​​in the security risk personnel sequence R i ; like R i ≤0, it is considered that the security risk personnel are within the security level path, the risk judgment value changes discontinuously, and the security risk personnel are marked as interval risk personnel.

7. The abnormal behavior security early warning system based on multi-video image analysis according to claim 1 is characterized in that: The method of obtaining the security high-risk personnel is as follows: Based on the determination of the dangerous areas within the priority inspection path, the security risk personnel in the dangerous areas are marked as high-risk security personnel.

8. The abnormal behavior security early warning system based on multi-video image analysis according to claim 7 is characterized in that: The method for obtaining the dangerous area in the priority screening path is as follows: Obtain the risk assessment values ​​of all security risk personnel in the priority screening path and build a risk personnel data group; Sorting the risk determination values ​​of the security risk personnel in the risk personnel data group, obtaining the security risk personnel corresponding to the maximum risk determination value as the central risk personnel; Taking the coordinates of the central risk personnel as the origin, establish the security path coordinate system; During the monitoring period, the security path coordinate system and the coordinates of all security risk personnel are obtained in real time, and the coordinates of each security risk personnel and the risk judgment value are stored as a risk group in the dynamic risk set; The neighborhood radius ε is calculated through the density clustering algorithm, and the regional judgment model is established to determine the dangerous areas within the priority inspection path.

9. The abnormal behavior security early warning system based on multi-video image analysis according to claim 8 is characterized in that: The neighborhood radius ε is obtained as follows: Obtain the variances σ1 and σ2 of the risk judgment values ​​of all security risk personnel in two adjacent monitoring periods in the dynamic risk set; Calculate the variance change of the risk judgment value of two adjacent monitoring periods σ, σ=|σ1-σ2|; Get the x-axis range of the security path coordinates in the dynamic risk set [x min ,x max ]、y-axis range [y min ,y max ]; By formula: Get the spatial range parameter S; Get the change of the spatial range parameter S between two adjacent monitoring periods, marked as S; Obtain the dynamic risk set, the moving speed V of each security risk personnel in adjacent monitoring cycles, and the rate of change Bh of the number of security risk personnel; By formula: Get the neighborhood radius ε, where α1, α2, and α3 are preset proportional coefficients; Among them, j represents the number of each security risk personnel in the dynamic risk set, and the value range of j is [1,m].

10. The abnormal behavior security early warning system based on multi-video image analysis according to claim 1, characterized in that: The security comparison model is adjusted and optimized in the following manner: Obtain video images of mislabeled persons and missed persons in the security comparison model in the feature comparison module; The abnormal behaviors in the video images of the omitted persons are stored in an abnormal behavior comparison library; The number of mislabeled personnel is compared with the number of security abnormal personnel to obtain the mislabeled personnel ratio; Obtain the number of missed persons, and perform ratio processing on the number of missed persons and the number of normal persons to obtain the missed person ratio; The weighted sum of the mislabeled personnel ratio and the omitted personnel ratio is processed to obtain the model training value; The number of training rounds of the model algorithm in the security comparison model is multiplied by the model training value to obtain the model adjustment value; Based on the model adjustment value, the number of training times of the model algorithm of the security comparison model is increased to achieve optimization adjustment of the security comparison model.

Images