A smart campus one-card pre-warning method based on video monitoring

Abstract

The application discloses a kind of based on video monitoring intelligent campus one-card early warning method, the method includes: obtaining the continuous frame image of gate machine card swiping personnel in preset time window collected by campus monitoring equipment;Using multimodal time series analysis mechanism, generate image quality change curve and behavior trajectory change curve, construct collaborative risk change curve, determine abnormal point using behavior trajectory change curve and preset standard trajectory change curve, segment the collaborative risk change curve, obtain multiple segmentation curves corresponding to collaborative risk characteristic values;Extract the segmentation curve with minimum collaborative risk characteristic value as target curve, determine face recognition image by analyzing the light direction of corresponding continuous frame image;Adaptive correction is carried out to face recognition image to obtain target image, face recognition is carried out to target image to obtain the matching result of card swiping personnel, generate the early warning instruction of gate machine and execute.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and in particular to a smart campus one-card pre-warning method based on video monitoring. BACKGROUND

[0002] In the process of people's life and growth, campus life is an essential part of everyone. As a vulnerable group, students live in a collective in the campus, and the safety of the campus is extremely important. In order to prevent non-related personnel from entering the school, most schools have adopted a card-based card entry system based on cost. However, simply using a one-card to enter the campus has a high security risk, for example, a non-school personnel can complete the card entry by illegally holding a one-card of a school personnel, for example, at a time when the personnel enters and exits the campus is relatively dense, because the gate machine has a certain mechanical delay, which leads to the premeditated mixing of non-card personnel, and other related safety factors. The purpose of the one-card pre-warning assisted by the campus monitoring video is to prevent the illegal card holder from entering the campus through the campus access control by using the one-card, so as to identify the card holder by using the monitoring video, and to determine the pre-warning condition according to the actual situation, so as to pre-warn the illegal card holder when entering the campus. Most existing face recognition technologies are for recognizing face images under the constraint condition of high-definition cameras, but because of the limitations of high-definition cameras under the constraint condition, that is, the cost is too high, it does not meet the needs of large-scale population-dense places, and other factors, leading to the fact that the face recognition technology of the campus gate high-definition camera is not popular. Therefore, the monitoring video collected by the monitoring device at the entrance of the campus is used to assist the one-card entry. SUMMARY

[0003] The present application provides a smart campus one-card pre-warning method based on video monitoring, which improves the accuracy and safety of the campus one-card pre-warning.

[0004] The present application provides a smart campus one-card pre-warning method based on video monitoring, which includes:

[0005] S101, obtaining the continuous frame images of the gate card personnel in a preset time window collected by the campus monitoring device;

[0006] S102, generating an image quality change curve and a behavior trajectory change curve by using a preset multi-modal time sequence analysis mechanism;

[0007] S103, constructing a cooperative risk change curve based on the image quality change curve and the behavior trajectory change curve, determining an abnormal point by using the behavior trajectory change curve and a preset standard trajectory change curve, segmenting the cooperative risk change curve, and obtaining a plurality of segmented curves corresponding to cooperative risk characteristic values;

[0008] S104, extract the segmentation curve with the minimum collaborative risk feature value as the target curve, and perform light direction analysis on the continuous frame images corresponding to the target curve to determine the face recognition image;

[0009] S105, performing adaptive correction on the face recognition image to obtain a target image, performing face recognition on the target image to obtain a matching result of the card swiping personnel, generating a warning instruction of the gate and executing the warning instruction.

[0010] Preferably, the preset time window is set as a time period formed between the time when the previous card swiping personnel passes through the gate and the time when the current card swiping personnel completes the card swiping action.

[0011] Preferably, the preset multi-modal time sequence analysis mechanism specifically includes:

[0012] A1, performing gray processing on the continuous frame images, based on each frame image, obtaining a quality value of the frame image according to the gray distribution information and the key occlusion degree of the target face region, fitting the quality value in the time sequence to obtain an image quality change curve Q(t); the target face region is set as the face image of the card swiping personnel in the frame image;

[0013] A2, based on the continuous frame images, performing center point tracking on the target face region to obtain a position vector of the center point relative to the monitoring device, including a distance value and an angle value, and quantifying the position vector to obtain an attitude change value of each frame image relative to the monitoring device;

[0014] A3, fitting the attitude change value in the time sequence to obtain a behavior trajectory change curve B(t).

[0015] Preferably, the quality value of the frame image is calculated according to the following formula:

[0016]

[0017] wherein, Qi is the quality value of the i-th frame image, σ is the gray variance value of the target face region, σ0 is a preset standard variance value, μ is the gray mean value of the target face region, μ0 is a preset standard gray mean value, β is a preset gray sensitivity coefficient, which is determined according to expert experience and actual situation; N is the number of key points detected in the target face region, N0 is the total number of key points of a standard face image, and α and β are preset weight values, respectively used to reflect the influence degree of the gray distribution information and the key occlusion degree on the image quality, and are set according to actual demand and expert experience.

[0018] Preferably, in the A2, the position vector is trajectory quantized to obtain a pose value of each frame of image relative to the monitoring device, including:

[0019] B1, based on the target face region center point of each frame of image, fusing the horizontal distance and the vertical distance thereof relative to the monitoring device to obtain a distance value;

[0020] B2, obtaining an angle deviation value of the center point relative to the monitoring device, calculating a change rate of the position vector of the current frame of image relative to the position vector of the previous frame of image, denoted as ;

[0021] B3, according to the change rate of the center point position vector of the current frame of image, calculating to obtain a pose change value:

[0022]

[0023] wherein, the pose change value is, the distance change rate is, the angle deviation change rate is, the preset weight coefficient is used for adjusting the influence degree of the distance change rate and the angle deviation change rate.

[0024] Preferably, the method for constructing the cooperative risk change curve specifically includes:

[0025] S201, based on the quality value and the pose change value of the corresponding time point of the continuous frames of image on the image quality change curve Q(t) and the behavior trajectory change curve B(t), obtaining the risk value of the corresponding time point;

[0026] S202, using the risk values corresponding to the continuous frames of image to perform fitting on the time sequence to generate the cooperative risk change curve.

[0027] Preferably, the acquisition method of the preset standard trajectory change curve is:

[0028] Collecting the behavior trajectory change curves of a large number of card swiping personnel who have successfully passed the gate machine in history, performing clustering analysis on all the behavior trajectory change curves to obtain a plurality of clusters, taking the cluster containing the largest number of behavior trajectory change curves as a target class, and taking the center behavior trajectory change curve of all the behavior trajectory change curves in the target class as the standard trajectory change curve.

[0029] Preferably, in the S103, the behavior trajectory change curve and the preset standard trajectory change curve are used to determine the abnormal points, and the cooperative risk change curve is segmented to obtain a plurality of segmentation curves corresponding to the cooperative risk characteristic values, including:

[0030] C1, obtaining the intersection of the behavior trajectory change curve and the standard trajectory change curve as an abnormal point;

[0031] C2, sequentially segmenting the collaborative risk change curve according to the time points corresponding to the abnormal points until all the time points corresponding to the abnormal points are traversed, to obtain all the segmented curves;

[0032] C3, obtaining the average value of the risk value of each segmented curve as the collaborative risk characteristic value of the segmented curve.

[0033] Preferably, in the S104, the determination manner of the face recognition image is:

[0034] determining the light direction of each frame of image to obtain the light dominant direction, and taking the target face region of the frame image with the maximum quality value of the light dominant direction and the image as the face recognition image;

[0035] The light dominant direction is determined as follows: obtaining the light direction of each frame of image in the continuous frame of image, counting the occurrence frequency of the light direction, and selecting the light direction with the highest occurrence frequency as the light dominant direction.

[0036] Preferably, the S105 further comprises:

[0037] S401, calculating the deviation value of the behavior trajectory change curve and the preset standard trajectory change curve;

[0038] S402, obtaining the card swiping pressure value and the sensing turbulence degree of the sensing position of the gate machine in the time window when the card swiping is sensed, and combining the deviation value to calculate the neighborhood turbulence factor of the card swiping personnel;

[0039] S403, generating a warning instruction of the gate machine according to the matching result and the neighborhood turbulence factor:

[0040] If the matching result is successful and the neighborhood turbulence factor is less than the preset turbulence threshold, the gate machine is controlled to be opened;

[0041] If the matching result is successful and the neighborhood turbulence factor is greater than the preset turbulence threshold, a warning voice is played and a campus security personnel is notified to pay attention to the gate machine, and the gate machine is controlled to be opened;

[0042] If the matching result fails, the gate machine is controlled to be closed, and the security personnel is notified.

[0043] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0044] By combining the campus monitoring video and the card swiping information, the identity and behavior of the card swiping personnel are verified in cooperation, illegal card holding of non-campus personnel into the campus is effectively prevented, and the campus safety is ensured; the multi-modal time sequence analysis mechanism is used to generate an image quality change curve and a behavior trajectory change curve, a cooperative risk change curve is constructed, an abnormal point is determined by analyzing the relationship between the curves, the cooperative risk change curve is cut, and the image corresponding to the curve segment with the minimum risk is selected for subsequent image enhancement processing, that is, the selection of the best face recognition image according to the risk value of the cooperative evaluation of the image quality and the dynamic image mapping behavior change characteristics can further improve the accuracy of face recognition, and then the intelligent early warning of the card swiping risk of the card swiping personnel is realized in combination with the neighborhood turbulence factor of the card swiping personnel; the warning instruction of the gate is generated according to the matching result and the neighborhood turbulence factor, different gate control strategies are adopted for card swiping personnel in different risk situations, and the intelligent level of gate management is improved. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 It is a flowchart of the intelligent campus card pre-warning method based on video monitoring of the embodiment of the application. DETAILED DESCRIPTION

[0046] In order to facilitate the understanding of the present application, the present application will be described in more detail below with reference to the relevant drawings; the preferred embodiments of the present application are shown in the drawings, but the present application can be realized in many different forms, and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0047] It should be noted that the terms "vertical", "horizontal", "up", "down", "left", "right" and similar expressions used herein are only for illustrative purposes and do not represent the only embodiment.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as understood by those skilled in the art to which the present application belongs; the terms used in the specification of the present application are only for the purpose of describing the specific embodiments and are not intended to limit the present application; the term "and / or" used herein includes any and all combinations of one or more related listed items.

[0049] Embodiment one: Figure 1 It is a flowchart of the intelligent campus card pre-warning method based on video monitoring of the embodiment of the application.

[0050] As Figure 1 shown, an intelligent campus card pre-warning method based on video monitoring includes the following steps:

[0051] S101, obtain the continuous frame images of the gate card personnel collected by the campus monitoring device in a preset time window, and the preset time window is set as a time period from the time when the previous card personnel passes through the gate to the time when the current card personnel completes the card action.

[0052] The campus monitoring device is arranged directly above the gate, and is used for collecting real-time video information in front of the gate.

[0053] S102, generate an image quality change curve and a behavior trajectory change curve by using a preset multi-modal time sequence analysis mechanism.

[0054] In some embodiments, the preset multi-modal time sequence analysis mechanism specifically includes:

[0055] A1, performing gray processing on the continuous frame images, and based on each frame image, obtaining a quality value of the frame image according to the gray distribution information of the target face region and the key occlusion degree, fitting the quality values in the time sequence to obtain an image quality change curve Q(t).

[0056] Specifically, the quality value of the frame image is calculated according to the following formula:

[0057]

[0058] wherein, is the quality value of the i-th frame image, is the gray variance value of the target face region, used for reflecting the face illumination uniformity, is a preset standard variance value, is the gray mean value of the target face region, is a preset standard gray mean value, is a preset gray sensitivity coefficient, which is determined according to expert experience and actual situation, for example, set to 50; the preset standard variance value and the standard gray mean value are determined according to the current time attribute and the environment attribute, and each time attribute and environment attribute (for example, daytime and sunny weather, evening and cloudy weather, etc.) is provided with a gray variance value and a standard gray mean value of a standard face image (in the case of uniform illumination and no occlusion), which are set according to actual situation and expert experience, and the present application will not be repeated here; is the number of key points detected in the target face region, is the total number of key points of the standard face image, and are preset weight values, respectively used for reflecting the influence degree of the gray distribution information and the key occlusion degree on the image quality, which are set according to actual demand and expert experience, and the sum of the two is 1, for example, is 0.4, is 0.6.

[0059] It should be noted that the target face region is set as the face image of the card swiping personnel in the frame image, and the acquisition method can utilize an existing face extraction model to frame and acquire. The acquisition method can utilize an open source face detection algorithm (such as a face detection function in OpenCV) or a deep learning model (such as MTCNN, RetinaFace, etc.), and the present application does not repeat the details. The card swiping personnel is determined as the personnel closest to the gate and the monitoring device. In addition, the key point detection method in the target face region is set as an image recognition model trained by a large number of labeled key point face images, and the key points are set as the facial features of the face image, which are used to reflect the completeness of the face image and the degree of face occlusion. For example, the key points include but are not limited to the left eye, the right eye, the nose tip, the upper lip, and the lower lip.

[0060] A2, based on the continuous frame images, the center point of the target face region is tracked to obtain a position vector of the center point relative to the monitoring device, including a distance value and an angle value, which is expressed as: The position vector is quantized to obtain a posture change value of each frame image relative to the monitoring device.

[0061] wherein, represents the position vector of the center point of the i-th frame image, represents the distance value, represents the angle value.

[0062] Specifically, the position vector is quantized to obtain a posture value of each frame image relative to the monitoring device, including:

[0063] B1, based on the center point of the target face region of each frame image, the horizontal distance and the vertical distance thereof relative to the monitoring device are fused to obtain the distance value.

[0064] wherein, the difference between the horizontal coordinate of the center point in the frame image and the horizontal coordinate of the monitoring device (which needs to be calibrated and converted according to the actual scene and the internal parameters of the monitoring device, which can refer to the conversion between the related image and the actual coordinate, and the present application does not repeat the details) is determined as the horizontal distance, and the difference between the vertical coordinate of the center point in the frame image and the vertical coordinate of the monitoring device (which also needs to be calibrated and converted according to the actual scene) is determined as the vertical distance; the distance value is calculated by using the Pythagorean theorem.

[0065] B2, the angle value of the center point relative to the monitoring device (the direction angle of the center point relative to the monitoring device, usually in radians or degrees, reflecting the offset degree of the position of the center point in the frame image relative to the vertical direction of the monitoring device) is obtained, and the change rate of the position vector of the current frame image relative to the position vector of the previous frame image (the absolute value of the difference of the position vector and the ratio of the previous frame position vector) is calculated, which is denoted as .

[0066] B3, calculate the posture change value according to the rate of change of the center point position vector of the current frame image:

[0067]

[0068] wherein, is the posture change value, is the distance change rate, is the angle change rate, is a preset weight coefficient for adjusting the influence degree of the distance change rate and the angle change rate. It can be understood that the angle change rate and the distance change rate cooperatively determine the posture change value. If the angle change rate is larger, it means that the action amplitude of the card swiping person is larger, and the distance change rate is increased on the basis of the distance change rate, further improving the posture change value. In this way, the action amplitude of the card swiping person can be avoided to be ignored. Because even if the forward speed of the card swiping person is normal, but if there is left and right shaking or pushing scene, it means that there is hidden risk, and there may be a certain cooperative relationship between the image quality.

[0069] A3, fit the posture change value in the time sequence to obtain the behavior trajectory change curve B(t).

[0070] S103, based on the image quality change curve and the behavior trajectory change curve, a cooperative risk change curve is constructed. The behavior trajectory change curve and the preset standard trajectory change curve are used to determine the abnormal points, and the cooperative risk change curve is segmented to obtain a plurality of segmentation curves, each segmentation curve corresponding to a cooperative risk feature value. The cooperative risk feature value is determined as the risk value mean value of the segmentation curve.

[0071] In some embodiments, the method for constructing the cooperative risk change curve specifically comprises:

[0072] S201, based on the quality value and the posture change value of the continuous frame images corresponding to the time points on the image quality change curve Q(t) and the behavior trajectory change curve B(t), the risk value corresponding to the time points is calculated.

[0073] Specifically, the risk value corresponding to the frame image is calculated according to the following formula:

[0074]

[0075] wherein, is the risk value, is the posture change value, is the image quality value.

[0076] S202, using the risk values corresponding to the continuous frame images, fitting in the time sequence to generate a cooperative risk change curve.

[0077] Specifically, the preset standard trajectory change curve is obtained in the following manner:

[0078] A large number of behavior trajectory change curves of card swiping personnel who have successfully passed the gate in history are collected, clustering analysis is performed on all the behavior trajectory change curves to obtain a plurality of clusters, a cluster containing the largest number of behavior trajectory change curves is taken as a target class, and a central behavior trajectory change curve (average curve) of all the behavior trajectory change curves in the target class is taken as the standard trajectory change curve.

[0079] The curve clustering can adopt a dynamic time warping (DTW) algorithm combined with a K-means clustering algorithm. The DTW algorithm is used to calculate the similarity between two curves, and the K-means clustering algorithm divides the curves into different clusters according to the similarity. The specific steps are as follows: 1. Initialize K cluster centers, and randomly select K behavior trajectory change curves as initial centers; 2. For each behavior trajectory change curve, calculate the DTW distance from each cluster center, and assign it to the nearest cluster; 3. Recalculate the center of each cluster, that is, take the average value of all curves in the cluster; 4. Repeat steps 2 and 3 until the cluster center no longer changes or the preset iteration number is reached; 5. Select the cluster containing the largest number of curves as the target class, and calculate the average value of all curves in the target class to obtain the standard trajectory change curve.

[0080] Specifically, the behavior trajectory change curve and the preset standard trajectory change curve are used to determine the abnormal points, the collaborative risk change curve is segmented to obtain a plurality of segmented curves corresponding to the collaborative risk feature values, including:

[0081] C1. Obtain the intersection of the behavior trajectory change curve and the standard trajectory change curve as the abnormal point.

[0082] C2. Segment the collaborative risk change curve according to the time points corresponding to the abnormal points in sequence until all the time points corresponding to the abnormal points are traversed to obtain all the segmented curves.

[0083] C3. Based on each segmented curve, obtain the average value of the risk value thereof as the collaborative risk feature value of the segmented curve.

[0084] S104. Extract the segmented curve with the smallest collaborative risk feature value as the target curve, and perform light direction analysis on the continuous frame images corresponding to the target curve to determine the face recognition image.

[0085] The determination of the face recognition image is as follows: determine the light direction of each frame image to obtain the light dominant direction, and take the target face region of the frame image with the largest quality value in the light dominant direction as the face recognition image.

[0086] It should be noted that the smaller the cooperative risk characteristic value is, the higher the image quality is and the smaller the posture change is, that is, the more stable the behavior track of the person is, which also reflects the stability of the image collection and the high quality, and also reflects the orderliness of the card swiping track of the card swiping person.

[0087] The method for analyzing the illumination direction of the image can refer to any one of the related prior art, and the present application will not be described here.

[0088] As an example, the target face region of each frame image is subjected to grayscale processing, and the gradient vector of each pixel is obtained by using a gradient operator (such as a Sobel operator, a Prewitt operator, etc.), and the gradient vector g is a two-dimensional vector composed of the gradient components of the pixel in the horizontal and vertical directions, that is, g=(gx, gy The horizontal gradient component gx represents the rate of change of the pixel value in the horizontal direction (i.e., the X-axis direction), and the vertical gradient component gy represents the rate of change of the pixel value in the vertical direction (i.e., the Y-axis direction), the size (i.e., the module length) of the gradient vector reflects the degree of change of the pixel value, and the direction of the gradient vector indicates the direction in which the pixel value changes fastest, and in the illumination analysis, the gradient vector can help identify the edges, textures and illumination changes in the image; a plurality of reference directions (such as 0°, 45°, 90°, 135°, etc.) are defined, and for each reference direction, the sum of the projections of the gradient vectors of all pixels in the direction is calculated to obtain the fitting vector in the reference direction, and in the specific calculation, the gradient vector of each pixel can be decomposed into the component in the reference direction, and the components of all pixels are added to obtain the length and direction of the fitting vector, assuming that there are N pixels, the gradient vector of the i-th pixel is gi=(gix, giy), and the reference direction is θ, then the projection of the pixel in the reference direction is The sum of the projections of all pixels in the reference direction is The size of the sum of the projections reflects the overall change degree of the gradient vectors of all pixels in the reference direction θ, and if the value is large, it means that the reference direction θ is dominant in the image.

[0089] In some embodiments, the continuous frame images corresponding to the target curve are subjected to illumination direction analysis to obtain the dominant illumination direction, which specifically includes: obtaining the illumination direction of each frame image in the continuous frame images, counting the occurrence frequency of the illumination direction, and selecting the illumination direction with the highest occurrence frequency as the dominant illumination direction.

[0090] S105, adaptively correct the face recognition image according to the light dominant direction (for example, use adaptive gamma correction algorithm to correct the light, improve the image quality) to obtain a target image, use the target image to perform face recognition to obtain the matching result of the card swiping personnel, generate a warning instruction of the gate and execute.

[0091] Wherein, the face recognition is specifically: inputting the target image into the campus face library for matching, if the matched face identity information is consistent with the card swiping information, the matching is successful; if the matched face identity information is inconsistent with the card swiping information or no face identity information is matched, the matching fails.

[0092] In some embodiments, step S105 further comprises:

[0093] S401, calculate the deviation value of the behavior trajectory change curve and the preset standard trajectory change curve:

[0094]

[0095] Wherein, f is the deviation value, t0 is the start time of the preset time window, t1 is the end time of the preset time window, is the behavior trajectory change curve function, is the standard trajectory change curve function.

[0096] In actual calculation, the discretization method can also be used, the preset time window is divided into several small time periods, the sum of the absolute values of the difference between the behavior trajectory change curve and the standard trajectory change curve in each time period is calculated, and then the deviation value is obtained by dividing the total number of time periods.

[0097] Therefore, through the analysis of the posture change value, the passive abnormal behavior of the card swiping personnel can be reflected, such as forced left and right swing, push and pull, etc.

[0098] S402, obtain the card swiping pressure value and the disturbance degree of the sensing position in the time window when the gate senses the card swiping (the average of the pressure value and the change rate of the sensing position at all adjacent time points in the time window is taken, and the weighted sum of the average change rate of the pressure value and the sensing position is obtained to obtain the disturbance degree), combined with the deviation value, the neighborhood disturbance factor of the card swiping personnel is calculated.

[0099] Wherein, the gate sensing card swiping area is provided with a pressure sensor and a position monitoring sensor, which are used to collect the card swiping pressure value and the sensing position information in real time.

[0100] Wherein, the neighborhood disturbance factor of the card swiping personnel is calculated according to the following formula:

[0101]

[0102] Wherein, k is a neighborhood turbulence factor, f is a deviation value, w is an induction turbulence degree, D is a trust factor, 、 are preset weight factors respectively, and are used to represent the influence degree of the deviation value and the turbulence degree on the neighborhood turbulence factor, the range is between 0 and 1, and is 1, which is set according to actual situation and scene specific requirements, for example, is 0.7, is 0.3.

[0103] It should be noted that the trust factor is used to reflect the correlation between the deviation value and the induction turbulence degree of the current card swiping personnel and the environment around him, and the trust factor can be determined according to other face regions in a preset field range of the center point of the target face region in the continuous frame image. The trust factor is determined by using the change rate of the distance value of the other face regions relative to the center point of the target face region in the continuous frame image. The greater the change rate is, the greater the trust factor is. If there is no other face region in the preset neighborhood range, the trust factor is determined as 0. Wherein, the preset neighborhood range is set as a range with the center point as the center and the preset distance value as the radius. The preset distance is set according to expert experience and actual situation (the distance value between the corresponding face region centers in the images of historical crowded or pushing personnel can be collected to set the distance value), which is used to measure the safety distance value between people.

[0104] It should be noted that when calculating the neighborhood turbulence factor, the deviation value, the induction turbulence degree and the trust factor need to be normalized, or the neighborhood turbulence factor can be directly normalized. The normalization method can refer to related prior art, which will not be described here.

[0105] S403, according to the matching result and the neighborhood turbulence factor, a warning instruction of the gate is generated, specifically:

[0106] If the matching result is successful and the neighborhood turbulence factor is less than a preset turbulence threshold, the gate is controlled to be opened. The preset turbulence threshold is used to measure the neighborhood turbulence degree, which is set according to actual situation and historical experience;

[0107] If the matching result is successful and the neighborhood turbulence factor is greater than the preset turbulence threshold, a warning voice is played and the campus security personnel is notified to pay attention to the gate continuously, and the gate is controlled to be opened. The warning voice is used for voice warning to prompt the personnel at the gate to disperse and maintain order;

[0108] If the matching result fails, the gate is controlled to be closed, and the security personnel is notified.

[0109] In summary, the image quality and behavior trajectory two-dimensional time series data are fused to construct a collaborative risk curve, breaking the limitations of single modal analysis; the quality is quantified by gray distribution variance (illumination uniformity) and key point occlusion rate, overcoming the defects of traditional methods relying only on clarity; the weighted fusion of the distance change rate and the angle change rate of the target face center point relative to the monitoring device is introduced to accurately capture minor abnormal postures; the intersection (abnormal point) of the behavior curve and the standard curve is used to segment the risk curve, realize local risk focusing, select the segmentation section with the minimum collaborative risk feature value (high quality + low posture change), and preferentially extract the images in this period for recognition, which significantly improves the face matching accuracy, solves the recognition degradation problem in complex lighting scenes through light dominant analysis, and reduces the false rejection rate; the neighborhood turbulence factor fuses three parameters of behavior deviation value, gate pressure turbulence, and surrounding personnel density to realize early warning of tailgating or abnormal pushing and other pre-planned intrusions.

[0110] Traditional static snapshots fail in uneven lighting and local occlusion (such as waving hands or bowing heads), and the optimal frame is dynamically selected through the time series quality curve; in the case of fraud or intrusion, the person may maintain normal speed, but there is a slight pushing and shaking, and the behavior trajectory change rate cooperates with the image quality fluctuation to achieve accurate capture; during peak hours, tailgating and pushing are common, and the neighborhood turbulence factor fuses gate pressure sensor data and surrounding personnel density to trigger differentiated gate early warning.

[0111] The technical solutions in the embodiments of the application have at least the following technical effects or advantages:

[0112] By combining campus monitoring video and card swiping information, the identity and behavior of the card swiping personnel are verified collaboratively, effectively preventing non-campus personnel from illegally entering the campus with a card, and ensuring campus safety; the multi-modal time series analysis mechanism is used to generate image quality change curves and behavior trajectory change curves, construct a collaborative risk change curve, determine abnormal points by analyzing the relationship between the curves, and cut the collaborative risk change curve to select the image corresponding to the curve segment with the smallest risk for subsequent image enhancement processing, i.e., selecting the best face recognition image according to the risk value of the collaborative evaluation of image quality and dynamic image mapping behavior change characteristics, which can further improve the accuracy of face recognition, and then combining the neighborhood turbulence factor of the card swiping personnel, the intelligent early warning of card swiping risk of the card swiping personnel is realized; according to the matching result and the neighborhood turbulence factor, the warning instruction of the gate is generated, different gate control strategies are adopted for card swiping personnel with different risks, and the intelligent level of gate management is improved.

[0113] Compared with simply relying on a card to swipe into school, increasing video monitoring and face recognition and other means greatly improves the difficulty of illegal card holders entering the campus, effectively reducing the risk of campus safety; By comprehensively considering image quality, behavior trajectory, card swiping pressure and sensing position and other factors, the neighborhood turbulence factor is calculated, which can more accurately judge the risk of the card swiping personnel and reduce false positives and false negatives; Not dependent on high-definition cameras, using existing monitoring equipment at the entrance of the campus can be realized, the cost is low, and it is suitable for large-scale population-dense places in the campus.

[0114] Focus on the face image analysis of the current card swiping personnel of the gate, the image quality and behavior change, without detailed image and behavior analysis of other personnel behind the card swiping personnel, can more efficiently analyze the turbulence factor of the card swiping personnel when swiping the card, reflect the risk situation of the current card swiping personnel passing through, reduce unnecessary image and behavior analysis, reduce the calculation complexity, and improve the real-time performance of the system; According to the size of the turbulence factor, a differentiated control strategy of the gate can be realized, the personnel with low risk are quickly released, the personnel with high risk are warned and focused on, and the safety and management efficiency of the campus are improved;

[0115] In summary, through the continuous frame images of the gate card swiping personnel collected by the campus monitoring equipment, the multi-modal time sequence analysis mechanism is used to generate the image quality change curve and the behavior trajectory change curve, the cooperative risk change curve is constructed, and then the light direction analysis, face recognition and gate warning instruction generation are performed, to guarantee the safety and accuracy of the campus gate passage. Realize the comprehensive monitoring and early warning of the gate card swiping personnel, can effectively identify the abnormal behavior that may exist in the card swiping process, such as being accompanied by a stranger to break in, etc., and provide strong support for the safety management of the campus.

[0116] The above only describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1.A method for early warning of a smart campus one-card based on video monitoring, characterized in that, The method comprises the following steps: S101, obtaining continuous frame images of a gate card personnel collected by a campus monitoring device within a preset time window; S102, generating an image quality change curve and a behavior trajectory change curve by using a preset multi-modal time sequence analysis mechanism, specifically comprising: A1, the gray scale of the continuous frame image is processed, based on each frame image, the quality value of the frame image is obtained according to the gray scale distribution information and the key shielding degree of the target face area, the quality value is fitted on the time sequence, and the image quality change curve Q(t) is obtained; The target face area is set as the face image of the card swiping personnel in the frame image; The quality value of the frame image is calculated according to the following formula: , Qi is the quality value of the i-th frame image, is the gray scale variance value of the target face area, is the preset standard variance value, is the gray scale mean value of the target face area, is the preset standard gray scale mean value, is the preset gray scale sensitive coefficient, which is determined according to expert experience and actual situation; is the number of key points detected in the target face area, is the total number of key points of the standard face image, and are preset weight values, respectively used for reflecting the influence degree of the gray scale distribution information and the key shielding degree on the image quality, and are set according to actual demand and expert experience. A2, based on the continuous frame image, the center point tracking is performed on the target face region to obtain a position vector of the center point relative to the monitoring device, including a distance value and an angle value, and the position vector is quantified to obtain a posture change value of each frame image relative to the monitoring device; B1, based on the center point of the target face region of each frame image, the horizontal distance and the vertical distance of the center point relative to the monitoring device are fused to obtain the distance value; B2, the angle value of the center point relative to the monitoring device is obtained, and the change rate of the position vector of the current frame image relative to the position vector of the last frame image is calculated, denoted as ; B3, according to the change rate of the center point position vector of the current frame image, the posture change value is calculated: , is the posture change value, is the distance change rate, is the angle change rate, is a preset weight coefficient, used for adjusting the influence degree of the distance change rate and the angle change rate; A3, fitting the posture change value on the time sequence to obtain the behavior trajectory change curve B(t); S103, constructing a collaborative risk change curve based on the image quality change curve and the behavior trajectory change curve, determining abnormal points by using the behavior trajectory change curve and a preset standard trajectory change curve, segmenting the collaborative risk change curve to obtain a plurality of segmented curves corresponding to collaborative risk feature values; S104, extracting the segmented curve with the smallest collaborative risk feature value as a target curve, and determining a face recognition image by performing light direction analysis on the continuous frame images corresponding to the target curve; S105, performing adaptive correction on the face recognition image to obtain a target image, performing face recognition on the target image to obtain a matching result of the card personnel, generating a warning instruction of the gate and executing the warning instruction. 2.The video monitoring based early warning method for smart campus one-card system according to claim 1, wherein, The preset time window is set as a time period from the time when the previous card personnel passes through the gate to the time when the current card personnel completes the card action. 3.The video monitoring based early warning method for smart campus one-card system according to claim 2, characterized in that, The method for constructing the collaborative risk change curve specifically comprises: S201, obtaining a risk value of a corresponding time point based on the quality value and the posture change value of the corresponding time point of the continuous frame images on the image quality change curve Q(t) and the behavior trajectory change curve B(t); S202, fitting the risk values corresponding to the continuous frame images on the time sequence to generate the collaborative risk change curve. 4.The video monitoring based early warning method for smart campus one-card system according to claim 1, wherein, The acquisition method of the preset standard trajectory change curve is: Collecting a large number of behavior trajectory change curves of card personnel who successfully pass through the gate in history, performing clustering analysis on all the behavior trajectory change curves to obtain a plurality of clusters, taking the cluster containing the most behavior trajectory change curves as a target class, and taking the central behavior trajectory change curve of all the behavior trajectory change curves in the target class as the standard trajectory change curve. 5.The video monitoring based early warning method for smart campus one-card system according to claim 2, characterized in that, In S103, the abnormal points are determined by using the behavior trajectory change curve and the preset standard trajectory change curve, the collaborative risk change curve is segmented to obtain a plurality of segmented curves corresponding to collaborative risk feature values, comprising: C1, obtaining the intersection of the behavior trajectory change curve and the standard trajectory change curve as an abnormal point; C2, segmenting the collaborative risk change curve according to the time points corresponding to the abnormal points in turn until all the time points corresponding to the abnormal points are traversed to obtain all the segmented curves; C3, obtaining the average value of the risk values of each segmented curve as the collaborative risk feature value of the segmented curve. 6.The video monitoring based early warning method for smart campus one-card system according to claim 2, characterized in that, In S104, the determination method of the face recognition image is: Determining the light direction of each frame image to obtain a light dominant direction, taking the target face region of the frame image with the maximum quality value in the light dominant direction as the face recognition image; The light dominant direction is determined as follows: obtaining the light direction of each frame image in the continuous frame images, counting the occurrence frequency of the light direction, and selecting the light direction with the highest occurrence frequency as the light dominant direction. 7.The video monitoring based early warning method for smart campus one-card system according to claim 1, wherein, S105 further comprises: S401, calculate the deviation value of the behavior trajectory change curve and the preset standard trajectory change curve; S402, obtain the sensing turbulence degree of the card swiping pressure value and the sensing position of the gate within the time window when the card is sensed, and calculate the neighborhood turbulence factor of the card swiping personnel combined with the deviation value; S403, generate a warning instruction of the gate according to the matching result and the neighborhood turbulence factor: If the matching result is successful and the neighborhood turbulence factor is less than the preset turbulence threshold, control the gate to open; If the matching result is successful and the neighborhood turbulence factor is greater than the preset turbulence threshold, play a warning voice and notify the campus security personnel to pay attention to the gate, and control the gate to open; If the matching result fails, control the gate to close and notify the security personnel.

Images