Smart campus all-purpose card early warning method based on video monitoring

Abstract

The invention discloses an intelligent campus all-purpose card early warning method based on video monitoring. The method comprises the following steps: obtaining continuous frame images of a gate card swiping person in a preset time window, wherein the continuous frame images are collected by campus monitoring equipment; generating an image quality change curve and a behavior trajectory change curve by using a multi-modal time sequence analysis mechanism, constructing a collaborative risk change curve, determining abnormal points by using the behavior trajectory change curve and a preset standard trajectory change curve, and segmenting the collaborative risk change curve to obtain a plurality of segmentation curves corresponding to collaborative risk characteristic values; extracting a segmentation curve with the minimum collaborative risk characteristic value as a target curve, and performing illumination direction analysis on the corresponding continuous frame images to determine a face recognition image; and performing adaptive correction on the face recognition image to obtain a target image, performing face recognition on the target image to obtain a matching result of the card swiping person, and generating and executing an early warning instruction of the gate. Therefore, the accuracy and safety of campus all-purpose card control are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a smart campus one-card early warning method based on video surveillance. Background Art

[0002] Campus life is an essential part of everyone's life. Students, a vulnerable group, live together on campus, making campus security extremely important. To prevent unauthorized access, most schools, driven by cost considerations, have adopted a swipe-based entry system based on a single card. However, relying solely on a single card for campus access presents significant security risks. For example, non-campus individuals could illegally gain access by swiping a card using a valid campus card. During busy times, mechanical delays in gates can lead to premeditated entry by unauthorized individuals. Campus surveillance video-assisted card alerts aim to prevent unauthorized cardholders from gaining access through campus gates. By using surveillance video to identify cardholders and assessing alert conditions based on actual conditions, an alert is issued when unauthorized cardholders enter the campus. Most existing facial recognition technologies focus on identifying faces captured by high-definition cameras under restricted conditions. However, due to the limitations of high-definition cameras under restricted conditions, such as high cost and unsuitability for use in densely populated areas, facial recognition technology for high-definition cameras at campus gates is not widely used. Therefore, it is necessary to utilize surveillance video captured by surveillance equipment at campus gates to supplement the card swipe system for school entry. Summary of the Invention

[0003] This application improves the accuracy and security of campus one-card warning by providing a smart campus one-card warning method based on video surveillance.

[0004] This application provides a smart campus card early warning method based on video surveillance, including: S101, obtaining continuous frame images of people swiping cards at gates within a preset time window, collected by campus monitoring equipment; S102, using a preset multimodal time series analysis mechanism to generate an image quality change curve and a behavior trajectory change curve; S103: constructing a collaborative risk change curve based on the image quality change curve and the behavior trajectory change curve, determining abnormal points using the behavior trajectory change curve and a preset standard trajectory change curve, and segmenting the collaborative risk change curve to obtain multiple segmentation curves corresponding to collaborative risk feature values; S104, extracting the segmentation curve with the smallest collaborative risk characteristic value as the target curve, and performing illumination direction analysis on the continuous frame images corresponding to the target curve to determine the face recognition image; S105, adaptively correcting the face recognition image to obtain a target image, performing face recognition on the target image to obtain a matching result of the card swiping person, generating and executing an early warning instruction for the gate machine.

[0005] Preferably, the preset time window is set to a time period formed between the moment when the previous card swiping person passed through the gate and the moment when the current card swiping person completed the card swiping action.

[0006] Preferably, the preset multimodal timing analysis mechanism specifically includes: A1. Perform grayscale processing on consecutive frame images. Based on each frame, obtain the frame image quality value according to the grayscale distribution information and key occlusion degree of the target face area. Fit the quality value on the time series to obtain the image quality change curve Q(t). The target face area is set to the face image of the person swiping the card in the frame image. A2. Based on continuous frame images, track the center point of the target face area to obtain the position vector of the center point relative to the monitoring device, including the distance value and the deflection value. Quantize the trajectory of the position vector to obtain the posture change value of each frame image relative to the monitoring device; A3. Fit the posture change value on the time series to obtain the behavior trajectory change curve B(t).

[0007] Preferably, the quality value of the frame image is calculated according to the following formula:

[0008] in, is the quality value of the i-th frame image, is the grayscale variance value of the target face area, is the preset standard deviation value, is the grayscale mean of the target face area, is the preset standard grayscale mean, It is the preset grayscale sensitivity coefficient, which is determined based on expert experience and actual conditions; The number of key points detected for the target face area, is the total number of key points in the standard face image, and These are preset weight values, which are used to reflect the impact of grayscale distribution information and key occlusion degree on image quality. They are set according to actual needs and expert experience.

[0009] Preferably, in A2, performing trajectory quantization on the position vector to obtain the posture value of each frame image relative to the monitoring device includes: B1. Based on the center point of the target face area in each frame image, the horizontal distance and vertical distance relative to the monitoring device are integrated to obtain the distance value; B2. Obtain the deflection value of the center point relative to the monitoring device and calculate the rate of change of the position vector of the current frame image relative to the position vector of the previous frame image, which is recorded as ; B3. Calculate the attitude change value based on the rate of change of the center point position vector of the current frame image:

[0010] in, is the attitude change value, is the distance change rate, is the rate of change of deflection angle, It is a preset weight coefficient used to adjust the influence of distance change rate and deflection change rate.

[0011] Preferably, the method for constructing the collaborative risk change curve specifically includes: S201, based on the quality values ​​and posture change values ​​of the consecutive frame images at corresponding time points on the image quality change curve Q(t) and the behavior trajectory change curve B(t), obtain the risk value at the corresponding time point; S202 , using the risk values ​​corresponding to the continuous frame images, fitting is performed on the time series to generate a collaborative risk change curve.

[0012] Preferably, the preset standard trajectory change curve is obtained in the following manner: A large number of historical behavioral trajectory change curves of card swipers who have successfully passed the gate are collected, and cluster analysis is performed on all behavioral trajectory change curves to obtain several clusters. The cluster containing the largest number of behavioral trajectory change curves is taken as the target class, and the central behavioral trajectory change curve of all behavioral trajectory change curves in the target class is taken as the standard trajectory change curve.

[0013] Preferably, in S103, the abnormal points are determined by using the behavior trajectory change curve and the preset standard trajectory change curve, and the collaborative risk change curve is segmented to obtain multiple segmentation curves corresponding to collaborative risk characteristic values, including: C1. Obtain the intersection of the behavior trajectory change curve and the standard trajectory change curve as the abnormal point; C2. Segment the collaborative risk change curve according to the time points corresponding to the outliers, until all the time points corresponding to the outliers are traversed to obtain all the segmented curves; C3. Based on each segmentation curve, obtain the average value of its risk value as the collaborative risk characteristic value of the segmentation curve.

[0014] Preferably, in S104, the facial recognition image is determined by: Determine the illumination direction of each frame image to obtain the dominant illumination direction, and use the target face area of ​​the frame image with the dominant illumination direction and the maximum image quality value as the face recognition image; The main illumination direction is determined as follows: obtaining the illumination direction of each frame image in the continuous frame images, counting the occurrence frequency of the illumination direction, and selecting the illumination direction with the highest occurrence frequency as the main illumination direction.

[0015] Preferably, the step S105 further includes: S401, calculating the deviation between the behavior trajectory change curve and the preset standard trajectory change curve; S402: Obtain the card swiping pressure value and the degree of induction turbulence at the sensing position within the time window when the gate senses the card swiping, and combine the deviation value to calculate the neighborhood turbulence factor of the card swiper; S403: Generate a warning instruction for the gate based on the matching result and the neighborhood turbulence factor: If the matching result is successful and the neighborhood turbulence factor is less than the preset turbulence threshold, the gate is controlled to open; If the match result is successful and the neighborhood turbulence factor is greater than the preset turbulence threshold, a warning voice will be played and the campus security personnel will be notified to pay attention to the gate and control the gate to open; If the matching result fails, the gate will be closed and the security personnel will be notified.

[0016] One or more technical solutions provided in this application have at least the following technical effects or advantages: By combining campus surveillance video and one-card card swiping information, the identity and behavior of the card swiper are collaboratively verified, effectively preventing non-campus personnel from illegally entering the campus with a card and ensuring campus safety; using a multimodal time series analysis mechanism to generate image quality change curves and behavior trajectory change curves, construct a collaborative risk change curve, determine the abnormal points by analyzing the relationship between the curves, and cut the collaborative risk change curve, and select the image corresponding to the curve segment with the lowest risk for subsequent image enhancement processing, that is, the best face recognition image is selected based on the risk value collaboratively evaluated by image quality and dynamic image mapping behavior change characteristics, which can further improve the accuracy of face recognition, and then combine the neighborhood turbulence factor of the card swiper to achieve intelligent early warning of the card swiping risk of the card swiper; generate warning instructions for the gate machine based on the matching results and the neighborhood turbulence factor, adopt different gate machine control strategies for card swipers in different risk situations, and improve the intelligence level of gate machine management. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The present invention is a flowchart of a video surveillance-based smart campus card early warning method. DETAILED DESCRIPTION

[0018] To facilitate understanding of the present invention, the present application will be described more comprehensively below with reference to the relevant drawings; the drawings show preferred embodiments of the present invention, but the present invention can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to enable a more thorough and comprehensive understanding of the disclosed content of the present invention.

[0019] It should be noted that the terms “vertical”, “horizontal”, “up”, “down”, “left”, “right” and similar expressions used in this document are for illustrative purposes only and do not represent the only implementation method.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains; the terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more of the associated listed items.

[0021] Example 1: Figure 1 The present invention is a flowchart of a smart campus one-card early warning method based on video surveillance according to an embodiment of the present invention.

[0022] like Figure 1 As shown, a smart campus card early warning method based on video surveillance includes the following steps: S101, obtaining continuous frame images of people swiping cards at the gate collected by campus monitoring equipment within a preset time window, where the preset time window is set to the time period formed by the moment when the previous person swiping the card passes through the gate to the moment when the current person swiping the card completes the card swiping action.

[0023] Among them, campus monitoring equipment is installed directly above the gate to collect real-time video information in front of the gate.

[0024] S102 , using a preset multimodal time series analysis mechanism to generate an image quality change curve and a behavior trajectory change curve.

[0025] In some embodiments, the preset multimodal timing analysis mechanism specifically includes: A1. Perform grayscale processing on continuous frame images. Based on each frame image, obtain the frame image quality value according to the grayscale distribution information and key occlusion degree of the target face area. Fit the quality value on the time series to obtain the image quality change curve Q(t).

[0026] Specifically, the quality value of the frame image is calculated according to the following formula:

[0027] in, is the quality value of the i-th frame image, is the grayscale variance value of the target face area, which is used to reflect the uniformity of facial illumination. is the preset standard deviation value, is the grayscale mean of the target face area, is the preset standard grayscale mean, is a preset grayscale sensitivity coefficient, determined based on expert experience and actual conditions, for example, set to 50; the preset standard variance value and standard grayscale mean are both determined based on the current time attribute and environmental attribute. For each time attribute and environmental attribute (for example, daytime and sunny weather, evening and cloudy weather, etc.), a grayscale variance value and standard grayscale mean of a standard face image (with uniform lighting and no obstructions) are set, and are set based on actual conditions and expert experience. This invention will not elaborate on this; The number of key points detected for the target face area, is the total number of key points in the standard face image, and are preset weight values, which are used to reflect the influence of grayscale distribution information and key occlusion degree on image quality. They are set according to actual needs and expert experience. The sum of the two is 1. For example, is 0.4, is 0.6.

[0028] It should be noted that the target facial region is set to the facial image of the person swiping the card in the frame image. This can be obtained by selecting the facial image using an existing face extraction model. This can be obtained using an open source face detection algorithm (such as the face detection function in OpenCV) or a deep learning model (such as MTCNN, RetinaFace, etc.). This is not described in detail in this invention. The person swiping the card is determined to be the person closest to the gate and monitoring equipment. In addition, the key point detection method in the target facial region is set to use an image recognition model trained using a large number of facial images labeled with key points. The key points are set to the facial features of the facial image to reflect the completeness of the facial image and the degree of facial occlusion. For example, key points include but are not limited to the left eye, right eye, nose tip, upper lip, and lower lip.

[0029] A2. Based on continuous frame images, track the center point of the target face area and obtain the position vector of the center point relative to the monitoring device, including the distance value and the deflection value, which is expressed as: , the position vector is trajectory quantized to obtain the attitude change value of each frame image relative to the monitoring device.

[0030] in, Represents the position vector of the center point of the i-th frame image, Indicates the distance value, Indicates the declination value.

[0031] Specifically, the position vector is quantized to obtain the posture value of each frame image relative to the monitoring device, including: B1. Based on the center point of the target face area in each frame image, its horizontal distance and vertical distance relative to the monitoring device are fused to obtain the distance value.

[0032] Among them, the difference between the horizontal coordinate of the center point in the frame image and the horizontal coordinate of the monitoring device (which needs to be calibrated and converted according to the actual scene and the internal parameters of the monitoring device, and can refer to the conversion between the relevant image and the actual coordinates, which is not elaborated in the present invention) is determined as the horizontal distance, and the difference between the vertical coordinate of the center point in the frame image and the vertical coordinate of the monitoring device (which also needs to be calibrated and converted according to the actual scene) is determined as the vertical distance; the distance value is calculated using the Pythagorean theorem calculation formula.

[0033] B2. Obtain the deflection value of the center point relative to the monitoring device (the direction angle of the center point relative to the monitoring device, usually in radians or degrees, reflecting the degree of deviation of the center point in the frame image relative to the vertical direction of the monitoring device). Calculate the rate of change of the position vector of the current frame image relative to the position vector of the previous frame image (the ratio of the absolute value of the difference between the position vectors and the position vector of the previous frame), recorded as .

[0034] B3. Calculate the attitude change value based on the rate of change of the center point position vector of the current frame image:

[0035] in, is the attitude change value, is the distance change rate, is the rate of change of deflection angle, It is a preset weight coefficient used to adjust the influence of the distance change rate and the deflection change rate. It can be understood that the deflection change rate and the distance change rate work together to determine the posture change value. If the deflection change rate is larger, it means that the amplitude of the card swiping person's movement is larger, then it will be increased on the basis of the distance change rate to further increase the posture change value. This can avoid ignoring the amplitude of the card swiping person's movement, because even if the card swiping person's forward speed is normal, if there is a swaying or pushing scene, it means that there is a hidden risk, which may imply a certain synergistic relationship with the image quality.

[0036] A3. Fit the posture change value on the time series to obtain the behavior trajectory change curve B(t).

[0037] S103: Based on the image quality change curve and the behavior trajectory change curve, a collaborative risk change curve is constructed. The abnormal points are determined using the behavior trajectory change curve and the preset standard trajectory change curve. The collaborative risk change curve is segmented to obtain multiple segmentation curves. Each segmentation curve corresponds to a collaborative risk characteristic value. The collaborative risk characteristic value is determined as the risk value mean of the segmentation curve.

[0038] In some embodiments, the method for constructing a collaborative risk change curve specifically includes: S201 , based on the quality values ​​and posture change values ​​of the consecutive frame images at corresponding time points on the image quality change curve Q(t) and the behavior trajectory change curve B(t), calculate the risk value at the corresponding time point.

[0039] Specifically, the risk value corresponding to the frame image is calculated according to the following formula:

[0040] in, is the risk value, is the attitude change value, is the image quality value.

[0041] S202 , using the risk values ​​corresponding to the continuous frame images, fitting is performed on the time series to generate a collaborative risk change curve.

[0042] Specifically, the preset standard trajectory change curve is obtained as follows: A large number of historical behavioral trajectory change curves of card swipers who have successfully passed the gate are collected, and cluster analysis is performed on all behavioral trajectory change curves to obtain several clusters. The cluster containing the largest number of behavioral trajectory change curves is taken as the target class, and the central behavioral trajectory change curve (average curve) of all behavioral trajectory change curves in the target class is taken as the standard trajectory change curve.

[0043] Curve clustering can be achieved by combining the Dynamic Time Warping (DTW) algorithm with the K-means clustering algorithm. The DTW algorithm calculates the similarity between two curves, while the K-means clustering algorithm groups the curves into clusters based on this similarity. The specific steps are as follows: 1. Initialize K cluster centers, randomly selecting K behavioral trajectory change curves as the initial centers. 2. For each behavioral trajectory change curve, calculate its DTW distance to each cluster center and assign it to the cluster with the closest distance. 3. Recalculate the center of each cluster by taking the average of all curves in that cluster. 4. Repeat steps 2 and 3 until the cluster center no longer changes or the preset number of iterations is reached. 5. Select the cluster containing the largest number of curves as the target class and calculate the average of all curves in the target class to obtain the standard trajectory change curve.

[0044] Specifically, the behavior trajectory change curve and the preset standard trajectory change curve are used to determine the abnormal points, and the collaborative risk change curve is segmented to obtain multiple segmentation curves corresponding to the collaborative risk characteristic values, including: C1. Obtain the intersection of the behavior trajectory change curve and the standard trajectory change curve as the abnormal point.

[0045] C2. Segment the collaborative risk change curve according to the time points corresponding to the outliers, until all the time points corresponding to the outliers are traversed to obtain all segmentation curves.

[0046] C3. Based on each segmentation curve, obtain the average value of its risk value as the collaborative risk characteristic value of the segmentation curve.

[0047] S104 , extracting the segmentation curve with the smallest collaborative risk characteristic value as the target curve, and performing illumination direction analysis on the continuous frame images corresponding to the target curve to determine the face recognition image.

[0048] The face recognition image is determined by determining the illumination direction of each frame image to obtain the dominant illumination direction, and taking the target face area of ​​the frame image with the dominant illumination direction and the maximum image quality value as the face recognition image.

[0049] It should be noted that the smaller the collaborative risk characteristic value, the higher the image quality and the smaller the posture change, that is, the more stable the person's behavior trajectory is. It can also reflect the stability of image acquisition and high quality, and the orderliness of the card swiping trajectory of the person swiping the card.

[0050] The method for analyzing the illumination direction of an image may refer to any one of the related prior arts, and the present invention will not elaborate on this.

[0051] As an example, the target face area of ​​each frame image is gray-scaled and the gradient vector of each pixel is obtained using a gradient operator (such as the Sobel operator, the Prewitt operator, etc.). The gradient vector g is a two-dimensional vector consisting of the gradient components of the pixel in the horizontal and vertical directions, that is, g=(gx,gy , the horizontal gradient component gx represents the rate of change of the pixel value in the horizontal direction (i.e., the X-axis direction), and the vertical gradient component gy represents the rate of change of the pixel value in the vertical direction (i.e., the Y-axis direction). The magnitude of the gradient vector (i.e., the modulus) reflects the severity of the pixel value change, while the direction of the gradient vector indicates the direction in which the pixel value changes fastest. In illumination analysis, the gradient vector can help identify information such as edges, textures, and illumination changes in the image; define multiple reference directions (such as 0°, 45°, 90°, 135°, etc.), and for each reference direction, calculate the sum of the projections of the gradient vectors of all pixels in this direction to obtain the fitting vector in the reference direction. In the specific calculation, the gradient vector of each pixel can be decomposed into components in the reference direction, and the components of all pixels are added to obtain the length and direction of the fitting vector. Assuming there are N pixels, the gradient vector of the i-th pixel is gi=(gix,giy), and the reference direction is θ, then the projection of the pixel in the reference direction is , the sum of the projections of all pixels in the reference direction is , the sum of the projections The size of reflects the overall degree of change of the gradient vectors of all pixels in the image in the reference direction θ. If the value is large, it means that in the reference direction θ; the reference direction with the largest sum of projections is selected as the illumination direction of the image.

[0052] In some embodiments, the illumination direction of the continuous frame images corresponding to the target curve is analyzed to obtain the dominant illumination direction, specifically including: obtaining the illumination direction of each frame image in the continuous frame images, counting the frequency of occurrence of the illumination direction, and selecting the illumination direction with the highest frequency as the dominant illumination direction.

[0053] S105, adaptively correct the face recognition image according to the dominant illumination direction (for example, use an adaptive gamma correction algorithm to perform illumination correction to improve image quality) to obtain a target image, use the target image to perform face recognition to obtain a matching result of the card swiping person, generate an early warning instruction for the gate machine and execute it.

[0054] Among them, face recognition is specifically as follows: the target image is input into the campus face database for matching. If the matched face identity information is consistent with the card swiping information, the match is successful; if the matched face identity information is inconsistent with the card swiping information or the face identity information is not matched, the match fails.

[0055] In some embodiments, step S105 further includes: S401, calculating the deviation between the behavior trajectory change curve and the preset standard trajectory change curve:

[0056] Where f is the deviation value, t0 is the start time of the preset time window, and t1 is the end time of the preset time window. is the behavior trajectory change curve function, is the standard trajectory change curve function.

[0057] In actual calculations, a discretization method can also be used to divide the preset time window into several small time periods, calculate the sum of the absolute values ​​of the differences between the behavior trajectory change curve and the standard trajectory change curve in each time period, and then divide it by the total number of time periods to obtain the deviation value.

[0058] Therefore, by analyzing the posture change value, it is possible to reflect the passive abnormal behavior of the card swiper, such as being forced to sway left and right, pushing, etc.

[0059] S402, obtain the turbulence degree of the card swiping pressure value and the sensing position within the time window when the gate senses the card swiping (take the average of the change rate of the pressure value and the sensing position at all adjacent moments in the time window, and weighted sum the average change rate of the pressure value and the sensing position to obtain the turbulence degree), and combine it with the deviation value to calculate the neighborhood turbulence factor of the card swiping person.

[0060] Among them, the gate machine's card swiping area is equipped with a built-in pressure sensor and a position monitoring sensor, which are used to collect real-time card swiping pressure values ​​and sensing position information respectively.

[0061] The neighborhood turbulence factor of the card swiper is calculated according to the following formula:

[0062] Among them, k is the neighborhood turbulence factor, f is the deviation value, w is the induced turbulence degree, and D is the credibility factor. 、 are preset weight factors, which are used to indicate the influence of deviation value and turbulence degree on neighborhood turbulence factor, ranging from 0 to 1 and summing to 1. They are set according to the actual situation and specific needs of the scene, for example, is 0.7, is 0.3.

[0063] It should be noted that the credibility factor is used to reflect the correlation between the deviation value and the degree of induced turbulence of the current card swiping person and their surrounding environment. The credibility factor can be determined based on other facial areas within the preset range of the center point of the target facial area in the continuous frame image. The credibility factor is determined by the rate of change of the distance values ​​of other facial areas relative to the center point of the target facial area in the continuous frame image. The greater the rate of change, the greater the credibility factor. If there are no other facial areas within the preset neighborhood range, the credibility factor is determined to be 0. Among them, the preset neighborhood range is set as a range with the center point as the center and the preset distance value as the radius. The preset distance is set based on expert experience and actual conditions (it can be set by collecting the distance values ​​between the centers of corresponding facial areas in images with historical crowding or pushing of people). It is used to measure the safe distance value between people.

[0064] It should be noted that when calculating the neighborhood turbulence factor, the deviation value, the degree of induced turbulence, and the credibility factor need to be normalized. The neighborhood turbulence factor can also be directly normalized. The normalization method can refer to the relevant existing technology and will not be elaborated in the present invention.

[0065] S403: Generate a warning instruction for the gate based on the matching result and the neighborhood turbulence factor, specifically: If the matching result is successful and the neighborhood turbulence factor is less than the preset turbulence threshold, the gate is controlled to open; the preset turbulence threshold is used to measure the degree of neighborhood turbulence and is set based on actual conditions and historical experience; If the match result is successful and the neighborhood turbulence factor is greater than the preset turbulence threshold, a warning voice will be played and the campus security personnel will be notified to continue to pay attention to the gate and control the gate to open. The warning voice is used to provide a voice warning to prompt people at the gate to evacuate and maintain order. If the matching result fails, the gate will be closed and the security personnel will be notified.

[0066] In summary, the dual-dimensional time series data of image quality and behavioral trajectory are integrated to construct a collaborative risk curve, breaking through the limitations of single modality analysis; the quality is quantified by grayscale distribution variance (light uniformity) and key point occlusion rate, overcoming the defect of traditional methods that only rely on clarity; the weighted fusion of the distance change rate and the angular change rate of the target face center relative to the monitoring equipment is introduced to accurately capture small abnormal postures; the intersection (abnormal point) of the behavior curve and the standard curve is used to segment the risk curve to achieve local risk focus, and the segment with the smallest collaborative risk characteristic value (high quality + low posture change) is selected. The images of this period are preferentially extracted for recognition, which significantly improves the accuracy of face matching. Through the analysis of the dominant illumination direction, the recognition degradation problem in complex lighting scenes is solved and the false rejection rate is reduced; the neighborhood turbulence factor integrates the three parameters of behavior deviation value, gate pressure turbulence, and surrounding personnel density to achieve early warning of premeditated intrusions such as following or abnormal pushing.

[0067] Traditional static capture fails when there is uneven lighting or partial occlusion (such as waving or lowering the head). The optimal frame is dynamically selected through the time-series quality curve. In the scene of card skimming or intrusion, people may maintain a normal speed, but there will be slight pushing and shaking. The rate of change of behavioral trajectory is coordinated with the image quality fluctuation to achieve accurate capture. During peak traffic hours, people are prone to tailgating, intrusion, pushing and arguing. The neighborhood turbulence factor integrates the gate pressure sensor data and the surrounding personnel density to trigger differentiated gate warnings.

[0068] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By combining campus surveillance video and one-card card swiping information, the identity and behavior of the card swiper are collaboratively verified, effectively preventing non-campus personnel from illegally entering the campus with a card and ensuring campus safety; using a multimodal time series analysis mechanism to generate image quality change curves and behavior trajectory change curves, construct a collaborative risk change curve, determine the abnormal points by analyzing the relationship between the curves, and cut the collaborative risk change curve, and select the image corresponding to the curve segment with the lowest risk for subsequent image enhancement processing, that is, the best face recognition image is selected based on the risk value collaboratively evaluated by image quality and dynamic image mapping behavior change characteristics, which can further improve the accuracy of face recognition, and then combine the neighborhood turbulence factor of the card swiper to achieve intelligent early warning of the card swiping risk of the card swiper; generate warning instructions for the gate machine based on the matching results and the neighborhood turbulence factor, adopt different gate machine control strategies for card swipers in different risk situations, and improve the intelligence level of gate machine management.

[0069] Compared with simply relying on a card to enter the school, the addition of video surveillance and facial recognition greatly increases the difficulty for illegal cardholders to enter the campus, effectively reducing campus security risks. By comprehensively considering multiple factors such as image quality, behavioral trajectory, card swiping pressure and sensing position, the neighborhood turbulence factor is calculated, which can more accurately judge the risk situation of card swipers and reduce false alarms and missed alarms. It does not rely on high-definition cameras and can be achieved using the existing monitoring equipment at the campus gate. It is low-cost and suitable for large-scale, densely populated places on campus.

[0070] Focusing on analyzing the image quality and behavioral changes of the facial image of the current card swiper at the gate, it eliminates the need for detailed image and behavioral analysis of other people behind the card swiper. It can more efficiently analyze the turbulence factor of the card swiper when swiping the card, indirectly reflecting the risk of the current card swiper passing through. This reduces unnecessary image and behavioral analysis, reduces computational complexity, and improves the real-time performance of the system. Based on the size of the turbulence factor, it can implement differentiated control strategies for the gate, quickly releasing people with lower risks and issuing early warnings and focusing on people with higher risks, thereby improving campus security and management efficiency. In summary, by using continuous frame images of people swiping gates captured by campus surveillance equipment, a multimodal time series analysis mechanism was used to generate image quality and behavioral trajectory change curves, constructing a collaborative risk change curve. This was then used to analyze lighting direction, identify faces, and generate gate warning instructions to ensure the safety and accuracy of campus gate access. This system achieves comprehensive monitoring and early warning of gate users, effectively identifying potential abnormal behavior during the swiping process, such as subsequent entry by strangers, providing strong support for campus security management.

[0071] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A smart campus card early warning method based on video surveillance, characterized in that: include: S101, obtaining continuous frame images of people swiping cards at gates within a preset time window, collected by campus monitoring equipment; S102, using a preset multimodal time series analysis mechanism to generate an image quality change curve and a behavior trajectory change curve; S103: constructing a collaborative risk change curve based on the image quality change curve and the behavior trajectory change curve, determining abnormal points using the behavior trajectory change curve and a preset standard trajectory change curve, and segmenting the collaborative risk change curve to obtain multiple segmentation curves corresponding to collaborative risk feature values; S104, extracting the segmentation curve with the smallest collaborative risk characteristic value as the target curve, and performing illumination direction analysis on the continuous frame images corresponding to the target curve to determine the face recognition image; S105, adaptively correcting the face recognition image to obtain a target image, performing face recognition on the target image to obtain a matching result of the card swiping person, generating and executing an early warning instruction for the gate machine.

2. The video surveillance-based smart campus card early warning method according to claim 1, characterized in that: The preset time window is set as a time period formed by the moment when the previous card swiping person passes through the gate to the moment when the current card swiping person completes the card swiping action.

3. The video surveillance-based smart campus card early warning method according to claim 2, characterized in that: The preset multimodal timing analysis mechanism specifically includes: A1. Perform grayscale processing on consecutive frame images. Based on each frame, obtain the frame image quality value according to the grayscale distribution information and key occlusion degree of the target face area. Fit the quality value on the time series to obtain the image quality change curve Q(t). The target face area is set to the face image of the person swiping the card in the frame image. A2. Based on continuous frame images, track the center point of the target face area to obtain the position vector of the center point relative to the monitoring device, including the distance value and the deflection value. Quantize the trajectory of the position vector to obtain the posture change value of each frame image relative to the monitoring device; A3. Fit the posture change value on the time series to obtain the behavior trajectory change curve B(t).

4. The video surveillance-based smart campus card early warning method according to claim 3, characterized in that: The quality value of the frame image is calculated according to the following formula: in, is the quality value of the i-th frame image, is the grayscale variance value of the target face area, is the preset standard deviation value, is the grayscale mean of the target face area, is the preset standard grayscale mean, It is the preset grayscale sensitivity coefficient, which is determined based on expert experience and actual conditions; The number of key points detected for the target face area, is the total number of key points in the standard face image, and These are preset weight values, which are used to reflect the impact of grayscale distribution information and key occlusion degree on image quality. They are set according to actual needs and expert experience.

5. The video surveillance-based smart campus card early warning method according to claim 3, characterized in that: In A2, the position vector is quantized to obtain the posture value of each frame image relative to the monitoring device, including: B1. Based on the center point of the target face area in each frame image, the horizontal distance and vertical distance relative to the monitoring device are integrated to obtain the distance value; B2. Obtain the deflection value of the center point relative to the monitoring device and calculate the rate of change of the position vector of the current frame image relative to the position vector of the previous frame image, which is recorded as ; B3. Calculate the attitude change value based on the rate of change of the center point position vector of the current frame image: in, is the attitude change value, is the rate of change of distance, is the rate of change of deflection angle, It is a preset weight coefficient used to adjust the influence of distance change rate and deflection change rate.

6. The video surveillance-based smart campus card early warning method according to claim 3, characterized in that: The method for constructing the collaborative risk change curve specifically includes: S201, obtaining a risk value at a corresponding time point based on the quality values ​​and posture change values ​​of consecutive frame images at corresponding time points on the image quality change curve Q(t) and the behavior trajectory change curve B(t); S202 , using the risk values ​​corresponding to the continuous frame images, fitting is performed on the time series to generate a collaborative risk change curve.

7. The video surveillance-based smart campus card early warning method according to claim 1, characterized in that: The method for obtaining the preset standard trajectory change curve is: A large number of historical behavioral trajectory change curves of card swipers who have successfully passed the gate are collected, and cluster analysis is performed on all behavioral trajectory change curves to obtain several clusters. The cluster containing the largest number of behavioral trajectory change curves is taken as the target class, and the central behavioral trajectory change curve of all behavioral trajectory change curves in the target class is taken as the standard trajectory change curve.

8. The video surveillance-based smart campus card early warning method according to claim 3, characterized in that: In S103, the behavior trajectory change curve and the preset standard trajectory change curve are used to determine the abnormal points, and the collaborative risk change curve is segmented to obtain multiple segmentation curves corresponding to collaborative risk feature values, including: C1. Obtain the intersection of the behavior trajectory change curve and the standard trajectory change curve as the abnormal point; C2. Segment the collaborative risk change curve according to the time points corresponding to the outliers, until all the time points corresponding to the outliers are traversed to obtain all the segmented curves; C3. Based on each segmentation curve, obtain the average value of its risk value as the collaborative risk characteristic value of the segmentation curve.

9. The video surveillance-based smart campus card early warning method according to claim 3, characterized in that: In S104, the facial recognition image is determined as follows: Determine the illumination direction of each frame image to obtain the dominant illumination direction, and use the target face area of ​​the frame image with the dominant illumination direction and the maximum image quality value as the face recognition image; The main illumination direction is determined as follows: obtaining the illumination direction of each frame image in the continuous frame images, counting the occurrence frequency of the illumination direction, and selecting the illumination direction with the highest occurrence frequency as the main illumination direction.

10. The video surveillance-based smart campus card early warning method according to claim 1, characterized in that: The S105 further includes: S401, calculating the deviation between the behavior trajectory change curve and the preset standard trajectory change curve; S402: Obtain the card swiping pressure value and the degree of induction turbulence at the sensing position within the time window when the gate senses the card swiping, and combine the deviation value to calculate the neighborhood turbulence factor of the card swiper; S403: Generate a warning instruction for the gate based on the matching result and the neighborhood turbulence factor: If the matching result is successful and the neighborhood turbulence factor is less than the preset turbulence threshold, the gate is controlled to open; If the match result is successful and the neighborhood turbulence factor is greater than the preset turbulence threshold, a warning voice will be played and the campus security personnel will be notified to pay attention to the gate and control the gate to open; If the matching result fails, the gate will be closed and the security personnel will be notified.

Images