Foreign person intrusion identification method, system and device, and storage medium

Abstract

The invention relates to the technical field of video monitoring, and particularly provides a foreign person intrusion identification method, system and device and a storage medium, and the method comprises the steps: obtaining snapshot event data of a camera, the snapshot event data comprising snapshot image data and snapshot attribute data; performing data structure mapping on the snapshot image data and the snapshot attribute data to generate to-be-identified data; performing face region extraction, posture correction and feature extraction on the to-be-recognized data to obtain similarity data and feature vectors of faces, and obtaining recognition result data based on the similarity data, the feature vectors of the faces and the snapshot attribute data; and judging the identification result data based on a preset judgment rule to obtain identity judgment data. According to the invention, monitoring snapshot is automatically processed, identity determination is completed, identification accuracy is improved, manual dependence is reduced, and missing detection is reduced.

Description

Technical Field

[0001] This invention belongs to the field of video surveillance technology, specifically relating to a method, system, device, and storage medium for identifying unauthorized personnel intrusion. Background Technology

[0002] With the development of intelligent monitoring equipment and artificial intelligence technology, enterprises, parks and data centers generally rely on video surveillance to achieve security management of personnel access.

[0003] However, existing technologies have significant drawbacks: First, traditional manual monitoring in duty rooms requires long-term screen monitoring, and staff are prone to fatigue during multi-point monitoring, nighttime monitoring, or low-traffic periods, leading to decreased attention and missed detections; second, manual judgment of intrusion behavior by outsiders relies on the naked eye, resulting in a long average response time, usually exceeding several minutes, making it impossible to guarantee immediate identification and handling, and easily missing the best opportunity for intervention; third, the accuracy of manual identification of personnel cannot be guaranteed, especially during high-traffic monitoring periods such as rush hours, making it easy to make misjudgments or misses, and difficult to effectively identify the risk of intrusion by outsiders. Summary of the Invention

[0004] In view of the above-mentioned shortcomings of the prior art, the present invention provides a method, system, device and storage medium for identifying unauthorized personnel intrusion, so as to solve the above-mentioned technical problems.

[0005] In a first aspect, the present invention provides a method for identifying unauthorized personnel intrusion, comprising: Acquire snapshot event data from the camera, the snapshot event data including snapshot image data and snapshot attribute data; Data structure mapping is performed on the captured image data and captured attribute data respectively to generate data to be identified; The face region is extracted, pose is corrected, and features are extracted from the data to be identified, resulting in similarity data and face feature vectors. Based on the similarity data, face feature vectors, and capture attribute data, the recognition result data is obtained. Based on preset judgment rules, the recognition result data is judged to obtain identity judgment data.

[0006] In one optional implementation, capture event data from the camera is acquired, the capture event data including captured image data and capture attribute data, including: Based on the callback mechanism pre-set by the camera, the capture event data sent by the camera after the capture event is triggered is read, the binary content of the image in the capture event data is extracted, and the binary content of the image is used as the capture image data. The attribute fields in the captured event data are parsed, the image generation time field is parsed into time information, the image size field is parsed into size information, the camera address field is parsed into address information, the event number field is parsed into event identification information, and the image quality related fields are parsed into quality information to obtain the captured attribute data; The captured image data is temporarily stored in the image buffer area, the captured attribute data is temporarily stored in the attribute buffer area, and the correspondence between the captured image data and the captured attribute data is saved.

[0007] In one optional implementation, data structure mapping is performed on the captured image data and captured attribute data respectively to generate data to be identified, including: The captured image data is used as the image content input, and the captured attribute data is used as the attribute content input. Data structure mapping is performed on the image content input and the attribute content input respectively, mapping the image content input to the image input field and the attribute content input to the attribute input field. The image input field and the attribute input field are combined to obtain the data to be identified, which includes the image input part and the attribute input part. The data to be identified corresponds to the data entity after the structured processing of the captured event data.

[0008] In one optional implementation, face region extraction, pose correction, and feature extraction are performed on the data to be identified to obtain similarity data and face feature vectors. Based on the similarity data, face feature vectors, and capture attribute data, recognition result data is obtained, including: The brightness of the captured image data is read pixel by pixel. The edge response of the read pixel brightness is analyzed based on the Sobel gradient operator. Then, the gradient change of the continuous pixels is calculated to generate face region data that represents the range of the face region. The face region data is processed for pose correction. The affine transformation matrix is ​​calculated by detecting the relative geometric relationship between the center points of the left and right eyes, the tip of the nose and the corner of the mouth in the face region data. The face region data is then rotated in two dimensions and resampled proportionally to obtain the aligned face input data. The aligned face input data is processed by feature mapping. By performing local convolution kernel sliding calculation, channel direction feature accumulation, feature tensor dimension compression and vector flattening on the image grid, feature vectors for representing the face are extracted. Based on the captured attribute data, the range of the personnel registration database to be compared is determined. The registration feature vector is read from the personnel registration database. The feature vector of the face and the registration feature vector are multiplied and summed in one dimension, the vector norm is solved and the cosine ratio is calculated to obtain the similarity data. Based on the preset data structure template, similarity data is filled into the matching degree field, facial feature vectors are filled into the feature expression field, and the attribute fields of the capture attribute data are filled into the corresponding event attribute fields. Based on the filled data structure template, field combination processing is performed, and the matching degree field, feature expression field, and event attribute field are sequentially concatenated to obtain recognition result data containing the matching degree field, feature expression field, and event attribute field.

[0009] In one optional implementation, the identification result data is judged based on preset judgment rules to obtain identity judgment data, including: Read the similarity data from the recognition results data to obtain the similarity value; The feature vector of the face is read from the recognition result data to obtain the personnel identification information corresponding to the feature vector; The capture attribute data is read from the recognition result data to obtain the time information, size information, address information, event identifier information and quality information in the capture attribute data; The similarity value is compared with the similarity threshold in the preset judgment rule to obtain the judgment result; The personnel identification information corresponding to the feature vector is read and processed to obtain the personnel identification information associated with the identification. Field extraction is performed on the snapshot attribute data, and the time information, size information, address information, event identifier information and quality information in the snapshot attribute data are extracted into attribute entries respectively; The judgment results, personnel identification information, and various attribute entries are arranged in order according to a preset data structure to generate identity judgment data.

[0010] In an optional implementation, the similarity value is compared with a similarity threshold in a preset judgment rule to obtain a judgment result, including: The similarity is normalized by converting it using a mapping formula to obtain a normalized similarity value. Perform a comparison between the normalized similarity value and the similarity threshold in the decision rule: A match is determined when the normalized similarity value is greater than or equal to the similarity threshold. When the normalized similarity value is less than the similarity threshold, it is judged as a mismatch.

[0011] Secondly, the present invention provides an intrusion detection system for unauthorized personnel, comprising: The data acquisition module is used to acquire the capture event data of the camera, which includes captured image data and capture attribute data; The structure mapping module is used to perform data structure mapping on the captured image data and captured attribute data respectively to generate the data to be recognized. The result recognition module is used to extract face regions, correct poses, and extract features from the data to be recognized, thereby generating similarity data and face feature vectors. Based on the similarity data, face feature vectors, and capture attribute data, the recognition result data is obtained. The identity determination module is used to determine the identity determination data based on preset determination rules.

[0012] In one optional implementation, the result recognition module includes: The face region generation unit reads the brightness of each pixel in the captured image data, analyzes the edge response of the read pixel brightness based on the Sobel gradient operator, and then performs boundary convergence calculation on the gradient changes of continuous pixels to generate face region data that represents the range of the face region. The face input data generation unit is used to perform pose correction processing on the face region data. It calculates the affine transformation matrix by detecting the relative geometric relationship between the center points of the left and right eyes, the tip of the nose and the corner of the mouth in the face region data, and performs two-dimensional rotation and proportional resampling on the face region data to obtain aligned face input data. The face feature vector extraction unit is used to perform feature mapping processing on the aligned face input data. It extracts feature vectors to represent the face by performing local convolution kernel sliding calculation, feature accumulation in the channel direction, dimensionality compression of the feature tensor, and vector flattening on the image grid. The similarity data generation unit is used to determine the range of the personnel registration database to be compared based on the captured attribute data, read the registration feature vector from the personnel registration database, and perform dimension-wise product summation, vector norm solution and cosine ratio calculation on the feature vector of the face and the registration feature vector to obtain similarity data. The result recognition unit is used to fill similarity data into the matching degree field, fill facial feature vectors into the feature expression field, and fill each attribute field of the captured attribute data into the corresponding event attribute field based on a preset data structure template. Based on the filled data structure template, field combination processing is performed, and the matching degree field, feature expression field, and event attribute field are sequentially concatenated to obtain recognition result data containing the matching degree field, feature expression field, and event attribute field.

[0013] Thirdly, a device is provided, comprising: Memory used to store intrusion detection programs for unauthorized personnel; A processor is configured to implement the steps of the intrusion detection method as provided in the first aspect when executing the intrusion detection program.

[0014] Fourthly, a computer-readable storage medium is provided, on which an intrusion detection program is stored, wherein when the intrusion detection program is executed by a processor, it implements the steps of the intrusion detection method provided in the first aspect.

[0015] The beneficial effects of this invention are as follows: The method, system, device, and storage medium for identifying intruders provided by this invention, through structured processing of camera-captured event data and completion of face region extraction, pose correction, feature extraction, and similarity calculation under a unified data format, achieve automated identification of personnel in the monitored images. Compared with existing monitoring methods that rely on manual screen monitoring and judgment, this significantly reduces the risk of missed detections due to human fatigue. By performing identity determination based on feature vector similarity and preset judgment rules, the identity recognition process has a stable judgment standard, avoiding the problem of misjudgment or missed detection by humans during high-traffic periods. By automatically acquiring and processing captured image data and captured attribute data in real time, the identification of intrusion behavior no longer relies on manual response, thereby greatly reducing the lag in manual judgment and achieving timely identification of potential intruders.

[0016] Furthermore, the design principle of this invention is reliable, the structure is simple, and it has a very wide range of application prospects. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of a device provided in an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0023] The intrusion detection method provided in this embodiment of the invention is executed by a computer device, and correspondingly, the intrusion detection system runs on the computer device.

[0024] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The implementing entity can be an intrusion detection system. Depending on the specific requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0025] like Figure 1 As shown, the method includes: S1. Obtain the capture event data of the camera, wherein the capture event data includes captured image data and capture attribute data.

[0026] S2. Perform data structure mapping on the captured image data and captured attribute data respectively to generate the data to be identified.

[0027] S3. Perform face region extraction, pose correction, and feature extraction on the data to be recognized to obtain similarity data and face feature vectors. Based on the similarity data, face feature vectors, and capture attribute data, obtain the recognition result data.

[0028] S4. Based on the preset judgment rules, the recognition result data is judged to obtain identity judgment data.

[0029] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0030] Acquire camera capture event data, which includes captured image data and captured attribute data, including: Based on the camera's pre-set callback mechanism, the system reads the capture event data sent by the camera after the capture event is triggered. It extracts the binary image content from the capture event data and uses this binary content as the captured image data. The system then parses the attribute fields in the capture event data: the image generation time field is parsed into time information, the image size field into size information, the camera address field into address information, the event number field into event identifier information, and the image quality-related fields into quality information, thus obtaining the capture attribute data. The captured image data is temporarily stored in the image buffer, and the capture attribute data is temporarily stored in the attribute buffer, and the correspondence between the captured image data and the capture attribute data is saved.

[0031] Specifically, the camera has been correctly installed and configured. In this embodiment, the Hikvision DS-2CD7147EWDV2-IZS camera is used.

[0032] Camera capture parameter configuration: {Face capture mode: Optimal capture; Number of captures: 1; Face pose filtering: Left / Right angle: 40°; Tilt / Up angle: 30°;} Call the SDK (provided by Hikvision) to set the snapshot event callback function: The security system's background program obtains camera information (IP address, port number, username, password, etc.) and calls the SDK to complete the camera login and connection operations; The system background program calls the SDK to register the snapshot event handling function as a callback function; (when the camera generates a snapshot event, the SDK will automatically call the registered callback function and pass the snapshot message to the callback function to realize the event response and processing). Specific steps for setting callback functions using the SDK: The Hikvision dynamic link library (DLL) is loaded using the `loadLibrary` function in the SDK, enabling the program to call the low-level instructions and interface functions defined in the SDK. After successful loading, the program can then utilize the SDK's capabilities such as device login, event listening, and data acquisition. After loading the local library, the system calls the NET_DVR_Init function to initialize the SDK environment. Initialization includes runtime environment configuration, internal handle creation, and global parameter settings, providing the runtime environment foundation for subsequent device login, arming, and callback registration. Assemble the camera connection parameters, including device IP address, communication port number, login account and password, and then call the NET_DVR_LOGIN_V40 function to complete the camera login operation; After successful login, the NET_DVR_SetDVRMessageCallBack_V30 function is called to register the preset snapshot event handling function OnSnapEventCallback into the SDK. When the camera captures an event (such as a face capture event COMM_UPLOAD_FACESNAP_RESULT), the SDK will automatically call this callback function, carrying the capture data: event type, image binary data, face coordinates, and other formatted data. After successful device login and callback function registration, the NET_DVR_SetupAlarmChan_V41 function is called to complete the camera arming operation; after the arming operation is completed, the camera can report various events to the system in real time, including motion detection, face capture and other events. Through the above alarm arming operations, when the camera detects a target event, it will automatically report the event message to the system, and the OnSnapEventCallback callback function registered in the previous step will be responsible for the business processing.

[0033] The specific implementation steps of the callback function are as follows: The security system's backend program (via the camera SDK) retrieves image, IP, and other information from the captured event message (after completing the face capture operation, the Hikvision camera pushes the capture message to a callback function registered via the SDK). The message is parsed in the callback function, and the key data includes: dwFacePicLen: Represents the binary length of the face sub-image. When the value is greater than 0, it means that the captured information contains a face image that can be used for recognition. pBuffer1: A pointer to the face sub-image data. The background program can extract the face capture image in JPEG format based on this pointer and its length. sDeviceIP: This is the IP address information of the camera device used to push messages. It is stored as a byte array, and the background program needs to convert this IP address into a standard IPv4 string format. After parsing the above fields, the program will assemble the image data, device IP, capture time and other information into a capture record for subsequent face recognition and event processing.

[0034] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0035] Data structure mapping is performed on the captured image data and captured attribute data respectively to generate data to be recognized, including: The captured image data is used as the image content input, and the captured attribute data is used as the attribute content input. Data structure mapping is performed on the image content input and the attribute content input respectively, mapping the image content input to the image input field and the attribute content input to the attribute input field. The image input field and the attribute input field are combined to obtain the data to be identified, which includes the image input part and the attribute input part. The data to be identified corresponds to the data entity after the structured processing of the captured event data.

[0036] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0037] The process involves extracting face regions, correcting pose, and extracting features from the data to be recognized, resulting in similarity data and facial feature vectors. Based on the similarity data, facial feature vectors, and captured attribute data, the recognition result data is obtained, including: The system performs pixel-by-pixel brightness readings on captured image data, analyzes the edge response of the read pixel brightness based on the Sobel gradient operator, and then performs boundary convergence calculations on the gradient changes of consecutive pixels to generate face region data representing the range of the face region. The face region data undergoes pose correction processing by calculating the affine transformation matrix through detecting the relative geometric relationships between the center points of the left and right eyes, the tip of the nose, and the corners of the mouth within the face region data. Two-dimensional rotation and proportional resampling are then performed on the face region data to obtain aligned face input data. Feature mapping processing is then performed on the aligned face input data by performing local convolution kernel sliding calculations on the image grid, feature accumulation in the channel direction, dimensionality compression of the feature tensor, and vector flattening to extract feature vectors representing the face. The scope of the personnel registration database to be compared is determined by the captured attribute data. The registration feature vector is read from the personnel registration database, and the feature vector of the face and the registration feature vector are multiplied and summed dimension by dimension, the vector norm is solved and the cosine ratio is calculated to obtain similarity data. Based on a preset data structure template, the similarity data is filled into the matching degree field, the feature vector of the face is filled into the feature expression field, and the attribute fields of the captured attribute data are filled into the corresponding event attribute fields. Based on the filled data structure template, field combination processing is performed, and the matching degree field, feature expression field and event attribute field are sequentially concatenated to obtain the recognition result data containing the matching degree field, feature expression field and event attribute field.

[0038] Specifically, the system reads captured image data from the data to be identified, loads the captured image data into a pixel buffer, and performs pixel-by-pixel brightness reading processing on the captured image data. For color images, the system converts the pixel values ​​of the R, G, and B channels into single-channel brightness values ​​according to preset weighting coefficients. For example, for each pixel, it calculates Y = 0.299R + 0.587G + 0.114B to obtain the brightness matrix of the entire image. After obtaining the brightness matrix, the system uses Sobel... The gradient operator performs edge response analysis on the brightness matrix, performing convolution operations on the brightness matrix in the horizontal and vertical directions to obtain the gradient values ​​of each pixel location in the horizontal and vertical gradient directions, and calculates the gradient magnitude and gradient direction accordingly. The system scans the gradient magnitude distribution of consecutive pixel locations, performs connectivity analysis on pixels with gradient magnitudes exceeding a preset gradient threshold, and performs boundary convergence calculation by aggregating adjacent high-gradient pixels, tracking boundary pixels, and filtering small regions, converging discrete edge responses into continuous boundary contours. In the set of boundary contours, the system further performs candidate region screening, selecting candidate boxes that meet the characteristics of face regions based on factors such as aspect ratio, area range, and region location. The candidate box that best meets the preset conditions is selected, and the image region corresponding to the candidate box is cropped out as face region data representing the range of the face region and stored in the region cache. Subsequently, the system processes the face region data... The system performs pose correction processing by using a pre-trained facial landmark detection algorithm within the face region data to detect the pixel coordinates of the center points of the left and right eyes, the tip of the nose, and the corners of the mouth. Geometric verification is performed on the detected landmark positions, and abnormal points that significantly deviate from the facial structure are removed. The rotation angle of the face is determined by calculating the angle between the line connecting the left and right eyes and the horizontal line. The scale information of the face in the vertical direction is determined by the relative distances between the eyes, the tip of the nose, and the corners of the mouth. Based on these relative geometric relationships, an affine transformation matrix is ​​constructed, containing rotation, scaling, and translation parameters. The system uses this affine transformation matrix to perform two-dimensional rotation and proportional resampling processing on the face region data. During rotation and scaling, bilinear interpolation or other preset interpolation algorithms are used to calculate the corresponding pixel value from the original face region data for each output pixel. The transformed face image is then normalized to a uniform size, such as a fixed width and height, resulting in aligned face input data.After completing pose correction, the system performs feature mapping processing on the aligned face input data. The system divides the aligned face input data into a fixed-size image grid, applies local convolution kernel sliding calculations to each grid region, performs convolution operations at each kernel position, calculates the convolution response value at that position, and slides the convolution kernel across the entire image plane to form a multi-channel feature response map. Subsequently, feature accumulation processing is performed on the feature responses along the channel direction. Multi-channel features can be aggregated through inter-channel summation, selection of the maximum value between channels, or preset channel combinations to obtain a feature tensor with compressed redundant information. Dimensionality compression processing is then performed on the obtained feature tensor, for example, by downsampling, pooling operations, or linear transformations to reduce the spatial dimension of the feature tensor and decrease the amount of feature data. After dimensionality compression, vector flattening is performed on the compressed feature tensor, unfolding the two-dimensional or three-dimensional feature tensor into a one-dimensional feature vector in a fixed order. The face feature vector is used to represent the face and is stored together with the captured attribute data. After obtaining the face feature vector, the system determines the range of personnel registration database to be accessed for this comparison based on the fields related to personnel database selection in the captured attribute data. It reads the registration feature vector corresponding to the candidate identity from the personnel registration database. For each registration feature vector, the system performs a dimension-wise multiplication and summation of the face feature vector and the registration feature vector, calculates the inner product of the two vectors, and simultaneously performs vector norm calculation on both vectors. The system sums the squares of all components of each vector and takes the square root to obtain the vector norm. After obtaining the inner product and norm, the system divides the inner product by the product of the two norms according to the cosine similarity calculation formula to obtain the cosine similarity value between the face feature vector and each registered feature vector. The cosine similarity can be normalized as needed, mapping the value range from [-1,1] to [0,1]. The calculated similarity result is used as the similarity data.

[0039] The system reads the corresponding similarity data, facial feature vectors, and capture attribute data from the similarity data cache, feature vector cache, and capture attribute data cache, respectively. Reading similarity data includes indexing the similarity data cache and obtaining the corresponding similarity value. Reading facial feature vectors involves performing vector indexing on the feature vector cache to obtain all dimension values. Reading capture attribute data includes obtaining time information, size information, camera address information, event identifier information, and image quality information. Field processing is then performed on the similarity data, facial feature vectors, and capture attribute data to convert the similarity data into numerical data. The system first organizes feature vectors into fixed-dimensional vector fields and capture attribute data into multiple attribute fields according to a preset field order. Then, it performs field filling according to a preset data structure template, filling similarity data into the matching degree field, facial feature vectors into the feature expression field, and each attribute field of the capture attribute data into its corresponding event attribute field. Based on the data structure template after field filling, it performs field combination processing, sequentially concatenating the matching degree field, feature expression field, and event attribute field to form recognition result data containing the matching degree field, feature expression field, and event attribute field.

[0040] Those skilled in the art can perform face region extraction, pose correction, feature extraction, and similarity calculation on the data to be identified based on the specific algorithm steps described above, thereby reproducing the recognition processing flow of this embodiment and obtaining the recognition result data in engineering implementation.

[0041] In one embodiment of the present invention, based on step S4, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0042] Based on preset judgment rules, the recognition result data is judged to obtain identity judgment data, including: Read the similarity data from the recognition results data to obtain the similarity value; The feature vector of the face is read from the recognition result data to obtain the personnel identification information corresponding to the feature vector; The capture attribute data is read from the recognition result data to obtain the time information, size information, address information, event identifier information and quality information in the capture attribute data; The similarity value is compared with the similarity threshold in the preset judgment rule to obtain the judgment result, including: normalizing the similarity and converting the similarity through a mapping formula to obtain a normalized similarity value; comparing the normalized similarity value with the similarity threshold in the judgment rule: when the normalized similarity value is greater than or equal to the similarity threshold, it is judged as a match; when the normalized similarity value is less than the similarity threshold, it is judged as a mismatch.

[0043] Specifically, the output feature vector is denoted as The system pre-stores at least one feature vector extracted by a similar network for each identity in the face database (personnel headshot materials exported from the security system), denoted as . , , where N is the number of registered identities in the face database.

[0044] During the comparison phase, the similarity between the feature vector to be tested and all vectors in the face database is calculated. This invention uses cosine similarity as the metric. Cosine similarity is defined as follows:

[0045] The theoretical range of S is [ [1,1]. To facilitate system judgment and standardize the threshold format, this invention performs interval mapping on the similarity to obtain a normalized similarity. The definition is as follows:

[0046] After mapping, The range of values ​​for is:

[0047] After this transformation, the similarity scores of this system uniformly fall within the [0,1] interval, making the threshold judgment logic more intuitive. The system compares the highest similarity score with the preset threshold. The system performs a comparison. The system determines that the input face matches the identity when the following conditions are met:

[0048] If the above conditions are not met, the result is judged as "mismatch" or "unknown identity". The similarity score threshold is an empirical parameter, which can be set as follows: .

[0049] The personnel identification information corresponding to the feature vector is read and processed to obtain the personnel identification information associated with identification; the fields of the captured attribute data are extracted, and the time information, size information, address information, event identification information and quality information in the captured attribute data are extracted as attribute entries respectively; the judgment result, personnel identification information and various attribute entries are placed in order according to the preset data structure to generate identity judgment data.

[0050] Specifically, the security system's backend program assembles image information and sends an HTTP POST request to invoke the AI ​​facial recognition service to perform personnel identification. Request data format: { faceImage: Avatar image binary stream} Returned data format: { code: Status code / / 200, 500 msg: Returned information description / / Request success or failure information data: { / / Recognition results faceId: Face ID (employee ID) similarity: degree of similarity faceBlur: The degree of blur in a face image. pitch: angle of elevation roll: roll angle yaw: yaw angle } } After receiving the data returned by the AI ​​service, the background program first parses the returned results (employee ID faceId, similarity, face pose angle pitch / roll / yaw, etc.), and then combines it with the device information (camera IP address, camera installation location) obtained from the camera callback event information in step 1 to assemble a complete capture and recognition record (camera IP, camera location, capture time, employee ID, similarity, face pose angle, recognition status, etc.). The background program judges the recognition result according to the recognition judgment rules, and the final recognition result is used for database entry or subsequent early warning business. Security system preset identification rule configuration parameters and thresholds: Similarity (75%): A threshold used to judge the recognition result; ≥75% indicates a successful match with the face database; Pitch angle (20°): The threshold used to determine the angle of looking up or down; Roll angle (20°): A threshold used to determine the angle of left and right head tilt; Yaw angle (20°): The threshold used to determine the angle at which the head turns left or right; Image sharpness (2): Image blur score threshold, the larger the value, the more blurred the image; After parsing and identifying the results, the background program will make a judgment based on the above configuration items and mark them as "internal personnel", "external personnel" or "unknown personnel".

[0051] The specific implementation logic for identifying internal and external personnel is as follows: Criteria for determining internal personnel: Similarity ≥ 75%; Criteria for determining migrant workers (outsiders): Similarity < 75%, and simultaneously meeting the following conditions: Image sharpness level <= 2; Pitch angle, roll angle, and yaw angle ≤|±20°|; Unknown persons (image quality is poor, lacking sufficient basis for judgment, requiring subsequent manual intervention) determination criteria: Similarity < 75%, and meets any of the following conditions: Image sharpness level >2; Significant angular deviations (pitch, roll, yaw > |±20°|). The security system's backend program completes the storage of camera-captured and recognized information and determines whether to trigger alerts or other business processes. After completing the image analysis and personnel identification, the system's backend program generates camera capture and recognition information by combining the recognition results with camera information and stores it in the database. After storage, the system determines whether to trigger an alert based on preset recognition rules and executes the corresponding business processes.

[0052] The format of the captured and recognized information entered into the database: captureTime: The time when information is entered into the database; receivingTime: The time it takes for the background to receive the captured data; captureDevIp: Camera IP address; captureDevDir: Camera location; facePicDir: The path to the captured face image; faceBlur: Blur level in a face image; Similarity: Comparison of similarity scores; faceId: Employee ID; state: Status indicator; (0: internal personnel; 1: external personnel; 2: unknown) eventState: Processing status; (0: Unprocessed; 1: Processed) outsider: Has the person been verified as an outsider? handlingOpinions: Recommendations for handling the issue; Implementation logic of the visitor warning pop-up window The system uses a polling mechanism to continuously send alerts to personnel in charge of security, displaying the alert interface. The specific logic is as follows: 1. The browser client periodically requests the backend to query the latest 3 unprocessed non-internal personnel snapshot records in a polling manner (every 5 seconds) (state: 1 or 2, eventState: 0). 2. If new data is found, a warning window will pop up on the client interface; 3. The warning pop-up window displays information including: captured face image, camera location, capture time, and similarity score; 4. Click Details / Process to verify and process the alert information: verify the identity of the personnel (for external personnel: set the outsider field to 1; for internal personnel: set the outsider field to 0), fill in the processing content, and click Submit to update the processing status (set the eventState field to 1). Note: Once it is confirmed that an unauthorized person has intruded, manual intervention should be carried out.

[0053] Information on personnel of the security system management company; each person can upload 1-5 headshots (.jpeg format); The personnel material library export function: Name the avatar materials in the format of "employee number_number subscript.jpeg" and package them into a folder, and finally generate a compressed file; All the source files in this compressed file serve as a face database for the AI ​​facial recognition service: the files themselves are used as face sources, and the filename prefix is ​​used as a tag. When the avatar material library is updated (for employee onboarding / offboarding or batch uploading / updating of avatars), the material library should be re-exported and handed over to algorithm engineers to update the face database.

[0054] In some embodiments, the intruder detection system may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the intruder detection system may be stored in the memory of a computer device and executed by at least one processor to perform (see details). Figure 1 (Description) Function for identifying unauthorized personnel intrusion.

[0055] In this embodiment, the intruder identification system can be divided into multiple functional modules based on its functions, such as... Figure 2 As shown. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, and is stored in memory. In this embodiment, the functions of each module will be described in detail in subsequent embodiments.

[0056] The data acquisition module is used to acquire the capture event data of the camera, which includes captured image data and capture attribute data; The structure mapping module is used to perform data structure mapping on the captured image data and captured attribute data respectively to generate the data to be recognized. The result recognition module is used to extract face regions, correct poses, and extract features from the data to be recognized, thereby generating similarity data and face feature vectors. Based on the similarity data, face feature vectors, and capture attribute data, the recognition result data is obtained. The identity determination module is used to determine the identity determination data based on preset determination rules.

[0057] In one possible implementation, the result recognition module includes: The face region generation unit reads the brightness of each pixel in the captured image data, analyzes the edge response of the read pixel brightness based on the Sobel gradient operator, and then performs boundary convergence calculation on the gradient changes of continuous pixels to generate face region data that represents the range of the face region. The face input data generation unit is used to perform pose correction processing on the face region data. It calculates the affine transformation matrix by detecting the relative geometric relationship between the center points of the left and right eyes, the tip of the nose and the corner of the mouth in the face region data, and performs two-dimensional rotation and proportional resampling on the face region data to obtain aligned face input data. The face feature vector extraction unit is used to perform feature mapping processing on the aligned face input data. It extracts feature vectors to represent the face by performing local convolution kernel sliding calculation, feature accumulation in the channel direction, dimensionality compression of the feature tensor, and vector flattening on the image grid. The similarity data generation unit is used to determine the range of the personnel registration database to be compared based on the captured attribute data, read the registration feature vector from the personnel registration database, and perform dimension-wise product summation, vector norm solution and cosine ratio calculation on the feature vector of the face and the registration feature vector to obtain similarity data. The result recognition unit is used to fill similarity data into the matching degree field, fill facial feature vectors into the feature expression field, and fill each attribute field of the captured attribute data into the corresponding event attribute field based on a preset data structure template. Based on the filled data structure template, field combination processing is performed, and the matching degree field, feature expression field, and event attribute field are sequentially concatenated to obtain recognition result data containing the matching degree field, feature expression field, and event attribute field.

[0058] Figure 3The intrusion detection method provided in the embodiments of this application can be applied to devices. Those skilled in the art will understand that the device structure involved in the embodiments of this invention does not constitute a limitation on the device. A device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the device includes, but is not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0059] The device 300 may include a processor 310, a memory 320, and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0060] The memory 320 can be used to store execution instructions of the processor 310. The memory 320 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 320 are executed by the processor 310, the device 300 is able to perform some or all of the steps in the above method embodiments.

[0061] The processor 310 serves as the control center of the storage device, connecting various parts of the electronic device via various interfaces and lines. It executes software programs and / or modules stored in the memory 320, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 310 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0062] The communication unit 330 is used to establish a communication channel, enabling the storage device to communicate with other devices, such as acquiring user data sent by other devices or sending user data to other devices.

[0063] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps provided in the embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0064] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or any other medium capable of storing program code. It includes several instructions to cause a computer device (which may be a personal computer, a server, or a second device, network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0065] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0066] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0067] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0068] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0069] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A method for identifying unauthorized personnel intrusion, characterized in that, include: Acquire snapshot event data from the camera, the snapshot event data including snapshot image data and snapshot attribute data; Data structure mapping is performed on the captured image data and captured attribute data respectively to generate data to be identified; The face region is extracted, pose is corrected, and features are extracted from the data to be identified, resulting in similarity data and face feature vectors. Based on the similarity data, face feature vectors, and capture attribute data, the recognition result data is obtained. Based on preset judgment rules, the recognition result data is judged to obtain identity judgment data.

2. The method according to claim 1, characterized in that, Acquire camera capture event data, which includes captured image data and captured attribute data, including: Based on the callback mechanism pre-set by the camera, the capture event data sent by the camera after the capture event is triggered is read, the binary content of the image in the capture event data is extracted, and the binary content of the image is used as the capture image data. The attribute fields in the captured event data are parsed, the image generation time field is parsed into time information, the image size field is parsed into size information, the camera address field is parsed into address information, the event number field is parsed into event identification information, and the image quality related fields are parsed into quality information to obtain the captured attribute data; The captured image data is temporarily stored in the image buffer area, the captured attribute data is temporarily stored in the attribute buffer area, and the correspondence between the captured image data and the captured attribute data is saved.

3. The method according to claim 1, characterized in that, Data structure mapping is performed on the captured image data and captured attribute data respectively to generate data to be recognized, including: The captured image data is used as the image content input, and the captured attribute data is used as the attribute content input. Data structure mapping is performed on the image content input and the attribute content input respectively, mapping the image content input to the image input field and the attribute content input to the attribute input field. The image input field and the attribute input field are combined to obtain the data to be identified, which includes the image input part and the attribute input part. The data to be identified corresponds to the data entity after the structured processing of the captured event data.

4. The method according to claim 1, characterized in that, The process involves extracting face regions, correcting pose, and extracting features from the data to be recognized, resulting in similarity data and facial feature vectors. Based on the similarity data, facial feature vectors, and captured attribute data, the recognition result data is obtained, including: The brightness of the captured image data is read pixel by pixel. The edge response of the read pixel brightness is analyzed based on the Sobel gradient operator. Then, the gradient change of the continuous pixels is calculated to generate face region data that represents the range of the face region. The face region data is processed for pose correction. The affine transformation matrix is ​​calculated by detecting the relative geometric relationship between the center points of the left and right eyes, the tip of the nose and the corner of the mouth in the face region data. The face region data is then rotated in two dimensions and resampled proportionally to obtain the aligned face input data. The aligned face input data is processed by feature mapping. By performing local convolution kernel sliding calculation, channel direction feature accumulation, feature tensor dimension compression and vector flattening on the image grid, feature vectors for representing the face are extracted. Based on the captured attribute data, the range of the personnel registration database to be compared is determined. The registration feature vector is read from the personnel registration database. The feature vector of the face and the registration feature vector are multiplied and summed in one dimension, the vector norm is solved and the cosine ratio is calculated to obtain the similarity data. Based on the preset data structure template, similarity data is filled into the matching degree field, facial feature vectors are filled into the feature expression field, and the attribute fields of the capture attribute data are filled into the corresponding event attribute fields. Based on the filled data structure template, field combination processing is performed, and the matching degree field, feature expression field, and event attribute field are sequentially concatenated to obtain recognition result data containing the matching degree field, feature expression field, and event attribute field.

5. The method according to claim 1, characterized in that, Based on preset judgment rules, the recognition result data is judged to obtain identity judgment data, including: Read the similarity data from the recognition results data to obtain the similarity value; The feature vector of the face is read from the recognition result data to obtain the personnel identification information corresponding to the feature vector; The capture attribute data is read from the recognition result data to obtain the time information, size information, address information, event identifier information and quality information in the capture attribute data; The similarity value is compared with the similarity threshold in the preset judgment rule to obtain the judgment result; The personnel identification information corresponding to the feature vector is read and processed to obtain the personnel identification information associated with the identification. Field extraction is performed on the snapshot attribute data, and the time information, size information, address information, event identifier information and quality information in the snapshot attribute data are extracted into attribute entries respectively; The judgment results, personnel identification information, and various attribute entries are arranged in order according to a preset data structure to generate identity judgment data.

6. The method according to claim 5, characterized in that, The similarity values ​​are compared with the preset similarity thresholds in the judgment rules to obtain the judgment results, including: The similarity is normalized by converting it using a mapping formula to obtain a normalized similarity value. Perform a comparison between the normalized similarity value and the similarity threshold in the decision rule: A match is determined when the normalized similarity value is greater than or equal to the similarity threshold. When the normalized similarity value is less than the similarity threshold, it is judged as a mismatch.

7. An intrusion detection system for unauthorized personnel, characterized in that, include: The data acquisition module is used to acquire the capture event data of the camera, which includes captured image data and capture attribute data; The structure mapping module is used to perform data structure mapping on the captured image data and captured attribute data respectively to generate the data to be recognized. The result recognition module is used to extract face regions, correct poses, and extract features from the data to be recognized, thereby generating similarity data and face feature vectors. Based on the similarity data, face feature vectors, and capture attribute data, the recognition result data is obtained. The identity determination module is used to determine the identity determination data based on preset determination rules.

8. The system according to claim 7, characterized in that, The result recognition module includes: The face region generation unit reads the brightness of each pixel in the captured image data, analyzes the edge response of the read pixel brightness based on the Sobel gradient operator, and then performs boundary convergence calculation on the gradient changes of continuous pixels to generate face region data that represents the range of the face region. The face input data generation unit is used to perform pose correction processing on the face region data. It calculates the affine transformation matrix by detecting the relative geometric relationship between the center points of the left and right eyes, the tip of the nose and the corner of the mouth in the face region data, and performs two-dimensional rotation and proportional resampling on the face region data to obtain aligned face input data. The face feature vector extraction unit is used to perform feature mapping processing on the aligned face input data. It extracts feature vectors to represent the face by performing local convolution kernel sliding calculation, feature accumulation in the channel direction, dimensionality compression of the feature tensor, and vector flattening on the image grid. The similarity data generation unit is used to determine the range of the personnel registration database to be compared based on the captured attribute data, read the registration feature vector from the personnel registration database, and perform dimension-wise product summation, vector norm solution and cosine ratio calculation on the feature vector of the face and the registration feature vector to obtain similarity data. The result recognition unit is used to fill similarity data into the matching degree field, fill facial feature vectors into the feature expression field, and fill each attribute field of the captured attribute data into the corresponding event attribute field based on a preset data structure template. Based on the filled data structure template, field combination processing is performed, and the matching degree field, feature expression field, and event attribute field are sequentially concatenated to obtain recognition result data containing the matching degree field, feature expression field, and event attribute field.

9. A device, characterized in that, include: Memory used to store intrusion detection programs for unauthorized personnel; A processor, configured to implement the steps of the intrusion detection method as described in any one of claims 1-6 when executing the intrusion detection program.

10. A computer-readable storage medium storing a computer program, characterized in that, The readable storage medium stores an intrusion detection program for unauthorized personnel, which, when executed by a processor, implements the steps of the intrusion detection method for unauthorized personnel as described in any one of claims 1-6.

Images