A face verification method and system based on feature vector comparison

Abstract

This invention discloses a face verification method and system based on feature vector comparison, belonging to the field of image detection technology. The method includes: receiving images of important individuals uploaded by users, and users establishing verification tasks; performing multimodal feature extraction and fusion on the selected important individual images to generate enhanced individual feature vectors; downloading material copies within the verification time period to a temporary area according to the verification task, using a lightweight face detection network with multi-scale fusion to detect faces frame by frame, simultaneously extracting scene features and encoding them to generate material face feature vectors, fusing them to form a multimodal fusion feature vector; determining a matching threshold according to the material copy type, comparing the material face feature vectors with the enhanced individual feature vectors layer by layer, combining scene features for auxiliary judgment and automatic labeling, and storing the results in the verification task database; deleting materials from the temporary area, generating a task report with a verification code, improving verification accuracy and result security.

Description

Technical Field

[0001] This invention belongs to the field of image detection technology, specifically a face verification method and system based on feature vector comparison. Background Technology

[0002] In today's era of digital information explosion, a massive amount of video and image material is constantly being generated. Quickly and accurately identifying specific important individuals from this material is a challenging yet crucial task. Traditional facial recognition methods often suffer from low accuracy, low processing efficiency, and difficulty in handling complex scenarios. For example, some methods based on simple feature matching are easily affected by factors such as lighting, angle, and occlusion, leading to a high false positive rate. Meanwhile, some review methods requiring extensive manual intervention not only consume significant manpower and time but also struggle to guarantee the consistency and objectivity of the review results. With the continuous development of artificial intelligence technology, deep learning-based facial recognition algorithms have made significant progress. However, how to effectively apply these advanced algorithms to the actual facial recognition process to achieve automated, efficient, and accurate facial recognition remains a pressing problem. Therefore, this invention proposes a facial recognition method and system based on feature vector comparison to address the aforementioned problems in existing technologies. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention proposes a face verification method and system based on feature vector comparison. The system receives images of important individuals uploaded by users, who then create verification tasks. Multimodal feature extraction and fusion are performed on the selected images to generate enhanced individual feature vectors. Based on the verification task, copies of materials within the verification period are downloaded to a temporary area. A lightweight face detection network with multi-scale fusion is used to detect faces frame by frame, simultaneously extracting scene features and encoding them to generate material face feature vectors. These are then fused to form a multimodal fusion feature vector. A matching threshold is determined according to the type of material copy, and the material face feature vectors are compared layer by layer with the enhanced individual feature vectors. Scene features are used to assist in judgment and automatic labeling. The results are stored in the verification task database. Materials in the temporary area are deleted, and a task report with a verification code is generated, improving verification accuracy and result security.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A face verification method based on feature vector comparison includes:

[0006] Users can upload images of important figures through the important figures database in the management center. At the same time, they can create review tasks through the tagging module, select the time period to be reviewed and the uploaded images of important figures.

[0007] Multimodal feature extraction and fusion are performed on selected images of important figures to generate enhanced feature vectors of the figures;

[0008] Based on the review task set by the user, download the material copies within the time period to be reviewed to the temporary area. Use a lightweight face detection network with multi-scale fusion to perform frame-by-frame face detection on the material copies in the temporary area, extract scene features of the material copies at the same time, and encode and convert each detected face image to obtain the material face feature vector. Then, fuse the material face feature vector with the scene features to form a multimodal fusion feature vector.

[0009] The matching threshold is determined based on the type of material copy. The face feature vector of the material in the obtained multimodal fusion feature vector is compared with the enhanced human feature vector in layers. The scene features are combined to assist in the judgment. The automatic labeling is performed based on the comparison results, and the comparison results and labeling information are stored in the review task database.

[0010] The temporary area material copy is deleted and the inode is released using the standard file system deletion command. The process is iterated until all materials have been processed, and a task report with a checksum is generated.

[0011] Specifically, the establishment of the review task through the labeling module includes:

[0012] The system calls the tagging module to receive the review task parameters input by the user. The review task parameters include the task name, the time period range to be reviewed, and one or more important person image identifiers selected from the important person database. The important person database is a database formed after the user uploads important person images through the management center, which stores important person images and their corresponding unique identifiers.

[0013] After verifying the legality of the received time period range to be reviewed, the system generates a review task identifier, associates the review task parameters with the review task identifier, and stores them in the pending task table of the review task database.

[0014] Specifically, the process of generating the enhanced character feature vector includes:

[0015] The system selects the image identifier of the important person corresponding to the important person based on the audit task parameters, and reads the image of the important person that uniquely corresponds to the image identifier from the important person database;

[0016] The received images of important figures to be reviewed are input into the RetinaFace model. The RetinaFace model identifies and crops the facial regions in the images of important figures to be reviewed, and obtains the cropped facial images.

[0017] The cropped face image is input into the CurricularFace model. The CurricularFace model extracts features and transforms vectors from the cropped face image to obtain the face feature vector of the important person to be reviewed.

[0018] A bio-inspired spiking neural network is used to enhance the facial feature vectors of important figures to be reviewed, generating enhanced feature vectors of the figures, which are then associated with the image identifiers of the important figures and stored in the feature database of important figures.

[0019] Specifically, the enhancement process of the bio-inspired spiking neural network includes:

[0020] Construct a sandwich structure comprising a time-coding layer, a pulse firing layer, and a synaptic plasticity modulation layer;

[0021] The facial feature vector of the important person to be reviewed is input into the time coding layer. The time coding layer assigns one time coding neuron to each dimension of the facial feature vector of the important person to be reviewed, converts the feature values ​​of each dimension into time series pulse signals, and weights all time series pulse signals through an exponential decay function to form a pulse sequence. Then, the pulse sequences of each dimension are time-aligned to form a standardized pulse sequence.

[0022] The time-coding layer transmits the generated standardized pulse sequence to the pulse firing layer. The pulse firing layer calculates the membrane potential for each dimension's standardized pulse sequence, with the initial value of the membrane potential being 0. When the membrane potential exceeds a preset threshold, the corresponding dimension's neuron fires a pulse and resets the membrane potential to the resting potential. The total number of pulse firings for each dimension is counted, and the ratio of the number of firings in a single dimension to the total number of firings in all dimensions is used as the weight of that dimension. Through weight filtering, the filtered pulse sequence and the weight parameters for each dimension are obtained.

[0023] The pulse firing layer inputs the filtered pulse sequence and weight parameters of each dimension into the synaptic plasticity adjustment layer. The synaptic plasticity adjustment layer uses the pulse time-dependent plasticity rule to adjust the synaptic weights according to the time difference between adjacent pulses to obtain the fused feature vector.

[0024] The synaptic plasticity modulation layer performs normalization on the generated fused feature vector, ultimately generating an enhanced character feature vector.

[0025] Specifically, the lightweight face detection network employing multi-scale fusion performs frame-by-frame face detection on the material copies in the temporary region, including:

[0026] The system reads a copy of the material to be processed from the temporary area and parses the copy frame by frame to extract a continuous sequence of image frames.

[0027] A spatiotemporal attention-guided detection mechanism is introduced to construct spatiotemporal feature maps for continuous image frame sequences. Intra-frame spatial attention weights and inter-frame temporal attention weights are calculated using a spatiotemporal attention approach, and these weights are integrated into a lightweight face detection network with multi-scale fusion. The spatial attention weights are used to highlight key features of the face region in a single frame image, while the temporal attention weights are used to associate face motion features between adjacent frames.

[0028] A lightweight face detection network with multi-scale fusion incorporating a spatiotemporal attention-guided detection mechanism is applied to each extracted image frame for face detection. By locating all face regions contained in each image frame, corresponding bounding box coordinates and confidence scores are generated for each detected face region.

[0029] The generated face detection results are filtered according to a preset confidence threshold, and face regions with confidence scores higher than the confidence threshold and their corresponding bounding box coordinates are retained to obtain the face detection results.

[0030] Specifically, the process of determining the matching threshold includes:

[0031] Establish a mapping table between material copy type and matching threshold to store the initial matching threshold corresponding to different material copy types;

[0032] Read the material copy type identifier in the review task and retrieve the corresponding initial matching threshold from the material copy type-matching threshold mapping table;

[0033] By combining the scene features of the extracted frames from the material copy, the initial matching threshold is dynamically adjusted to obtain the final adapted matching threshold.

[0034] Specifically, the obtained facial feature vector of the source material is compared with the enhanced human feature vector in a hierarchical manner, including:

[0035] An inverted index is pre-constructed for all enhanced character feature vectors; the inverted index uses the dimension value of the enhanced character feature vector as the index key, and each index key is associated with a set of enhanced character feature vectors containing the corresponding dimension value;

[0036] Once a multimodal fusion feature vector is obtained, the top N dimensions with the highest values ​​in the face feature vector of the source material are extracted as key dimension values. Based on the constructed inverted index, a search is performed to select an enhanced human feature vector set containing the key dimension values ​​as a preliminary candidate set.

[0037] Calculate the cosine similarity and Manhattan distance between the face feature vector of the source material and each enhanced person feature vector in the preliminary candidate set. Sort the results in descending order according to the weighted calculation of the two, and select the top K enhanced person feature vectors to form a subset of candidate feature vectors.

[0038] The obtained candidate feature vector subsets are weighted using Euclidean distance, Mahalanobis distance, and Bach distance to calculate the precise similarity between the source face feature vector and each candidate vector and then sort them. At the same time, scene features in the multimodal fusion feature vector are combined for auxiliary judgment to obtain the final similarity ranking result.

[0039] If the exact similarity value ranked first in the final similarity ranking results is greater than or equal to the preset matching threshold, it is determined that the face in the material is successfully matched with the corresponding important person, and a comparison result containing the person's identifier and exact similarity value is generated. At the same time, the corresponding person's identifier is tagged for the copy of the frame material.

[0040] If the exact similarity value of the first-ranked result in the final similarity ranking is less than the preset matching threshold, it is judged as a matching failure and marked with an unknown or reserved special label.

[0041] The comparison results and tagging information are associated, encapsulated, and stored in the review task database; the tagging information includes timestamp, material file ID, frame number, and tag content; the review task database adopts a strategy combining time-series partitioning and hash table partitioning.

[0042] Specifically, the audit task database adopts a strategy combining time-series partitioning and hash-based table partitioning, including:

[0043] The number of tables in the audit task database is preset to M;

[0044] When a new review task is generated, the system obtains the unique identifier of the review task as the partition key of the hash partition table, and uses the hash partition algorithm to perform hash calculation on the review task identifier to obtain the hash value.

[0045] The obtained hash value is moduloed with the preset number of table partitions M, and the table partition where the audit task is stored is determined based on the modulo result.

[0046] Within the defined partitioned tables, a time-based partitioning strategy is adopted, with each month serving as the partitioning unit to store the comparison results and labeling information generated each month into the corresponding month's partitioned table.

[0047] Create a table partitioning routing table to record the correspondence between the audit task identifier range and the partitioned table and the sub-table. Output the table partitioning routing table and the audit task database after partitioning.

[0048] Specifically, the step of deleting the temporary area material copy and releasing the inode using the standard file system deletion command includes:

[0049] After completing all frame processing and face comparison for a media copy, the system calls the storage management service to send a standard file system deletion command to the media copy in the temporary area, notifying the storage device to perform a file deletion operation. At the same time, it releases the file system inode, updates the temporary area storage status table, and marks the media copy as processed. The temporary area storage status table is a table that records the processing status of each media copy in the temporary area. The file system inode is a data structure used to record the storage location, size attributes, and access permissions of the media copy files.

[0050] A face verification system based on feature vector comparison includes: a management center module, a feature fusion module, a material detection module, a tagging module, and a report generation module;

[0051] The management center module is used to receive images of important figures uploaded by users and build a database, and is also used to review the initial parameter configuration and storage of tasks.

[0052] The feature fusion module is used to preprocess, perform model calculations and feature fusion on images of important figures to generate enhanced feature vectors of the figures.

[0053] The material detection module is used to download copies of the materials to be reviewed and parse them frame by frame. It completes face detection through a lightweight face detection network that integrates a spatiotemporal attention-guided detection mechanism and extracts scene features and generates face feature vectors for the materials.

[0054] The tagging module is used to determine the appropriate matching threshold, calculate the similarity between the face feature vector of the material and the enhanced human feature vector through hierarchical comparison, combine scene features to assist in judgment and complete automatic tagging, and store the results in the review task database.

[0055] The report generation module is used to generate task reports with verification codes based on data from the audit task database.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] 1. This invention proposes a face verification method based on feature vector comparison. It generates enhanced feature vectors by extracting and fusing multimodal features from images of important figures. Simultaneously, a lightweight face detection network with multi-scale fusion is used to detect faces in the source material frame by frame and fuse scene features to form a multimodal fusion feature vector. A matching threshold is dynamically determined based on the source material copy type to conduct hierarchical comparison. This method can accurately capture key facial features and scene-related information, significantly reducing the probability of false positives and false negatives. The lightweight detection network and efficient processing mechanism for temporary source material also effectively improve the overall verification speed, adapting to the needs of large-scale source material verification.

[0058] 2. This invention proposes a face verification method based on feature vector comparison, which provides strong protection for data security and process standardization. It uses deletion commands to promptly delete temporary area material copies and release index nodes, avoiding redundant data from occupying storage resources or causing information leakage risks. The verification results and tagging information are stored in the database in an orderly manner, and finally a task report with a verification code is generated, ensuring that the verification process is traceable and the results are tamper-proof, meeting the requirements for secure data storage and improving the credibility of the verification results. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of a face verification method based on feature vector comparison according to the present invention;

[0060] Figure 2 This is a flowchart illustrating the principle of a face verification method based on feature vector comparison according to the present invention.

[0061] Figure 3 This is an architecture diagram of a face verification system based on feature vector comparison according to the present invention. Detailed Implementation

[0062] Example 1

[0063] Please see Figure 1 and Figure 2 The present invention provides an embodiment of a face verification method based on feature vector comparison, the method comprising S1 to S5:

[0064] S1: Users upload images of important figures through the important figures database in the management center. At the same time, they create review tasks through the tagging module, selecting the time period to be reviewed and the uploaded images of important figures.

[0065] S2: Perform multimodal feature extraction and fusion on selected important figures' images to generate enhanced figure feature vectors;

[0066] S3: Based on the review task established by the user, download the material copies within the time period to be reviewed to the temporary area, use a lightweight face detection network with multi-scale fusion to perform frame-by-frame face detection on the material copies in the temporary area, extract scene features of the material copies at the same time, and encode and convert each detected face image to obtain the material face feature vector, and fuse the material face feature vector with the scene features to form a multimodal fusion feature vector;

[0067] Furthermore, the facial feature vectors of the important figures to be reviewed and the facial feature vectors of the source material have a dimension of 512 and a modulus of 1.

[0068] Furthermore, the synchronous extraction of scene features from the source material replicas involves analyzing the non-face regions and overall image attributes of the current frame by using a lightweight face detection network with multi-scale fusion to detect faces frame by frame within the same frame processing cycle. This is achieved by parallel invocation of the feature generation unit, which in turn analyzes the scene change information of adjacent frames. Ultimately, this generates feature data that reflects the scene environment of the current frame, ensuring that scene feature extraction, face detection, and source material face feature vector generation are synchronized in time, and that the scene features accurately correspond to the source material face feature vectors. Specifically, this includes:

[0069] (1) After the system reads the copy of the material to be processed from the temporary area and parses the image frame sequence frame by frame, when any frame image enters the lightweight face detection network of multi-scale fusion for face detection, the system will synchronously copy the frame image through the data distribution interface and transmit it to the feature generation unit to ensure that face detection and scene feature extraction are based on the same original data of the same frame, and avoid synchronization deviation caused by data transmission delay or version difference.

[0070] (2) The feature generation unit preprocesses the received frame image. First, it performs image resolution normalization to adjust the image size to a fixed size. Then, it performs color space conversion to convert the RGB color space to the HSV color space. Because the HSV space can more clearly distinguish hue, saturation and brightness, it is more in line with the human eye's perception logic of scene color and brightness, which is convenient for subsequent extraction of scene color features. Then, it performs noise removal operation and uses Gaussian filtering algorithm to smooth the converted image, filter out random noise generated during image acquisition or transmission, and avoid noise interference with the accurate extraction of scene features to obtain the preprocessed frame image. Among them, normalization, HSV color space and Gaussian filtering algorithm are existing technologies in this field and are not creative solutions of this application. They will not be described in detail here.

[0071] (3) Static scene feature extraction; The static scene feature extraction focuses on the inherent attributes of the current single frame image, mainly including four dimensions: The first dimension is scene color feature extraction. First, the image in HSV space is divided into 12 intervals according to hue. The pixel ratio of each hue interval in the image is counted. Then, the mean and variance of saturation and the mean and variance of brightness are calculated. The hue ratio, saturation statistics, and brightness statistics are integrated to form color feature data. The second dimension is scene texture feature extraction. The brightness channel of HSV space is converted into a grayscale image using the gray-level co-occurrence matrix algorithm. Then, multiple non-overlapping local regions are selected in the grayscale image. For each local region, four key parameters of the gray-level co-occurrence matrix are calculated, such as contrast, correlation, energy, and homogeneity. Contrast reflects the degree of difference in gray levels in the local region. For example, a scene with rough texture has high contrast. Correlation reflects the degree of association of gray levels in the local region. Energy reflects the uniformity of texture. Homogeneity reflects the degree of similarity of gray levels in the local region. Then, for all local regions The four parameters are averaged to obtain the texture feature data of the entire frame image; the third dimension is scene structure feature extraction, which extracts edge information in the image through edge detection algorithm, identifies the main contours in the image, and then counts the number, length and distribution density of these contours, while judging the preliminary characteristics of the scene type formed by the contours, such as a large number of straight contours may correspond to indoor or architectural scenes, and irregular curve contours may correspond to natural scenes, forming structural feature data; the fourth dimension is scene lighting feature extraction, which combines the mean and variance of the brightness channel to further analyze the distribution of highlight areas and shadow areas in the image, counts the area ratio of highlight areas, the area ratio and shape of shadow areas, and calculates the uniformity of lighting to form lighting feature data. Finally, the color feature data, texture feature data, structural feature data and lighting feature data are integrated to form the static scene features of the current frame. Among them, the mean and variance, gray-level co-occurrence matrix algorithm and edge detection algorithm are existing technologies in this field and are not inventive solutions of this application, and will not be described in detail here;

[0072] (4) Perform dynamic scene feature extraction; the dynamic scene feature extraction mainly relates the scene changes between the current frame and adjacent frames, making up for the limitation of static features only reflecting single-frame information. First, the system records the frame number of the current processing frame and retrieves the static scene feature data of the previous frame. The frame number of the previous frame is the current frame number minus one. If the current frame is the first frame of the material copy, the static scene feature data of the previous frame is set to be consistent with the initial static scene feature data of the current frame to avoid the inability to calculate dynamic features when there are no adjacent frames. Then, the difference values ​​of each dimension of the static scene features between the current frame and the previous frame are calculated. For color features, the absolute difference of the hue ratio, the absolute difference of the mean and variance of saturation, and the absolute difference of the mean and variance of brightness are calculated. For textures, the absolute difference of the hue ratio, the absolute difference of the mean and variance of saturation, and the absolute difference of the mean and variance of brightness are calculated. The system calculates the absolute differences of four parameters: contrast, correlation, energy, and homogeneity. For structural features, it calculates the absolute differences of contour quantity, length, and distribution density. For illumination features, it calculates the absolute differences of highlight area proportion, shadow area proportion, and illumination uniformity. Then, it analyzes the changing trends of these differences to determine whether there are dynamic changes in the scene, and simultaneously counts the magnitude and type of scene changes between adjacent frames. Finally, it integrates the difference values, change magnitude, and change type to form the dynamic scene features of the current frame, thereby capturing information about dynamic changes in the scene.

[0073] (5) After obtaining the static scene features and dynamic scene features of the current frame, the two types of features are fused to form complete scene features;

[0074] (6) After completing the extraction of scene features of the current frame and generating standardized scene feature vectors, the system will obtain the face region information and material face feature vectors obtained by face detection of the same frame image. Then, the system will establish an association relationship through the frame number and face identifier, bind the scene feature vector of the current frame with all material face feature vectors corresponding to the frame, and ensure that each material face feature vector can be matched with the scene features of its frame to avoid feature mismatch. At the same time, the associated scene feature vector, material face feature vector and corresponding frame number, face identifier and other information will be temporarily stored.

[0075] S4: Determine the matching threshold based on the material copy type, perform hierarchical comparison between the material face feature vector and the enhanced human feature vector in the obtained multimodal fusion feature vector, combine scene features to assist in judgment, automatically label according to the comparison results, and store the comparison results and labeling information in the review task database;

[0076] Furthermore, by incorporating scene features to aid judgment, automatic labeling is performed based on the comparison results, and the comparison results and labeling information are stored in the review task database, including:

[0077] (1) After completing the hierarchical comparison of the face feature vector and the enhanced person feature vector, the system first retrieves the multimodal fusion feature vector of the current frame, separates the scene feature vector extracted in the previous synchronous process, and obtains the hierarchical comparison result of the frame. Then, the system establishes an association with the face identifier through the frame number to ensure that each hierarchical comparison result can accurately correspond to the scene feature vector of its frame, and avoids the situation where the scene features of different frames are mismatched with the comparison results. For example, if two faces are detected in any frame and two hierarchical comparison results are generated respectively, the system binds the same scene feature vector of the frame to each comparison result.

[0078] (2) Obtain the scene features extracted in S3;

[0079] (3) For different situations of the hierarchical comparison results, the rationality is verified by combining each sub-dimension of the scene features;

[0080] If the layered comparison result is a successful match, then the matching is judged from the perspective of scene features to determine whether it conforms to the actual scene logic. For example, first, the lighting features are checked. If the proportion of the highlight area in the current frame exceeds 0.3 or the proportion of the shadow area exceeds 0.4, it indicates that the face may be overexposed or occluded. At this time, it is necessary to check whether the precise similarity value is at least 0.1 higher than the matching threshold. If it does not reach the threshold, the matching is considered to be risky and needs to be marked as pending verification. Next, the structural features are analyzed. If the number of contours in the current frame exceeds 50 and the distribution density is high, it indicates that there are dense objects in the scene, and there may be face misidentification. It is necessary to confirm whether the bounding box coordinates of the face are in a relatively clear and unobstructed area in the image. If the overlap between the bounding box and other object contours exceeds 0.3, it is also marked as pending verification. Finally, the dynamic scene features are referenced. If the change type is rapid camera movement and the change amplitude exceeds 0.5, it indicates that the image may be blurry. The accuracy of the face encoding conversion needs to be checked again. If a feature missing prompt appears during the encoding process, it is marked as pending verification.

[0081] If the layered comparison result is a match failure, then it is determined whether the misjudgment is due to scene factors: If the lighting features show that the lighting uniformity of the current frame is higher than 0.8, the structural features show that the face bounding box is complete and non-overlapping, and the dynamic scene features do not change significantly, that is, the change range is less than 0.2, it means that the face image is clear and the scene is stable. In this case, if the difference between the precise similarity value and the matching threshold is less than 0.05, it is determined that there may be a missed judgment and it needs to be marked as pending review; if the scene features show that there is serious noise or occlusion in the current frame, such as the texture feature contrast being lower than 0.2, then it is determined that the match failure is reasonable and no additional marking is required.

[0082] (4) After completing the scene feature auxiliary judgment, the system executes the corresponding automatic labeling rules according to the final judgment result. The final judgment result includes successful and reasonable matching, successful matching pending review, failed matching but reasonable matching, and failed matching pending review.

[0083] For successful and reasonable matches, the system generates tags containing important person identifiers, precise similarity values, and timestamps. At the same time, scene feature summaries are added to the tags, such as uniform lighting, stable scene, and valid match.

[0084] For cases where a match is successfully found and awaits review, the tag will include not only the identifier of the important person, the similarity value, and the timestamp, but will also clearly indicate the reason for the review, such as overexposure of highlights or similarity approaching the threshold, and will be automatically assigned to the task list of the reviewer.

[0085] For cases where a match fails but is reasonable, the system will tag the person as an unknown. If the face in that frame has a successful match record in a copy of the material that is not its own, the system will trace the historical detection situation through the tag association.

[0086] For cases where matching fails and needs to be reviewed, the label will indicate the suspected omission and the reason. If the scene is clear and the similarity is close to the threshold, it will also be included in the review process.

[0087] (5) Compare the results and tagging information and encapsulate them in a structured manner, including: extracting the core information from the comparison results, including material file ID, frame number, face identifier, important person identifier, precise similarity value, matching threshold, and key data of each stage of hierarchical comparison; integrating the tagging information, including tag type, tag content, tag timestamp, scene feature summary, and review status; mapping the core information and tagging information in the comparison results to the corresponding fields according to the field specifications of the review task database, for example, the material file ID corresponds to the file_id field in the database, and the tag type corresponds to the tag_type field, to ensure that each information item can accurately match the database field and avoid field mismatch or data loss during storage; performing format verification on the encapsulated data to check whether there are empty fields or data format errors. If there are errors, return to the previous step to re-encapsulate. If the verification passes, generate a structured data package, in which the tag type includes valid match, pending review, unknown person, and suspected missed judgment.

[0088] (6) The system determines the storage location of the structured data packet based on the time-series partitioning and hash table combination strategy adopted by the audit task database. Then, it calls the database write interface to write the structured data packet into the determined partition table. At the same time, the transaction mechanism is started to ensure the atomicity of the data writing process. If an exception occurs during the writing process, the transaction is rolled back to avoid inconsistency caused by partial data writing. If the writing is successful, the transaction is committed.

[0089] (7) After writing, the system builds an index for the data: using the material file ID and frame number as the composite index key, it associates the data with the storage address in the database, and builds separate indexes for important person identifiers and tag type fields; finally, it updates the storage status table of the database to record the storage completion time and index information of the data, ensuring the efficiency of data management and query.

[0090] S5: Use the standard file system deletion command to delete the temporary area material copy and release the inode. Iterate until all materials are processed and generate a task report with a checksum.

[0091] The step of deleting the temporary area material copy and releasing the inode using the standard file system deletion command includes:

[0092] After completing all frame processing and face comparison for a media copy, the system calls the storage management service to send a standard file system deletion command to the media copy in the temporary area, notifying the storage device to perform a file deletion operation. At the same time, it releases the file system inode, updates the temporary area storage status table, and marks the media copy as processed. The storage device in the temporary area includes a mechanical hard disk. The temporary area storage status table is a table that records the processing status of each media copy in the temporary area. The file system inode is a data structure used to record the storage location, size attributes, and access permissions of the media copy files.

[0093] Furthermore, the construction process of the standard file system deletion command includes: the storage management service first queries the logical storage path and file metadata of the material copy in the storage device based on the temporary storage path of the material copy, obtains the inode number corresponding to the material file through the inode association table of the file system, and then locates all storage block association information recorded in the inode to ensure that all storage areas that need to be cleaned can be accurately located; the command content is constructed according to the file deletion command format specification of the corresponding operating system, and the command includes the operation type, the absolute path of the target file, the storage device identifier, and the command execution priority; the file metadata includes the file size, creation time, and access permissions; the storage block association information includes the physical storage block address and the number of blocks.

[0094] Furthermore, the process of notifying the storage device to perform a file deletion operation includes: after the standard file system deletion instruction is constructed, the storage management service sends the deletion instruction to the storage device corresponding to the temporary area through the file system interface; after receiving the instruction, the storage device first verifies the integrity and legality of the instruction, confirming that the target file path exists, the current operating account has file deletion permissions, and the storage device is in a normal read / write state. After the verification is successful, it returns a confirmation signal that the instruction was successfully received; the storage device starts a file cleanup mechanism to erase and mark the file storage block specified by the instruction, and at the same time releases the storage block associated resources to prevent the storage block from being occupied by the file; during the process, the storage device provides real-time feedback on the cleanup progress to the storage management service, such as the number of marked storage blocks and the number of remaining unmarked storage blocks, until all target storage blocks are processed, and returns the result that the deletion instruction has been executed.

[0095] Furthermore, the process of releasing file system inodes includes: while the storage device executes file deletion instructions, the storage management service processes the release operation of file system inodes in parallel to avoid process delays caused by waiting for garbage collection to complete. First, the system queries the file system's inode table for the inode entry corresponding to the media file copy based on the media file ID. This inode entry contains key information such as the media file's storage location, file size, creation time, access permissions, and associated storage block address. The storage management service first marks the inode entry as pending release to prevent accidental operation of the inode by processes other than its own. Then, it deletes the association between the inode entry and the storage block address, clearing the list of storage block addresses recorded in the inode entry to ensure that subsequent allocation of storage blocks for new files does not reference released blocks. Next, it checks whether there are any dependencies associated with the inode. For example, hard links, soft links, and unclosed process file handles are handled as follows: If hard links exist, all hard links are deleted first, and the hard link count is updated; if unclosed process handles exist, a notification is sent to the corresponding process that the file is about to be released, requesting the process to close the handle. After all handles are closed, the inode release operation is performed; finally, the inode access lock is released, the inode entry is marked as free, and added to the system's inode free pool for use by newly created files. At the same time, the file system's inode usage statistics are updated, completing the full release of inodes; the inode usage statistics include the number of free inodes and the number of used inodes.

[0096] Furthermore, after completing the execution of the standard file system delete command and the release of the inode, the storage management service immediately updates the temporary storage status table to ensure that the system can monitor the storage resources of the temporary area in real time. First, the system locates the record entry corresponding to the material copy in the temporary storage status table. This record entry includes fields such as material file ID, temporary storage path, processing status, storage size, cleanup start time, and cleanup completion time. The processing status includes "processing," "pending cleanup," and "completed." The processing status field is updated from "pending cleanup" to "completed," explicitly marking the end of the cleanup process for this material copy. The cleanup completion time field is updated to the current system time, recording the completion time of the cleanup operation. The storage size field is updated to zero, indicating that this material copy no longer occupies temporary storage resources. Simultaneously, cleanup information, including the delete command execution result and the inode release status, is recorded in the cleanup result field. After the update is complete, the update records in the status table are backed up, and the field values ​​before and after the update, the operation account, the operation time, and other information are stored in the system's operation log database, retaining at least three months of log records.

[0097] The process of establishing an audit task through the tagging module includes:

[0098] S1.1: The system calls the tagging module to receive the review task parameters input by the user; the review task parameters include the task name, the time period range to be reviewed, and one or more important person image identifiers selected from the important person database; the important person database is a database formed after the user uploads important person images through the management center, which stores important person images and their corresponding unique identifiers.

[0099] S1.2: After verifying the legality of the received time period range to be reviewed, the system generates a review task identifier, associates the review task parameters with the review task identifier, and stores them in the pending task table of the review task database.

[0100] Furthermore, the system performs legality verification from three dimensions, including: The first dimension is the chronological relationship between the start and end times. The system compares the converted start and end timestamps. If the start timestamp is greater than or equal to the end timestamp, it indicates a logical contradiction that the start time is later than the end time, and the system determines the verification to fail, returning a message that the start time cannot be later than the end time. If the start timestamp is less than the end timestamp, the system proceeds to the second dimension verification. The second dimension is the matching verification between the time period and the system's processable time range. The system's preset processable time range is three years before the current system time to one day after the current system time. Setting three years before the current system time avoids processing overly old historical data, and setting one day after the current system time allows users to pre-process data one day in advance. For material review tasks scheduled for the following day, if the start time of the review period is more than three years earlier than the current system time or the end time is more than one day later than the current system time, the system determines that the verification has failed, returns a prompt that the time period is outside the system's processing range, and informs the user of the valid time range. If it is within the processing range, the system proceeds to the third dimension of verification. The third dimension is the matching verification between the time period and the time range of the stored materials. The system queries the time distribution of the stored materials in the material management database. If there are no stored materials within the review period, such as when no monitoring recordings were generated within the time period selected by the user, the system prompts that there are no materials to be reviewed within that time period and asks the user if they want to adjust the time period. If there are stored materials, the system determines that the logical legality verification of the review period has passed and generates a result indicating that the time period verification has passed.

[0101] The process of generating the enhanced character feature vector includes:

[0102] S2.1: The system selects the image identifier of the important person corresponding to the important person based on the audit task parameters, and reads the image of the important person that uniquely corresponds to the image identifier from the important person database;

[0103] S2.2: Input the received image of the important person to be reviewed into the RetinaFace model. The RetinaFace model recognizes and crops the face region in the image of the important person to be reviewed, and obtains the cropped face image.

[0104] Furthermore, before the received images of important figures to be reviewed are input into the RetinaFace model, they need to be preprocessed, including grayscale conversion, noise filtering, contrast enhancement, and size normalization.

[0105] Furthermore, the face cropping process of the RetinaFace model includes:

[0106] (1) Perform format conversion on the preprocessed images of important figures, including normalizing the pixel values ​​to the range of [0,1] and adding batch dimensions to form a tensor format of [1,3,112,112];

[0107] (2) Multi-scale features are extracted through the ResNet-50 backbone network, which passes through convolutional layers, batch normalization layers, ReLU activation functions and residual blocks in sequence to output feature maps. The ResNet-50 backbone network is existing technology in this field and is not an inventive solution of this application. It will not be described in detail here.

[0108] (3) Generate anchor boxes based on feature maps, filter out face regions through bounding box regression and confidence scores, crop face regions according to the optimal bounding box coordinates, and obtain cropped face images to ensure that only the core face regions are retained and background interference is removed.

[0109] S2.3: Input the cropped face image into the CurricularFace model. The CurricularFace model performs feature extraction and vector transformation on the cropped face image to obtain the face feature vector of the important person to be reviewed.

[0110] Furthermore, the feature vector generation process of the CurricularFace model includes:

[0111] (1) Standardize the cropped face image and convert it into the model input format;

[0112] (2) Deep features are extracted by the improved ResNet-50 network, which passes through convolutional layers, residual block groups and global average pooling layers in sequence to convert the two-dimensional feature map into a one-dimensional feature vector. The improved ResNet-50 network is the existing technology in this field and is not an inventive solution of this application. It will not be described in detail here.

[0113] (3) Feature mapping is performed through fully connected layers. The first fully connected layer performs a linear transformation on the pooled features and activates them with ReLU. The second fully connected layer further optimizes the features, and the output vector is the baseline face feature vector.

[0114] (4) Verify whether the vector dimension is 512, whether the values ​​are distributed in the range of [-1,1], and whether the magnitude is within a reasonable range (0.8-1.2). If the verification fails, the feature extraction process is re-executed.

[0115] S2.4: Use a bio-inspired spiking neural network to enhance the facial feature vector of the important person to be reviewed, generate an enhanced feature vector of the person, and associate it with the image identifier of the important person and store it in the feature database of the important person.

[0116] The enhancement process of the bio-inspired spiking neural network includes:

[0117] S2.41: Construct a sandwich structure comprising a time-coding layer, a pulse firing layer, and a synaptic plasticity modulation layer;

[0118] S2.42: Input the facial feature vector of the important person to be reviewed into the time coding layer. The time coding layer assigns one time coding neuron to each dimension of the facial feature vector of the important person to be reviewed, converts the feature values ​​of each dimension into time series pulse signals, weights all time series pulse signals through an exponential decay function to form a pulse sequence, and performs time alignment on the pulse sequences of each dimension to form a standardized pulse sequence.

[0119] Furthermore, the pulse conversion process of the time-coding layer is as follows:

[0120] (1) Assign one time-encoding neuron to each dimension of the baseline face feature vector to convert the feature values ​​into time-series pulse signals;

[0121] (2) The larger the eigenvalue, the higher the pulse firing frequency;

[0122] (3) The pulse signal is weighted by the exponential decay function to obtain the pulse sequence, wherein the recent pulse has a higher weight. The exponential decay function is the prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0123] (4) Time-align the pulse sequences in all dimensions to ensure that subsequent processing is performed at the same time step.

[0124] Furthermore, the initial assignment of characteristic responses and weights for the pulse firing layer includes:

[0125] (1) Each neuron calculates the membrane potential based on the frequency and decay weight of the input pulse. The initial value is 0. The membrane potential accumulates with the pulse input and decays naturally over time.

[0126] (2) When the membrane potential exceeds the threshold, a pulse is emitted, and the membrane potential is reset to the resting potential. The number of pulses emitted in each dimension is counted.

[0127] (3) The weight of any dimension is equal to the ratio of the number of pulses emitted in that dimension to the total number of pulses emitted, ensuring that the total weight is 1, highlighting important feature dimensions.

[0128] S2.43: The time-coding layer transmits the generated standardized pulse sequence to the pulse firing layer. The pulse firing layer calculates the membrane potential for each dimension's standardized pulse sequence, with the initial value of the membrane potential being 0. When the membrane potential exceeds a preset threshold, the corresponding dimension's neuron fires a pulse and resets the membrane potential to the resting potential. The total number of pulse firings for each dimension is counted, and the ratio of the number of firings in a single dimension to the total number of firings in all dimensions is used as the weight of that dimension. Through weight filtering, the filtered pulse sequence and the weight parameters for each dimension are obtained.

[0129] S2.44: The pulse firing layer inputs the obtained filtered pulse sequence and weight parameters of each dimension into the synaptic plasticity adjustment layer. The synaptic plasticity adjustment layer adopts the pulse time-dependent plasticity rule and adjusts the synaptic weights according to the time difference of adjacent pulses to obtain the fused feature vector.

[0130] Furthermore, the process of adjusting synaptic weights includes:

[0131] (1) Based on the pulse timing dependence plasticity rule, if the time difference between adjacent pulses is less than 5 time steps, the weight increases by 5%; if the time difference is greater than 10 time steps, the weight decreases by 3%, thus enhancing the weight of complementary features.

[0132] (2) If the weight of a single dimension is greater than 70%, the balancing mechanism is triggered to avoid feature redundancy. The balancing mechanism is to reduce the dominant weight and increase the secondary weight.

[0133] (3) The feature vector is enhanced by weighting the optimized weights and then the invalid features are suppressed by the ReLU function.

[0134] S2.45: The synaptic plasticity adjustment layer performs normalization on the generated fused feature vector, ultimately generating an enhanced character feature vector.

[0135] The lightweight face detection network employing multi-scale fusion performs frame-by-frame face detection on the material copies in the temporary region, including:

[0136] S3.1: The system reads the material copy to be processed from the temporary area and parses the material copy frame by frame to extract a continuous sequence of image frames;

[0137] S3.2: A spatiotemporal attention-guided detection mechanism is introduced. A spatiotemporal feature map is constructed for a continuous sequence of image frames. Intra-frame spatial attention weights and inter-frame temporal attention weights are calculated using a spatiotemporal attention approach. These weights are then integrated into a lightweight face detection network with multi-scale fusion. The spatial attention weights are used to highlight key features of the face region in a single frame image. The temporal attention weights are used to associate face motion features between adjacent frames.

[0138] Furthermore, the specific steps in S3.2 include:

[0139] (1) Preprocess the continuous image frame sequence parsed from the source copy;

[0140] (2) Based on the preprocessed continuous image frame sequence, construct a spatiotemporal feature map, including:

[0141] Multi-scale feature extraction is performed on single-frame images in the preprocessed continuous image frame sequence: low-scale features, medium-scale features, and high-scale features are extracted through different convolutional layers of a lightweight convolutional neural network. The feature maps at each scale retain the spatial dimension. Low-scale features include edges and textures, corresponding to image details; medium-scale features include facial component outlines, corresponding to local structures; and high-scale features include the overall shape of the face, corresponding to global information.

[0142] By stacking feature maps of the same scale from consecutive image frames along the time dimension, a spatiotemporal feature map is formed.

[0143] (3) Calculate the intra-frame spatial attention weights; The calculation of the intra-frame spatial attention weights focuses on a single-frame image. By strengthening the features of key facial regions and suppressing background interference, the accuracy of face detection is improved. Specifically, it includes: First, perform spatial feature saliency analysis: For each single-frame spatial feature map of the spatiotemporal feature map, calculate the feature response value of each spatial location, i.e., pixel point. Compress the channel dimension to 1 through global average pooling to obtain the feature response map. The higher the response value, the more likely the location is to belong to the facial region. Second, construct the spatial attention weight matrix: Input the feature response map into the sigmoid activation function and map the response value to the interval between 0 and 1 to obtain a weight matrix with the same size as the spatial feature map. The higher the weight value, the higher the attention of the corresponding spatial location.

[0144] (4) Introducing prior knowledge of the face to constrain the spatial attention weight matrix includes: First, loading a pre-trained face heatmap model; the face heatmap model is trained with a large number of face images and can output a probability distribution heatmap of key facial parts, such as eyes, nose and mouth; Second, multiplying the spatial attention weight matrix with the heatmap pixel by pixel to strengthen the weight of high probability areas in the heatmap, such as further increasing the weight of the eye area, and weakening the weight of the background area, such as reducing the weight of the background texture area; Third, performing weight normalization, calculating the sum of all elements of the weight matrix, dividing each element by the sum to ensure that the sum of the elements of the weight matrix is ​​1, avoiding the impact of weight scale differences on subsequent feature fusion. The optimized spatial attention weight matrix can accurately highlight the features of key facial areas in a single frame image.

[0145] (5) Calculate the inter-frame temporal attention weights; the inter-frame temporal attention weights calculation focuses on the correlation between adjacent frames, and solves the problem of detection instability caused by facial blurring and occlusion in dynamic scenes by capturing facial motion features, specifically including:

[0146] The first step is to extract inter-frame motion features: For the same scale spatial feature maps of two adjacent frames in the spatiotemporal feature map, such as the high-scale feature maps of frame t and frame t+1, calculate the feature difference between the corresponding spatial positions of the two frames to obtain the motion feature map. The larger the difference, the more intense the motion at that position, such as feature changes caused by face movement.

[0147] The second step is to calculate the basic weights for temporal attention: global max pooling is performed on the motion feature map to compress the channel dimension to 1, resulting in a motion intensity map. Then, the intensity values ​​are mapped to the range of 0 to 1 using the sigmoid activation function to obtain the basic weight matrix for temporal attention. The higher the weight value, the more important the motion features at the corresponding spatial location.

[0148] (6) Combining motion consistency to optimize the weight of the temporal attention base weight matrix, including: First, analyze the motion feature map of adjacent frames, calculate the motion direction and speed of the same spatial position in consecutive frames. If the motion direction of any position is consistent for 3 consecutive frames and the speed is stable, then the position is determined to be an effective motion area, and its time weight is multiplied by 1.2 to strengthen it; if the motion direction changes frequently and the speed fluctuates greatly, then it is determined to be an invalid motion area, and its time weight is multiplied by 0.8 to weaken it; Second, perform time dimension normalization, sum the time weight of the same spatial position in consecutive frames, divide the weight of each frame by the sum, and ensure that the weight of the position in the time dimension is 1, so as to avoid the weight of any frame being too high and causing the motion feature imbalance. The optimized temporal attention weight matrix can effectively associate the face motion features of adjacent frames and improve the continuity and stability of face detection in dynamic scenes.

[0149] (7) By using a multi-dimensional fusion strategy, the spatial attention weight matrix and the temporal attention weight matrix are integrated into a lightweight face detection network with multi-scale fusion to achieve attention-guided face detection. This includes: First, performing intra-scale weight fusion: For each scale of spatial feature map, the corresponding spatial attention weight matrix and temporal attention weight matrix are weighted and summed to obtain the spatiotemporal attention fusion weight matrix of that scale; Second, the spatiotemporal attention fusion weight matrix is ​​multiplied element-wise with the spatial feature map of the corresponding scale to obtain the attention-enhanced feature map; The attention-enhanced feature map retains the spatial features of the key facial regions in a single frame and integrates the motion features of adjacent frames, thereby improving the recognizability of facial features; Third, the attention-enhanced feature maps of different scales are adjusted to the same spatial size by upsampling or downsampling, and then the adjusted feature maps are spliced ​​according to the channel dimension to form a multi-scale fusion feature map; Fourth, the multi-scale fusion feature map is input into the detection head of the lightweight detection network, and the detection head can output the face detection results through anchor box generation, classification prediction and regression prediction. During the detection process, the spatiotemporal attention-enhanced feature map can help the detection head more accurately identify face regions, reduce the impact of background interference and dynamic blur, and improve the accuracy and recall rate of face detection.

[0150] S3.3: Apply a lightweight face detection network with multi-scale fusion that incorporates a spatiotemporal attention-guided detection mechanism to each extracted image frame for face detection. By locating all face regions contained in each image frame, generate corresponding bounding box coordinates and confidence scores for each detected face.

[0151] Furthermore, a preset number of anchor boxes are generated for each spatial location of each multi-scale fused feature map to cover faces of different sizes and shapes;

[0152] Furthermore, after incorporating the multi-scale fusion feature map and anchor box input into the detection head of the lightweight face detection network with a spatiotemporal attention-guided detection mechanism, face classification and bounding box regression tasks are performed in parallel. The classification branch outputs the probability that each anchor box belongs to a face through two convolutional layers, and uses the sigmoid activation function to map the probability to the interval between 0 and 1 to obtain a confidence score. The regression branch also uses two convolutional layers to predict the offset between the anchor box and the real face box, which is used to correct the anchor box position to a more accurate face region. Finally, the face bounding box coordinates and confidence score of each frame are output to complete the single-frame face detection.

[0153] S3.4: Filter the generated face detection results according to the preset confidence threshold, retain the face regions with confidence scores higher than the confidence threshold and their corresponding bounding box coordinates, and obtain the face detection results.

[0154] The process of determining the matching threshold includes:

[0155] S4.1: Establish a mapping table between material copy type and matching threshold to store the initial matching threshold corresponding to different material copy types;

[0156] Furthermore, the specific steps of S4.1 include: First, the system constructs a three-level material copy type system based on acquisition device, scene type, and format characteristics, and formulates and encodes classification standards; second, it collects sample data for each type, labels matching relationships, and selects high-quality samples; next, it calculates the similarity of sample pairs, determines the initial threshold based on the error rate index, and calibrates it according to sample quality; then, it designs a mapping table structure containing fields such as type identifier, classification information, and threshold, selects a combination of in-memory database and relational database for storage, and establishes an index; subsequently, it writes the threshold data into the mapping table, and ensures data reliability through integrity, accuracy, and format verification; finally, it develops query and update interfaces, sets up access control and performance optimization mechanisms, and completes the establishment and activation of the material copy type-matching threshold mapping table.

[0157] S4.2: Read the material copy type identifier in the review task and retrieve the corresponding initial matching threshold from the material copy type-matching threshold mapping table;

[0158] S4.3: Combine the scene features of the extracted frames from the material copy, dynamically adjust the initial matching threshold, and obtain the final matching threshold.

[0159] Furthermore, the specific steps of S4.3 include:

[0160] (1) Perform preprocessing and dimension division of scene feature data. The system retrieves the complete scene feature data of the extracted frames of the material copy from the temporary storage area. The scene feature data includes static features and dynamic features.

[0161] (2) Validity screening of scene feature data, removing invalid data caused by frame image damage or abnormal feature extraction, and then classifying them by dimension into two categories: static features and dynamic features;

[0162] (3) Assigning weights to scene feature dimensions, including: The system sets a basic weight for each feature dimension based on the degree of influence of different scene features on the face feature comparison results. Among them, the lighting feature has the most direct impact on the quality of face images. Overexposure of highlights and occlusion of shadows will significantly change the presentation of face features. Therefore, the basic weight of the lighting feature is set to the highest. For example, the basic weights of the proportion of highlight area, the proportion of shadow area, and the uniformity of lighting are set to 0.2, 0.2, and 0.15, respectively. The number and distribution density of contours in the structural features will affect the accuracy of face detection. In scenes with dense objects, face misidentification is likely to occur. Therefore, the basic weights of the number, length, and distribution density of contours are set to 0.1, 0.05, and 0.05, respectively. Dynamic scene features The magnitude and type of variation in features affect the clarity of facial images. Rapid camera movement can easily lead to image blurring. Therefore, the basic weights for magnitude and type of variation are set to 0.15 and 0.1, respectively. Texture features have relatively less interference with facial features. The basic weights for contrast, correlation, energy, and homogeneity are set to 0.02, 0.02, 0.01, and 0.01, respectively. The system fine-tunes the basic weights based on the actual scene attributes of the material copy, such as indoor monitoring material, outdoor snapshot material, and high-definition video material. For example, outdoor material is more affected by changes in lighting. The total weight of lighting features can be increased by 0.05, while the weight of structural features can be decreased by 0.05 to ensure that the weight allocation is adapted to the actual characteristics of the material scene.

[0163] (4) For the scene features of each valid frame, the system calculates the deviation between the actual value of each feature dimension and the standard threshold, and determines the adjustment coefficient of the dimension accordingly. Taking the illumination feature as an example, the standard threshold for illumination uniformity is preset to 0.6. If the illumination uniformity of any frame is 0.8, which exceeds the standard threshold of 0.2, it means that the lighting conditions of the frame are good and the facial features are clear. At this time, the adjustment coefficient is calculated to be 0.9. If the adjustment coefficient is less than 1, it means that the matching threshold needs to be reduced. The specific calculation logic is: the adjustment coefficient is equal to the difference obtained by subtracting 1 from the first difference. The first difference is the average of the difference between the actual value and the standard threshold. If the illumination uniformity is 0.4, which is lower than the standard threshold of 0.2, it means that the lighting conditions are poor and the facial features may be distorted. Then the adjustment coefficient is equal to the sum obtained by adding 1 to the first difference. That is, the adjustment coefficient of the illumination feature of the frame is 1.1.

[0164] (5) For the number of contours in the structural features, the preset standard threshold is 30. If the number of contours in any frame is 40, which exceeds the standard threshold of 10, it means that the scene objects are dense and easily interfere with face recognition. Then the adjustment coefficient is equal to the sum of 1 and the second difference. The second difference is the difference between the actual value and the standard threshold, and then multiplied by 0.01. If the number of contours is 20, which is lower than the standard threshold of 10, the scene is simple. Then the adjustment coefficient is equal to the difference between 1 and the second difference.

[0165] (6) In dynamic scene features, the standard threshold for the change amplitude is 0.3. If the change amplitude of any frame is 0.5, which exceeds the standard threshold of 0.2, it indicates that the camera is moving fast and the image may be blurry. The adjustment coefficient is equal to the sum of 1 and the third difference. The third difference is the difference between the actual value and the standard threshold, multiplied by 0.6. If the change amplitude is 0.1, which is lower than the standard threshold of 0.2, the scene is stable. The adjustment coefficient is equal to the difference between 1 and the third difference.

[0166] (7) The adjustment coefficient for each feature dimension is calculated according to the logic of (4)-(6);

[0167] (8) The system multiplies the adjustment coefficient of each scene feature dimension of each valid frame with its corresponding weight to obtain the weighted adjustment value of each dimension, and then sums the weighted adjustment values ​​of all dimensions to obtain the comprehensive adjustment coefficient of the frame.

[0168] (9) Retrieve the initial matching threshold corresponding to the material copy type from the material copy type-matching threshold mapping table, multiply the initial matching threshold by the comprehensive adjustment coefficient to obtain the threshold after preliminary adjustment. Subsequently, the system will perform a second fine-tuning based on the scene feature trend of the material copy. If the scene features of the material frame show a gradually improving trend, such as the lighting uniformity gradually increasing and the change amplitude gradually decreasing, it indicates that the facial features of subsequent frames will be clearer, and the preliminary adjustment threshold can be reduced by 0.02. If the scene features show a gradually deteriorating trend, such as the proportion of highlight areas gradually increasing and the number of contours gradually increasing, the system will perform a second fine-tuning based on the scene feature trend. If the initial threshold is not met, the initial adjustment threshold needs to be increased by 0.02. Finally, the system checks whether the final threshold is within the preset reasonable range, such as 0.6-0.85. If it is within the range, the value is determined as the final matching threshold. If it is outside the range, such as the final threshold calculation result being 0.58, which is lower than the minimum reasonable value of 0.6, the final threshold is set to 0.6. If the calculation result is 0.88, which is higher than the maximum reasonable value of 0.85, it is set to 0.85. This ensures that the final threshold meets the actual needs of the scene features and guarantees the accuracy and reliability of face comparison, thus completing the dynamic adjustment of the initial matching threshold.

[0169] The obtained facial feature vector of the source material is compared with the enhanced human feature vector in a hierarchical manner, including:

[0170] A1: Pre-construct an inverted index for all enhanced character feature vectors; the inverted index uses the dimension value of the enhanced character feature vector as the index key, and each index key is associated with a set of enhanced character feature vectors containing the corresponding dimension value;

[0171] A2: After obtaining a multimodal fusion feature vector, extract the top N dimensions with the highest values ​​from the face feature vector of the source material as key dimension values, and perform retrieval based on the constructed inverted index to filter out the enhanced human feature vector set containing the key dimension values ​​as the preliminary candidate set;

[0172] A3: Calculate the cosine similarity and Manhattan distance between the face feature vector of the source material and each enhanced human feature vector in the preliminary candidate set. Sort the enhanced human feature vectors in descending order according to the weighted calculation results of the two, and select the top K enhanced human feature vectors to form a subset of candidate feature vectors. The calculation formulas for cosine similarity and Manhattan distance are existing technologies in this field and are not the inventive solutions of this application, and will not be elaborated here.

[0173] A4: The obtained subset of candidate feature vectors is weighted using Euclidean distance, Mahalanobis distance, and Bhattacharyya distance to calculate the precise similarity between the source face feature vector and each candidate vector and then sorted. At the same time, scene features in the multimodal fusion feature vector are combined for auxiliary judgment to obtain the final similarity ranking result. The calculation formulas for Euclidean distance, Mahalanobis distance, and Bhattacharyya distance are existing technologies in this field and are not inventive solutions of this application, and will not be elaborated here.

[0174] If the exact similarity value ranked first in the final similarity ranking results is greater than or equal to the preset matching threshold, it is determined that the face in the material is successfully matched with the corresponding important person, and a comparison result containing the person's identifier and exact similarity value is generated. At the same time, the corresponding person's identifier is tagged for the copy of the frame material.

[0175] If the exact similarity value of the first-ranked result in the final similarity ranking is less than the preset matching threshold, it is judged as a matching failure and marked with an unknown or reserved special label.

[0176] A5: The comparison results and tagging information are associated, encapsulated, and stored in the review task database; the tagging information includes timestamp, material file ID, frame number, and tag content; the review task database adopts a strategy combining time-series partitioning and hash table partitioning.

[0177] The audit task database employs a strategy combining time-series partitioning and hash-based table partitioning, including:

[0178] A5.1: Pre-set the number of tables in the audit task database to M;

[0179] A5.2: When a new review task is generated, the system obtains the unique identifier of the review task as the partition key of the hash partition table, and uses the hash partition table algorithm to perform hash calculation on the review task identifier to obtain the hash value. The hash partition table algorithm is the prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0180] A5.3: Perform a modulo operation between the obtained hash value and the preset number of table partitions M, and determine the table partition where the audit task is stored based on the modulo result;

[0181] A5.4: Within the defined partitioned tables, a time partitioning strategy is adopted, with the month as the partitioning unit, and the comparison results and labeling information generated each month are stored in the partitioned table of the corresponding month;

[0182] A5.5: Establish a table partitioning routing table, record the correspondence between the audit task identifier range and the partitioned table and the sub-table, and output the table partitioning routing table and the audit task database after partitioning.

[0183] Example 2

[0184] Please see Figure 3 Another embodiment of the present invention provides: a face verification system based on feature vector comparison, comprising:

[0185] The module includes a management center module, a feature fusion module, a material detection module, a tagging module, and a report generation module.

[0186] The management center module is used to receive images of important figures uploaded by users and build a database. It is also used for the initial parameter configuration and storage of review tasks, providing basic data and task basis for the face review process.

[0187] The feature fusion module is used to preprocess images of important figures, perform model calculations and feature fusion, and generate enhanced feature vectors of the figures.

[0188] The material detection module is used to download copies of materials to be reviewed and analyze them frame by frame. It completes face detection through a lightweight face detection network with multi-scale fusion that integrates a spatiotemporal attention-guided detection mechanism. Simultaneously, it extracts scene features and generates face feature vectors for the materials, providing material-side data for subsequent multimodal fusion and comparison.

[0189] The tagging module is used to determine the appropriate matching threshold. It calculates the similarity between the facial feature vector of the source material and the enhanced human feature vector through hierarchical comparison. It combines scene features to assist in judgment and complete automatic tagging. The results are stored in the review task database to provide data support for task report generation.

[0190] The report generation module is used to generate task reports with verification codes based on data from the audit task database, ensuring data security and traceability of task results.

[0191] The management center module includes: a key personnel database unit and a personnel tagging unit;

[0192] The Important People Database unit is used to receive images of important people uploaded by users, assign a unique identifier to each image, and store the images of important people and their corresponding identifiers to form a structured database, ensuring that the target image can be accurately retrieved through the identifier.

[0193] The personnel tagging unit provides an input interface for review task parameters. It receives parameters such as the task name, the time period range to be reviewed, and the image identifier of important personnel from the user. It also performs legality verification on the input time period range to be reviewed, such as determining whether the time period has logical contradictions and whether it is within the system's processing range. It generates a unique review task identifier, associates the review task parameters with the identifier, and stores them in the pending task table of the review task database.

[0194] The feature fusion module includes: a preprocessing unit, a feature calculation unit, and a heuristic fusion unit;

[0195] The preprocessing unit is used to retrieve the corresponding important person image from the important person database unit according to the important person image identifier in the audit task parameters, perform face region detection and alignment, image normalization, and illumination compensation operations on the retrieved image, and output the preprocessed face image.

[0196] The feature calculation unit is used to input the preprocessed face image into the RetinaFace model and the CurricularFace model respectively to obtain the enhanced human feature vector.

[0197] The heuristic fusion unit is used to construct a sandwich structure with a bio-inspired spiking neural network as the core, which includes a time-encoding subunit, a spiking subunit, and a synaptic plasticity regulation subunit. This structure completes the weighted fusion of two features and outputs an enhanced human feature vector.

[0198] The material detection module includes: a material parsing unit and a feature generation unit;

[0199] The material parsing unit is used to download material copies within the corresponding time period from the system data source according to the time period range to be reviewed in the review task parameters, store them in the temporary area, and perform frame-by-frame parsing on the material copies in the temporary area to extract continuous image frame sequences;

[0200] The feature generation unit is used to encode and transform the output face region, generate the face feature vector of the source material, extract the scene features of the source material copy at the same time, and fuse the face feature vector of the source material with the scene features to form a multimodal fusion feature vector.

[0201] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of the present invention without departing from the spirit and scope of the present invention. All of these variations are within the protection scope of the present invention.

[0202] If the technical solution disclosed herein involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution disclosed herein involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. A face auditing method based on feature vector comparison, characterized in that, The method comprises the following steps: The user uploads the important person image through the important person library of the management center, and at the same time, establishes an audit task through a tagging module, selects a to-be-audited time period range and an uploaded important person image; Multi-modal feature extraction and fusion are performed on the selected important person image to generate an enhanced person feature vector; According to the audit task established by the user, the material copy in the to-be-audited time period range is downloaded to a temporary area, a multi-scale fusion lightweight face detection network is used to perform frame-by-frame face detection on the material copy in the temporary area, the scene features of the material copy are extracted synchronously, and each detected face image is encoded and converted to obtain a material face feature vector, the material face feature vector and the scene features are fused to form a multi-modal fusion feature vector; According to the type of the material copy, a matching threshold is determined, the material face feature vector in the obtained multi-modal fusion feature vector is compared with the enhanced person feature vector in a hierarchical manner, scene features are used for auxiliary judgment, an automatic tagging is performed according to the comparison result, and the comparison result and the tagging information are stored in an audit task database; A file system standard deletion instruction is used to delete the material copy in the temporary area and release the index node, and the iteration is performed until the processing of the full material is completed, and a task report with a check code is generated; The generation process of the enhanced person feature vector comprises: The system selects an important person image identifier corresponding to the important person according to the audit task parameters, reads the important person image uniquely corresponding to the important person image identifier from the important person library, the audit task parameters comprise a task name, a to-be-audited time period range, and one or more important person image identifiers selected from the important person library, and the important person library is a database formed after the user uploads the important person image through the management center and stores the important person image and the corresponding unique identifier; The received to-be-audited important person image is input into a RetinaFace model, the face region in the to-be-audited important person image is recognized and cropped through the RetinaFace model, and a cropped face image is obtained; The cropped face image is input into a CurricularFace model, the CurricularFace model is used for feature extraction and vector conversion on the cropped face image, and a face feature vector of the to-be-audited important person is obtained; A biological heuristic pulse neural network is used for enhancement processing on the face feature vector of the to-be-audited important person to generate an enhanced person feature vector, and the enhanced person feature vector is stored in association with the important person image identifier in an important person feature library.

2. The face auditing method based on feature vector comparison according to claim 1, characterized in that, The establishment of the audit task through the tagging module comprises: The system calls the tagging module, receives the audit task parameters input by the user, and performs legality verification on the received to-be-audited time period range, generates an audit task identifier, and stores the audit task parameters in association with the audit task identifier in the to-be-processed task table of the audit task database. The enhancement processing process of the biological heuristic pulse neural network comprises:

3. The face verification method based on feature vector comparison according to claim 2, characterized in that, A sandwich structure comprising a time coding layer, a pulse firing layer and a synaptic plasticity adjustment layer is constructed; ​ The face feature vector of the important person to be audited is input into the time coding layer, the time coding layer assigns 1 time coding neuron to each dimension of the face feature vector of the important person to be audited, converts the feature values of each dimension into a time sequence pulse signal, weights all the time sequence pulse signals through an exponential decay function, forms a pulse sequence, and performs time alignment on the pulse sequence of each dimension to form a standardized pulse sequence; The time coding layer transmits the generated standardized pulse sequence to the pulse emission layer, the pulse emission layer performs membrane potential calculation on the standardized pulse sequence of each dimension, and the initial value of the membrane potential is 0; when the membrane potential exceeds a preset threshold value, the neuron of the corresponding dimension emits a pulse and resets the membrane potential to a resting potential; the total number of pulse emissions of each dimension is counted, and the ratio of the number of emissions of a single dimension to the total number of emissions of all dimensions is taken as the weight of the dimension; through weight screening, a screened pulse sequence and dimension weight parameters are obtained; The pulse emission layer transmits the obtained screened pulse sequence and dimension weight parameters to the synaptic plasticity adjustment layer, the synaptic plasticity adjustment layer adopts a pulse time-dependent plasticity rule to adjust synaptic weights according to the time difference between adjacent pulses, and obtains a fusion feature vector; The synaptic plasticity adjustment layer performs normalization processing on the generated fusion feature vector to finally generate an enhanced person feature vector.

4. The face verification method based on feature vector comparison according to claim 3, characterized in that, The multi-scale fusion lightweight face detection network is used for performing frame-by-frame face detection on the material copy in the temporary area, including: The system reads the material copy to be processed from the temporary area, and performs frame-by-frame analysis on the material copy to extract a continuous image frame sequence; A spatio-temporal attention guiding detection mechanism is introduced to construct a spatio-temporal feature map for the continuous image frame sequence, calculate the intra-frame spatial attention weight and the inter-frame temporal attention weight through the spatio-temporal attention manner, and integrate the intra-frame spatial attention weight and the inter-frame temporal attention weight into the multi-scale fusion lightweight face detection network; the spatial attention weight is used to highlight the key features of the face region in a single frame image; the temporal attention weight is used to associate the face motion features between adjacent frames; The multi-scale fusion lightweight face detection network integrated with the spatio-temporal attention guiding detection mechanism is applied to each frame of image to perform face detection, and the boundary box coordinates and confidence scores corresponding to each detected face region are generated by locating all face regions in each frame of image; The generated face detection results are screened according to a preset confidence threshold, and the face regions and their corresponding boundary box coordinates with a confidence score higher than the confidence threshold are retained to obtain the face detection results.

5. The face verification method based on feature vector comparison according to claim 4, characterized in that, The determination process of the matching threshold value includes: A material copy type-matching threshold value mapping table is established to store the initial matching threshold values corresponding to different material copy types; The material copy type identifier in the audit task is read, and the corresponding initial matching threshold value is retrieved from the material copy type-matching threshold value mapping table; The initial matching threshold value is dynamically adjusted in combination with the scene features of the extracted frames of the material copy to obtain a finally adapted matching threshold value.

6. The face verification method based on feature vector comparison according to claim 5, characterized in that, The obtained material face feature vector and the enhanced person feature vector are compared in layers, including: An inverted index is constructed in advance for all enhanced person feature vectors, the inverted index taking the dimension values of the enhanced person feature vectors as index keys, and each index key being associated with a set of enhanced person feature vectors containing the corresponding dimension values; After obtaining a multi-modal fusion feature vector, the numbers of the top N dimensions with the highest values in the material person face feature vector are extracted as key dimension values, the set of enhanced person feature vectors containing the key dimension values is filtered out as a preliminary candidate set based on the constructed inverted index; The cosine similarity and Manhattan distance between the material person face feature vector and each enhanced person feature vector in the preliminary candidate set are calculated, the top K enhanced person feature vectors are selected according to the descending order of the weighted calculation results of the two, and a candidate feature vector subset is formed; The Euclidean distance, Mahalanobis distance and Bhattacharyya distance are used to calculate the accurate similarity between the material person face feature vector and each candidate vector, and the scene feature in the multi-modal fusion feature vector is used for auxiliary judgment to obtain the final similarity ranking result; If the accurate similarity value ranked first in the final similarity ranking result is greater than or equal to a preset matching threshold, it is determined that the material face matches the corresponding important person successfully, and a comparison result containing the person identifier and the accurate similarity value is generated, and the corresponding person identifier is labeled on the material copy of the frame; If the accurate similarity value ranked first in the final similarity ranking result is less than the preset matching threshold, it is determined that the matching fails, and an unknown or reserved special identification label is labeled; The comparison result and the labeling information are associated and encapsulated, and stored in an audit task database; the labeling information includes a timestamp, a material file ID, a frame number, and label content; the audit task database adopts a strategy combining time sequence partitioning and hash partitioning.

7. The face verification method based on feature vector comparison as claimed in claim 6, wherein, The audit task database adopts a strategy combining time sequence partitioning and hash partitioning, comprising: The number of partition tables of the audit task database is preset as M; When a new audit task is generated, the system obtains a unique identifier of the audit task as a partition table key of the hash partition table, and performs hash calculation on the audit task identifier using a hash partition table algorithm to obtain a hash value; The obtained hash value is subjected to modulo operation with the preset number of partition tables M, and the storage partition table of the audit task is determined according to the modulo result; The time partitioning strategy is used in the determined partition table, and the comparison results and labeling information generated each month are stored in the partition table of the corresponding month; A partition table routing table is established to record the correspondence between the range of audit task identifiers and the partition table and the partition table, and the partition table routing table and the partition table after the audit task database are output.

8. The face verification method based on feature vector comparison as claimed in claim 7, wherein, The temporary zone material copy is deleted and the index node is released using the file system standard deletion instruction, comprising: After processing all frames and face comparison of a material copy, the system calls the storage management service to send a file system standard deletion instruction to the material copy in the temporary area, informs the storage device to perform file deletion operation, releases the file system index node, updates the temporary area storage state table, and marks the material copy processing completion; the temporary area storage state table is a table recording the processing state of each material copy in the temporary area; the file system index node is a data structure recording the storage location, size attribute, and access right of the material copy file.

9. A face verification system based on feature vector matching, used to implement the face verification method based on feature vector matching in any one of claims 1-8, characterized in that, Comprise: a management center module, a feature fusion module, a material detection module, a marking module, and a report generation module; the management center module is configured to receive important person images uploaded by users and construct a database, and to audit initial parameter configuration and storage of a task; the feature fusion module is configured to preprocess important person images, perform model calculation and feature fusion, and generate enhanced person feature vectors; the material detection module is configured to download material copies to be audited and perform frame-by-frame analysis, complete face detection through a multi-scale fusion lightweight face detection network of a spatiotemporal attention guide detection mechanism, synchronously extract scene features, and generate material face feature vectors; the marking module is configured to determine an adaptive matching threshold, complete similarity calculation of material face feature vectors and enhanced person feature vectors through hierarchical comparison, combine scene features to assist in judgment, and complete automatic marking, and store the results to an audit task database; the report generation module is configured to generate a task report with a check code based on data of the audit task database.

Images