Face auditing method and system based on feature vector comparison

Abstract

The invention discloses a face auditing method and system based on feature vector comparison, and belongs to the technical field of image detection.The face auditing method comprises the steps that an important figure image uploaded by a user is received, and the user establishes an auditing task; performing multi-modal feature extraction and fusion on the selected important figure image to generate an enhanced figure feature vector; downloading a material copy in a to-be-audited time period to a temporary area according to the audit task, detecting a face frame by frame by adopting a multi-scale fusion lightweight face detection network, synchronously extracting scene features and encoding to generate a material face feature vector, and fusing to form a multi-modal fusion feature vector; determining a matching threshold according to the type of a material copy, comparing a material face feature vector with an enhanced character feature vector in a layered manner, performing auxiliary judgment and automatic marking in combination with scene features, and storing a result into an auditing task database; and deleting the temporary area material, generating a task report with a check code, and improving the auditing accuracy and the result security.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image detection, and in particular relates to a face auditing method and system based on feature vector comparison. BACKGROUND

[0002] In today's era of digital information explosion, a large amount of video and image materials are continuously generated, and in these materials, it is a challenging and crucial task to quickly and accurately identify specific important figures. Traditional face auditing methods often have low recognition accuracy, low processing efficiency, and difficulty in dealing with complex scenes, for example, some methods based on simple feature matching are easily affected by factors such as light, angle, and occlusion, resulting in a high misjudgment rate; and some auditing methods that require a lot of human intervention not only consume a lot of manpower and time cost, but also are difficult to guarantee the consistency and objectivity of the auditing results. With the continuous development of artificial intelligence technology, face recognition algorithms based on deep learning have made significant progress, however, how to effectively apply these advanced algorithms to the actual face auditing process to realize automated, efficient, and accurate face auditing is still a problem to be solved. Therefore, the present application proposes a face auditing method and system based on feature vector comparison to solve the above problems existing in the prior art. SUMMARY

[0003] In view of the deficiencies of the prior art, the present application proposes a face auditing method and system based on feature vector comparison, which receives important figure images uploaded by users, and the users establish auditing tasks; multi-modal feature extraction and fusion are performed on selected important figure images to generate enhanced figure feature vectors; according to the auditing task, copies of materials in the time period to be audited are downloaded to a temporary area, a multi-scale fusion lightweight face detection network is used to detect faces frame by frame, scene features are extracted and encoded to generate material face feature vectors, and multi-modal fusion feature vectors are formed by fusion; then, the matching threshold is determined according to the type of the material copy, the material face feature vectors and the enhanced figure feature vectors are compared layer by layer, the scene features are used to assist in judgment, and automatic labeling is performed, and the results are stored in the auditing task database; the temporary area materials are deleted, a task report with a check code is generated, and the auditing accuracy and result security are improved.

[0004] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0005] A face auditing method based on feature vector comparison, comprising:

[0006] The user uploads important figure images through the important figure library of the management center, and at the same time, establishes an auditing task through a labeling module, selects a time period to be audited and uploaded important figure images;

[0007] Multi-modal feature extraction and fusion are performed on selected important person images to generate enhanced person feature vectors;

[0008] According to the audit task established by the user, the material copy in the to-be-audited time period range is downloaded to the temporary area, the multi-scale fusion lightweight face detection network is used for frame-by-frame face detection on the material copy in the temporary area, the scene features of the material copy are extracted synchronously, and each detected face image is encoded and converted to obtain a material face feature vector, the material face feature vector and the scene features are fused to form a multi-modal fusion feature vector;

[0009] According to the material copy type, the matching threshold is determined, the material face feature vector in the obtained multi-modal fusion feature vector is compared with the enhanced person feature vector in a hierarchical manner, the scene features are used for auxiliary judgment, automatic labeling is performed according to the comparison result, and the comparison result and the labeling information are stored in the audit task database;

[0010] The file system standard deletion instruction is used to delete the material copy in the temporary area and release the index node, and the iteration is performed until the full material processing is completed, and a task report with a check code is generated.

[0011] Specifically, the audit task is established by the labeling module, including:

[0012] The system calls the labeling module to receive the audit task parameters input by the user; the audit task parameters include a task name, a to-be-audited time period range, and one or more important person image identifiers selected from an important person library; the important person library is a database formed by the user uploading important person images through a management center, and stores important person images and corresponding unique identifiers;

[0013] After the system verifies the legality of the received to-be-audited time period range, an audit task identifier is generated, and the audit task parameters and the audit task identifier are associated and stored in the to-be-processed task table of the audit task database.

[0014] Specifically, the generation process of the enhanced person feature vector includes:

[0015] The system selects the important person image identifier corresponding to the important person according to the audit task parameters, and reads the important person image uniquely corresponding to the important person image identifier from the important person library;

[0016] The received to-be-audited important person image is input into the RetinaFace model, the face region in the to-be-audited important person image is recognized and cropped through the RetinaFace model, and a cropped face image is obtained;

[0017] input the cropped face image into the CurricularFace model, and perform feature extraction and vector conversion on the cropped face image through the CurricularFace model to obtain a face feature vector of the important person to be audited;

[0018] The face feature vector of the important person to be audited is enhanced by using a biological heuristic pulse neural network to generate an enhanced person feature vector, which is stored in association with the important person image identifier in the important person feature library.

[0019] Specifically, the enhancement process of the biological heuristic pulse neural network includes:

[0020] A sandwich structure including a time coding layer, a pulse firing layer and a synaptic plasticity adjustment layer is constructed.

[0021] The face feature vector of the important person to be audited is input into the time coding layer, and the time coding layer assigns 1 time coding neuron to each dimension of the face feature vector of the important person to be audited, converts the feature values of each dimension into time sequence pulse signals, weights all the time sequence pulse signals by an exponential decay function, forms a pulse sequence, and time-aligns the pulse sequences of each dimension to form a standardized pulse sequence.

[0022] The time coding layer transmits the generated standardized pulse sequence to the pulse firing layer, the pulse firing layer calculates the membrane potential of the standardized pulse sequence of each dimension, and the initial value of the membrane potential is 0. When the membrane potential exceeds a preset threshold, the neuron of the corresponding dimension fires a pulse and resets the membrane potential to the resting potential. The total number of pulse firings of each dimension is counted, and the ratio of the number of firings of a single dimension to the total number of firings of all dimensions is taken as the weight of the dimension. Through weight screening, the screened pulse sequence and the weight parameters of each dimension are obtained.

[0023] The pulse firing layer transmits the obtained screened pulse sequence and the weight parameters of each dimension to the synaptic plasticity adjustment layer, which uses a pulse timing-dependent plasticity rule to adjust the synaptic weights according to the time difference between adjacent pulses to obtain a fusion feature vector.

[0024] The synaptic plasticity adjustment layer performs normalization processing on the generated fusion feature vector to finally generate an enhanced person feature vector.

[0025] Specifically, the lightweight face detection network using multi-scale fusion performs frame-by-frame face detection on the material copies in the temporary area, including:

[0026] The system reads the material copies to be processed from the temporary area, and performs frame-by-frame analysis on the material copies to extract a continuous image frame sequence.

[0027] The spatio-temporal attention guiding detection mechanism is introduced, a spatio-temporal feature map is constructed for a continuous image frame sequence, the intra-frame spatial attention weight and the inter-frame temporal attention weight are calculated in a spatio-temporal attention manner, and the intra-frame spatial attention weight and the inter-frame temporal attention weight are integrated into the multi-scale fusion lightweight face detection network; the spatial attention weight is used to highlight the key features of the face region in a single frame image; and the temporal attention weight is used to associate the face motion features between adjacent frames;

[0028] The multi-scale fusion lightweight face detection network integrated with the spatio-temporal attention guiding detection mechanism is applied to each extracted image for face detection, all face regions contained in each image are located, and corresponding bounding box coordinates and confidence scores are generated for each detected face region;

[0029] The generated face detection results are filtered according to a preset confidence threshold, face regions and their corresponding bounding box coordinates with a confidence score higher than the confidence threshold are retained, and face detection results are obtained.

[0030] Specifically, the determination process of the matching threshold includes:

[0031] A material copy type-matching threshold mapping table is established to store initial matching thresholds corresponding to different material copy types;

[0032] The material copy type identifier in the review task is read, and the corresponding initial matching threshold is retrieved from the material copy type-matching threshold mapping table;

[0033] The initial matching threshold is dynamically adjusted in combination with the scene features of the extracted frame of the material copy to obtain a finally adapted matching threshold.

[0034] Specifically, the obtained material face feature vector and the enhanced person feature vector are compared in layers, including:

[0035] An inverted index is constructed in advance for all enhanced person feature vectors; the inverted index takes the dimension value of the enhanced person feature vector as an index key, and each index key is associated with an enhanced person feature vector set containing the corresponding dimension value;

[0036] When a multi-modal fusion feature vector is obtained, the numbers of the top N dimensions with the highest values in the material face feature vector in the multi-modal fusion feature vector are extracted as key dimension values, the constructed inverted index is searched, and an enhanced person feature vector set containing the key dimension values is selected as a preliminary candidate set;

[0037] Cosine similarity and Manhattan distance between the material face feature vector and each enhanced person feature vector in the obtained preliminary candidate set are calculated, the result is weighted according to the descending order, and the top K enhanced person feature vectors are selected to form a candidate feature vector subset;

[0038] The obtained candidate feature vector subset is used to calculate the accurate similarity between the material face feature vector and each candidate vector by weighting Euclidean distance, Mahalanobis distance and Bhattacharyya distance, and to sort, and the scene feature in the multi-modal fusion feature vector is combined for auxiliary judgment to obtain the final similarity ranking result.

[0039] If the accurate similarity value ranked first in the final similarity ranking result is greater than or equal to the preset matching threshold, it is determined that the material face matches the corresponding important person successfully, and the comparison result containing the person identifier and the accurate similarity value is generated, and the corresponding person identifier is labeled for the frame material copy;

[0040] If the accurate similarity value ranked first in the final similarity ranking result is less than the preset matching threshold, it is determined that the matching fails, and an unknown or reserved special identification label is labeled;

[0041] The comparison result and the labeling information are associated and encapsulated, and stored in the audit task database; the labeling information includes timestamp, material file ID, frame number, label content; the audit task database adopts a strategy combining time sequence partition and hash partition table.

[0042] Specifically, the audit task database adopts a strategy combining time sequence partition and hash partition table, which comprises:

[0043] The number of partition tables of the audit task database is preset as M;

[0044] When a new audit task is generated, the system obtains the unique identifier of the audit task as the partition table key of the hash partition table, and performs hash calculation on the audit task identifier by using a hash partition table algorithm to obtain a hash value;

[0045] The obtained hash value is subjected to a modulo operation with the preset number of partition tables M, and the storage partition table of the audit task is determined according to the modulo result;

[0046] The time partition strategy is used in the determined partition table, and the comparison result and the labeling information generated each month are stored in the partition table of the corresponding month;

[0047] A partition table routing table is established to record the correspondence between the audit task identifier range and the partition table and the partition table, and the partition table routing table and the partitioned audit task database are output.

[0048] Specifically, the deleting the temporary area material copy and releasing the inode by using the file system standard deletion instruction comprises:

[0049] After all the frames of a material copy are processed and the face comparison is completed, the system calls the storage management service, sends a file system standard deletion instruction to the material copy in the temporary area, informs the storage device to perform a file deletion operation, releases the file system inode, updates the temporary area storage state table, and marks that the processing of the material copy is completed; the temporary area storage state table is a table recording the processing state of each material copy in the temporary area; and the file system inode is a data structure used for recording the storage location, size attribute, and access permission of the material copy file.

[0050] A face auditing system based on feature vector comparison, comprising a management center module, a feature fusion module, a material detection module, a marking module, and a report generation module.

[0051] The management center module is used for receiving important person images uploaded by a user and constructing a database, and is also used for initial parameter configuration and storage of an auditing task.

[0052] The feature fusion module is used for pre-processing the important person images, model calculation, and feature fusion, and is used for generating enhanced person feature vectors.

[0053] The material detection module is used for downloading a material copy to be audited and performing frame-by-frame analysis, completing face detection through a multi-scale fusion lightweight face detection network of a fusion space-time attention guide detection mechanism, synchronously extracting scene features, and generating material face feature vectors.

[0054] The marking module is used for determining an adaptive matching threshold, completing similarity calculation of the material face feature vectors and the enhanced person feature vectors through hierarchical comparison, combining scene feature auxiliary judgment, and completing automatic marking, and is used for storing the result to an auditing task database.

[0055] The report generation module is used for generating a task report with a check code based on the data of the auditing task database.

[0056] Compared with the prior art, the face auditing system has the following beneficial effects:

[0057] 1.The present application proposes a face auditing method based on feature vector comparison, which extracts and fuses multi-modal features from important person images to generate enhanced person feature vectors, simultaneously uses a multi-scale fusion lightweight face detection network to detect material faces frame by frame and fuses scene features to form multi-modal fusion feature vectors, dynamically determines the matching threshold combined with material copy types to carry out hierarchical comparison, can accurately capture face key features and scene association information, greatly reduces the probability of false negatives and false positives; the lightweight detection network and the temporary area material efficient processing mechanism also effectively improve the overall auditing speed, which is suitable for large-scale material auditing demand.

[0058] 2.The present application proposes a face auditing method based on feature vector comparison, which has strong security and process standardization, uses deletion instructions to delete temporary area material copies and release index nodes in time, avoids redundant data occupying storage resources or causing information leakage risk; the auditing results and tagging information are stored in the database in order, and the final task report with check code is generated, which ensures that the auditing process is traceable and the results are tamper-proof, meets the data security storage requirements, and improves the credibility of the auditing results. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 Fig. 1 is a schematic diagram of a face auditing method based on feature vector comparison of the present application;

[0060] Figure 2 Fig. 2 is a principle flow chart of a face auditing method based on feature vector comparison of the present application;

[0061] Figure 3 Fig. 3 is a system architecture diagram of a face auditing system based on feature vector comparison of the present application. DETAILED DESCRIPTION

[0062] Embodiment 1

[0063] Please refer to Figure 1 and Figure 2 , the present application provides an embodiment: a face auditing method based on feature vector comparison, the method comprises S1-S5:

[0064] S1: the user uploads important person images through the important person library of the management center, at the same time, establishes an auditing task through the tagging module, selects the time range to be audited and the important person images uploaded;

[0065] S2: multi-modal feature extraction and fusion are performed on the selected important person images to generate enhanced person feature vectors;

[0066] S3: According to the audit task established by the user, download the material copy in the to-be-audited time period range to the temporary area, use the multi-scale fusion lightweight face detection network to perform frame-by-frame face detection on the material copy in the temporary area, synchronously extract the scene features of the material copy, and perform encoding conversion on each detected face image to obtain a material face feature vector, and fuse the material face feature vector and the scene features to form a multi-modal fusion feature vector;

[0067] Further, the dimensions of the to-be-audited important person face feature vector and the material face feature vector are 512, and the module length is 1.

[0068] Further, the synchronous extraction of the material copy scene features is in the same frame processing period of using the multi-scale fusion lightweight face detection network to detect faces frame by frame on the material copy, by parallel calling of a feature generation unit to analyze the non-face region and the overall image attributes of the current processing frame, while associating the scene change information of adjacent frames, finally generating feature data reflecting the scene environment of the current frame, ensuring that the scene feature extraction, face detection and material face feature vector generation are synchronized in the time dimension, and the scene features can be accurately corresponded with the material face feature vector, specifically including:

[0069] (1) After the system reads the to-be-processed material copy from the temporary area and parses the image frame sequence frame by frame, when any frame image enters the multi-scale fusion lightweight face detection network for face detection, the system will synchronously copy a copy of the frame image through a data distribution interface and transmit it to the feature generation unit, to ensure that face detection and scene feature extraction are based on the same original data of the same frame, avoiding synchronization deviation caused by data transmission delay or version difference;

[0070] (2) The feature generation unit pre-processes the received frame image, first performs image resolution normalization to adjust the image size to a fixed size, then performs color space conversion to convert the RGB color space to the HSV color space, because the HSV space can more clearly distinguish the hue, saturation and brightness, and is more consistent with the human eye's perception logic of scene color and brightness, facilitating subsequent extraction of scene color features, and then performs noise removal operation, using a Gaussian filter algorithm to smooth the converted image to filter random noise generated in the image acquisition or transmission process, avoiding noise interference with the accurate extraction of scene features, to obtain the pre-processed frame image, wherein the normalization, HSV color space and Gaussian filter algorithm are prior art contents in the field, and are not the inventive scheme of the present application, and will not be described here.

[0071] (3) static scene feature extraction is performed; the static scene feature extraction focuses on inherent attributes of a current single frame image, mainly including four dimensions: a first dimension is scene color feature extraction; an image in HSV space is divided into 12 intervals according to hue, a pixel proportion of each hue interval in the image is counted, a mean value and a variance of saturation and a mean value and a variance of brightness are calculated, hue proportion, saturation statistical value and brightness statistical value are integrated to form color feature data; a second dimension is scene texture feature extraction; a gray level co-occurrence matrix algorithm is used to convert a brightness channel of the image in HSV space into a gray scale image, then a plurality of non-overlapping local regions are selected in the gray scale image, four key parameters of a gray level co-occurrence matrix, such as contrast, correlation, energy and homogeneity, are calculated for each local region; the contrast reflects a difference degree of gray scales in the local region, such as a high contrast of a scene with rough texture; the correlation reflects an association degree of gray scales in the local region; the energy reflects a uniformity degree of texture; the homogeneity reflects a similarity degree of gray scales in the local region; the four parameters of all local regions are averaged respectively to obtain texture feature data of the entire frame image; a third dimension is scene structure feature extraction; edge information in the image is extracted through an edge detection algorithm to identify main outlines in the image, then a number, a length and a distribution density of the outlines are counted, and scene type preliminary features formed by the outlines are judged, such as a large number of straight line outlines possibly corresponding to indoor or building scenes, and irregular curve outlines possibly corresponding to natural scenes, to form structure feature data; a fourth dimension is scene light feature extraction; a mean value and a variance of the brightness channel are combined to further analyze a distribution of highlight regions and shadow regions in the image, an area proportion of the highlight regions, an area proportion and a shape of the shadow regions are counted, and a uniformity of light is calculated to form light feature data; finally, color feature data, texture feature data, structure feature data and light feature data are integrated to constitute static scene features of the current frame; the mean value and the variance, the gray level co-occurrence matrix algorithm and the edge detection algorithm are prior art contents in the field and are not the inventive scheme of the present application, and are not described herein again;

[0072] (4) Dynamic scene feature extraction is performed; the dynamic scene feature extraction is mainly related to the scene change of the current frame and the adjacent frame, which makes up for the limitation of static features reflecting only single-frame information. First, the system records the frame number of the current processing frame, and calls the static scene feature data of the previous frame, wherein the frame number of the previous frame is one less than the current frame number, and if the current frame is the first frame of the material copy, the static scene feature data of the previous frame is set to be consistent with the initial static scene feature data of the current frame, so as to avoid the inability to calculate dynamic features when there is no adjacent frame; then, the difference values of each dimension of the static scene features of the current frame and the previous frame are calculated, for color features, the absolute difference value of the hue proportion, the absolute difference value of the mean and variance of the saturation, and the absolute difference value of the mean and variance of the brightness are calculated; for texture features, the absolute difference values of the contrast, correlation, energy and homogeneity are calculated; for structure features, the absolute difference values of the number, length and distribution density of the outline are calculated; for lighting features, the absolute difference values of the highlight area proportion, the shadow area proportion and the lighting uniformity are calculated; then, the trend of the difference values is analyzed to determine whether the scene has dynamic changes, and the amplitude and type of the scene change between adjacent frames are counted; finally, the difference values, the change amplitude and the change type are integrated to form the dynamic scene features of the current frame, and the scene dynamic change information is captured;

[0073] (5) After obtaining the static scene features and dynamic scene features of the current frame, the two types of features are fused to form complete scene features;

[0074] (6) After completing the scene feature extraction of the current frame and generating the standardized scene feature vector, the system obtains the face region information obtained by face detection of the same frame image and the material face feature vector, then establishes an association relationship through the frame number and the face identifier, binds the scene feature vector of the current frame with all the material face feature vectors corresponding to the frame, ensures that each material face feature vector can correspond to the scene features of the frame where it is located, avoids feature mismatch, and temporarily stores the associated scene feature vector, material face feature vector and corresponding frame number, face identifier and other information.

[0075] S4: According to the material copy type, determine the matching threshold, compare the material face feature vector and the enhanced person feature vector in the obtained multi-modal fusion feature vector, combine the scene feature auxiliary judgment, automatically mark according to the comparison result, and store the comparison result and marking information to the review task database;

[0076] Further, in combination with the scene feature auxiliary judgment, the comparison result is automatically marked according to the comparison result, and the comparison result and the marking information are stored in the review task database, which comprises:

[0077] (1) After completing the hierarchical comparison of the material face feature vector and the enhanced person feature vector, the system first retrieves the multi-modal fusion feature vector of the current frame, separates the scene feature vector extracted previously from it, and obtains the hierarchical comparison result of the frame. Then, the system establishes an association between the frame number and the face identifier to ensure that each hierarchical comparison result can accurately correspond to the scene feature vector of the frame where it is located, avoiding the mismatch of scene features and comparison results in different frames. For example, if two faces are detected in any frame, two hierarchical comparison results are generated, and the system binds the same scene feature vector of the frame to each comparison result;

[0078] (2) Obtain the scene feature extracted in S3;

[0079] (3) For different situations of hierarchical comparison results, reasonable verification is performed in combination with each sub-dimension of the scene feature;

[0080] If the hierarchical comparison result is a match success, it is judged from the perspective of the scene feature whether the match conforms to the actual scene logic. For example, first, check the lighting feature. If the high-light area ratio of the current frame exceeds 0.3 or the shadow area ratio exceeds 0.4, it indicates that the face may be overexposed or occluded. At this time, it is necessary to verify whether the accurate similarity value is at least 0.1 higher than the matching threshold. If it does not meet the requirement, it is determined that the match is at risk and needs to be marked for review. Then, analyze the structure feature. If the number of contours in the current frame exceeds 50 and the distribution density is high, it indicates that the objects in the scene are dense, which may cause face misidentification. It is necessary to confirm whether the bounding box coordinates of the face are in a relatively clear and unoccluded area in the image. If the overlap degree of the bounding box and other object contours exceeds 0.3, it is also marked for review. Finally, refer to the dynamic scene feature. If the change type is fast lens movement and the change amplitude exceeds 0.5, it indicates that the image may be blurred, and it is necessary to recheck the accuracy of the face encoding conversion. If there is a feature missing prompt during the encoding process, it is marked for review;

[0081] If the hierarchical comparison result is a match failure, it is determined whether the misjudgment is caused by scene factors. If the lighting feature shows that the lighting uniformity of the current frame is higher than 0.8, the structure feature shows that the face bounding box is complete and has no overlap, and the dynamic scene feature has no obvious change, i.e., the change amplitude is less than 0.2, it indicates that the face image is clear and the scene is stable. At this time, if the difference between the accurate similarity value and the matching threshold is less than 0.05, it is determined that there may be a missed judgment, which needs to be marked for review. If the scene feature shows that there is severe noise or occlusion in the current frame, such as a texture feature contrast less than 0.2, it is determined that the match fails and is reasonable, and no additional marking is needed;

[0082] (4) After completing the scene feature assisted judgment, the system executes the corresponding automatic marking rules according to the final judgment result, which includes match success and reasonable, match success for review, match failure and reasonable, and match failure for review.

[0083] For the case of successful matching and reasonableness, the system generates a label containing the important person identifier, the precise similarity value, and the tagging timestamp, while appending a scene feature abstract, such as uniform illumination and stable scene, in the label;

[0084] For the case of successful matching to be reviewed, the label contains the important person identifier, the similarity value, and the timestamp, and explicitly labels the reason to be reviewed, such as high light overexposure and similarity close to the threshold, and is automatically assigned to the task list of the reviewer;

[0085] For the case of failed matching and reasonableness, the system will label the unknown person, and if the face in the frame appears in the matching success record in the non-self material copy, the system will trace back the historical detection situation through the label association;

[0086] For the case of failed matching to be reviewed, the label will label the suspected missed judgment and the reason, such as clear scene and similarity close to the threshold, and is also included in the review process;

[0087] (5) Structured packaging of comparison results and tagging information, including: extracting core information from comparison results, including material file ID, frame number, face identification, important person identifier, precise similarity value, matching threshold, and key data in each stage of hierarchical comparison; integrating tagging information, including label type, label content, tagging timestamp, scene feature abstract, and review status; mapping the core information and tagging information in the comparison results to the corresponding fields according to the field specification of the audit task database, for example, the material file ID corresponds to the file_id field in the database, the label type corresponds to the tag_type field, ensuring that each information item can accurately match the database field, avoiding field mismatch or data loss during storage; performing format checking on the packaged data to check if there are empty fields or data format errors, if there are errors, return to the previous step to repackage, if the verification is passed, generate a structured data package, wherein the label type includes valid matching, to be reviewed, unknown person, and suspected missed judgment;

[0088] (6) The system determines the storage location of the structured data package according to the time sequence partitioning and hash partitioning strategy adopted by the audit task database, then calls the database write interface to write the structured data package into the determined partition table, and at the same time, starts the transaction mechanism to ensure the atomicity of the data writing process, if an exception occurs during writing, the transaction is rolled back to avoid inconsistency caused by partial data writing; if the writing is successful, the transaction is committed;

[0089] (7) After writing is completed, the system constructs an index for the data: taking the material file ID and frame number as the joint index key, associating the storage address of the data in the database, and constructing separate indexes for the important person identifier and the label type field; finally, the storage state table of the database is updated to record the storage completion time and index information of the data, ensuring the efficiency of data management and query.

[0090] S5: delete the temporary area material copy and release the index node by using the file system standard deletion instruction, and iterate until the full material processing is completed, to generate a task report with a check code.

[0091] The deleting the temporary area material copy and releasing the index node by using the file system standard deletion instruction comprises:

[0092] After completing the processing of all frames and face comparison of a material copy, the system calls the storage management service to send a file system standard deletion instruction to the material copy in the temporary area, to notify the storage device to perform file deletion operation, release the file system index node, update the temporary area storage state table, and mark the processing completion of the material copy; the storage device of the temporary area comprises a mechanical hard disk; the temporary area storage state table is a table recording the processing state of each material copy in the temporary area; the file system index node is a data structure recording the storage location, size attribute, and access permission of the material copy file.

[0093] Further, the construction process of the file system standard deletion instruction comprises: the storage management service first queries the logical storage path and file metadata of the material copy in the storage device according to the temporary area storage path of the material copy, obtains the index node number corresponding to the material file through the index node association table of the file system, and then locates all the storage block association information recorded in the index node, to ensure accurate positioning of all storage areas needing to be cleaned; the instruction content is constructed according to the file deletion instruction format specification of the corresponding operating system, and the instruction contains the operation type, absolute path of the target file, storage device identifier, and instruction execution priority; the file metadata comprises file size, creation time, and access permission; the storage block association information comprises physical storage block address and block quantity.

[0094] Further, the process of notifying the storage device to perform the file deletion operation includes: after the file system standard deletion instruction is constructed, the storage management service sends the deletion instruction to the storage device corresponding to the temporary area through the file system interface; after the storage device receives the instruction, it first checks the integrity and legality of the instruction, confirms that the target file path exists, the current operation account has file deletion permission, and the storage device is in a normal read-write state, and returns a confirmation signal indicating that the instruction is received successfully after the check is passed; the storage device starts the file cleaning mechanism, performs erasure operation and marking on the file storage block specified in the instruction, and releases the storage block associated resources corresponding to the file to avoid the storage block being occupied by the file; during the process, the storage device feeds back the cleaning progress to the storage management service in real time, such as the number of marked storage blocks, the number of remaining storage blocks to be marked, until all target storage blocks are processed, and the result of the deletion instruction execution is returned.

[0095] Further, the process of releasing the file system index node includes: while the storage device executes the file deletion instruction, the storage management service will process the release operation of the file system index node in parallel to avoid process delay caused by waiting for the garbage collection to be completed. First, the system queries the index node entry corresponding to the material file copy from the index node table of the file system according to the material file ID, and the index node entry contains the storage location, file size, creation time, access permission, associated storage block address and other key information of the material file; the storage management service will first mark the state of the index node entry as to be released to prevent non-self processes from misoperating the index node during the release process; then the association between the index node entry and the storage block address is deleted, and the storage block address list recorded in the index node entry is emptied to ensure that subsequent new file allocation of storage blocks will not reference the released blocks; secondly, it is checked whether there are dependent items associated with the index node, such as hard links, soft links, and un-closed process file handles; if there are hard links, all hard links are deleted and the hard link count is updated; if there are un-closed process handles, the process is notified that the file will be released, and the process is required to close the handle; after all handles are closed, the index node release operation is performed; finally, the access lock of the index node is released, the index node entry is marked as idle, and is added to the index node idle pool of the system for use by newly created files, and the index node usage statistical information of the file system is updated to complete the comprehensive release of the index node; the index node usage statistical information includes the number of idle index nodes and the number of used index nodes.

[0096] Further, the storage management service updates the temporary area storage state table immediately after completing the file system standard deletion instruction execution and the index node release, to ensure that the system can grasp the storage resource situation of the temporary area in real time. First, the system locates the record entry corresponding to the material copy in the temporary area storage state table, and the record entry includes the material file ID, the temporary storage path, the processing state, the storage occupation size, the cleaning start time, the cleaning completion time, and the like. The processing state includes processing, to be cleaned, and completed. The processing state field is updated from to be cleaned to completed, to clearly mark the end of the cleaning process of the material copy. The cleaning completion time field is supplemented with the current system time to record the completion time of the cleaning operation. The storage occupation size field is updated to zero to indicate that the material copy no longer occupies the temporary area storage resource. At the same time, the cleaning information is recorded in the cleaning result field, including the deletion instruction execution result and the index node release state. After the update is completed, the update record of the state table is backed up, and the field values before and after the update, the operation account, the operation time, and the like are stored in the operation log database of the system, and the log record is retained for at least three months.

[0097] The audit task is established by the marking module, including:

[0098] S1.1: The system calls the marking module to receive the audit task parameters input by the user; the audit task parameters include the task name, the to-be-audited time period range, and one or more important person image identifiers selected from the important person library; the important person library is a database formed by the user uploading important person images through the management center, and stores important person images and corresponding unique identifiers;

[0099] S1.2: After the system verifies the legality of the received to-be-audited time period range, an audit task identifier is generated, and the audit task parameters and the audit task identifier are associated and stored in the to-be-processed task table of the audit task database.

[0100] Further, the system verifies legality from three dimensions, including: the first dimension is the verification of the chronological relationship between the start time and the end time. Comparing the converted start time stamp and end time stamp, if the start time stamp is greater than or equal to the end time stamp, it means that there is a logical contradiction that the start time is later than the end time in the time period, the system determines that the verification fails, and returns a prompt that the start time cannot be later than the end time; if the start time stamp is less than the end time stamp, the second dimension verification is entered; the second dimension is the matching verification of the time period and the system processable time range. The system presets the processable time range as three years before the current system time to one day after the current system time. Three years before the current system time is set to avoid processing old historical data, and one day after the current system time is set to allow the user to book the material review task of the next day one day in advance. If the start time of the time period to be reviewed is earlier than three years before the current system time or the end time is later than one day after the current system time, the system determines that the verification fails, returns a prompt that the time period exceeds the system processable range, and informs the user of the legal time range; if it is within the processable range, the third dimension verification is entered; the third dimension is the matching verification of the time period and the stored material time range. The system queries the time distribution of the stored materials in the material management database. If there is no stored material in the time period to be reviewed, such as no monitoring video generated in the time period selected by the user, the system prompts that there is no material to be reviewed in the time period, and asks the user whether to adjust the time period; if there is a stored material, the system determines that the logical legality verification of the time period to be reviewed is passed, and generates a result that the time period verification is passed.

[0101] The generation process of the enhanced person feature vector includes:

[0102] S2.1: The system selects the important person image identifier corresponding to the important person according to the review task parameters, and reads the important person image uniquely corresponding to the important person image identifier from the important person library;

[0103] S2.2: The received important person image to be reviewed is input into the RetinaFace model, and the face region in the important person image to be reviewed is identified and cropped through the RetinaFace model to obtain the cropped face image;

[0104] Further, before the received important person image to be reviewed is input into the RetinaFace model, preprocessing is also required, including: grayscale conversion, noise filtering, contrast enhancement, and size standardization.

[0105] Further, the face cropping process of the RetinaFace model includes:

[0106] (1) Format conversion is performed on the pre-processed important person image, including normalizing the pixel value to the range of [0, 1] and adding a batch dimension to form a [1, 3, 112, 112] tensor format;

[0107] (2) Multi-scale features are extracted through a ResNet-50 backbone network, which sequentially passes through a convolutional layer, a batch normalization layer, a ReLU activation function, and a residual block, and outputs a feature map. The ResNet-50 backbone network is prior art in the field and is not part of the inventive concept of the present application, and will not be described here.

[0108] (3) Anchor boxes are generated based on the feature map, and the face region is selected through bounding box regression and confidence score. The face region is cropped according to the optimal bounding box coordinates to obtain the cropped face image, ensuring that only the core face region is retained and background interference is removed.

[0109] S2.3: The cropped face image is input into the CurricularFace model, and the CurricularFace model is used to extract features and convert vectors from the cropped face image to obtain the face feature vector of the important person to be audited;

[0110] Further, the feature vector generation process of the CurricularFace model includes:

[0111] (1) The cropped face image is standardized and converted to the model input format;

[0112] (2) Deep features are extracted through an improved ResNet-50 network, which sequentially passes through a convolutional layer, a residual block group, and a global average pooling layer, and converts a two-dimensional feature map to a one-dimensional feature vector. The improved ResNet-50 network is prior art in the field and is not part of the inventive concept of the present application, and will not be described here.

[0113] (3) Feature mapping is performed through a fully connected layer. The first fully connected layer performs linear transformation on the pooled features and is activated by ReLU. The second fully connected layer further optimizes the features, and the output vector is the reference face feature vector.

[0114] (4) Verify whether the vector dimension is 512, the numerical value is distributed in the [-1, 1] interval, and the modulus length is in a reasonable range (0.8-1.2). If the verification fails, the feature extraction process is re-executed.

[0115] S2.4: The face feature vector of the important person to be audited is enhanced using a biologically inspired spiking neural network to generate an enhanced person feature vector, which is stored in association with the important person image identifier in the important person feature library.

[0116] The enhanced processing procedure of the bio-inspired spiking neural network comprises:

[0117] S2.41: constructing a sandwich structure comprising a time encoding layer, a spiking layer and a synaptic plasticity regulation layer;

[0118] S2.42: inputting the face feature vector of the important person to be audited into the time encoding layer, the time encoding layer assigning 1 time encoding neuron to each dimension of the face feature vector of the important person to be audited, converting the feature values of each dimension into time series pulse signals, weighting all the time series pulse signals through an exponential decay function, forming a pulse sequence, and time aligning the pulse sequences of each dimension to form a standardized pulse sequence;

[0119] Further, the pulse conversion procedure of the time encoding layer comprises:

[0120] (1) assigning 1 time encoding neuron to each dimension of the reference face feature vector, and converting the feature values into time series pulse signals;

[0121] (2) the greater the feature value, the higher the pulse firing frequency;

[0122] (3) weighting the pulse signals through an exponential decay function to obtain a pulse sequence, wherein the recent pulses have higher weights, and the exponential decay function is a prior art content in the field and is not the inventive scheme of the present application, which is not described here;

[0123] (4) time aligning the pulse sequences of all dimensions to ensure subsequent processing at the same time step.

[0124] Further, the feature response and weight preliminary distribution of the spiking layer comprise:

[0125] (1) each neuron calculates the membrane potential according to the frequency and decay weight of the input pulse, the initial value is 0, the membrane potential accumulates with the pulse input and naturally decays with time;

[0126] (2) when the membrane potential exceeds the threshold, a pulse is fired, and the membrane potential is reset to the resting potential at the same time, and the number of pulse firings of each dimension is counted;

[0127] (3) the weight of any dimension is equal to the ratio of the number of pulse firings of the dimension to the total number of firings, ensuring that the sum of the weights is 1 and highlighting important feature dimensions.

[0128] S2.43: The time coding layer transmits the generated standardized pulse sequence to the pulse firing layer, the pulse firing layer performs membrane potential calculation on the standardized pulse sequence of each dimension, and the initial value of the membrane potential is 0, when the membrane potential exceeds the preset threshold, the corresponding dimension neuron fires a pulse and resets the membrane potential to the resting potential, the total number of pulse firings of each dimension is counted, and the ratio of the number of firings of a single dimension to the total number of firings of all dimensions is taken as the weight of the dimension, and the screened pulse sequence and the weight parameter of each dimension are obtained through weight screening;

[0129] S2.44: The pulse firing layer transmits the obtained screened pulse sequence and the weight parameter of each dimension to the synaptic plasticity adjustment layer, the synaptic plasticity adjustment layer adopts the pulse timing-dependent plasticity rule to adjust the synaptic weight according to the time difference of adjacent pulses, and obtains a fusion feature vector;

[0130] Further, the process of adjusting the synaptic weight comprises:

[0131] (1) Based on the pulse timing-dependent plasticity rule, if the time difference between adjacent pulses is less than 5 time steps, the weight is increased by 5%; if the time difference is greater than 10 time steps, the weight is reduced by 3%, and the weight of the complementary feature is enhanced;

[0132] (2) If the weight of a single dimension accounts for more than 70%, trigger the balancing mechanism to avoid feature redundancy, wherein the balancing mechanism is to reduce the dominant weight and increase the secondary weight;

[0133] (3) The feature vector is weighted and enhanced through the optimized weight, and then the invalid features are suppressed through the ReLU function.

[0134] S2.45: The synaptic plasticity adjustment layer performs normalization processing on the generated fusion feature vector, and finally generates an enhanced person feature vector.

[0135] The lightweight face detection network adopting multi-scale fusion performs frame-by-frame face detection on the material copies in the temporary area, comprising:

[0136] S3.1: The system reads the material copies to be processed from the temporary area, and performs frame-by-frame analysis on the material copies to extract a continuous image frame sequence;

[0137] S3.2: Introduce a spatiotemporal attention guided detection mechanism, construct a spatiotemporal feature map for the continuous image frame sequence, calculate the intra-frame spatial attention weight and the inter-frame temporal attention weight through the spatiotemporal attention method, and fuse the intra-frame spatial attention weight and the inter-frame temporal attention weight into the multi-scale fusion lightweight face detection network; the spatial attention weight is used to highlight the key features of the face region in a single frame image; the temporal attention weight is used to associate the face motion features between adjacent frames;

[0138] Further, the specific steps of S3.2 include:

[0139] (1) Preprocessing the continuous image frame sequence parsed from the material copy;

[0140] (2) Constructing a spatio-temporal feature map according to the preprocessed continuous image frame sequence, including:

[0141] Multi-scale feature extraction on single-frame images in the preprocessed continuous image frame sequence: through different convolution layers of a lightweight convolutional neural network, low-scale features, medium-scale features, and high-scale features are extracted respectively, and each scale of feature map retains spatial dimensions, wherein the low-scale features include edges and textures, corresponding to image details, the medium-scale features include face component contours, corresponding to local structures, and the high-scale features include face overall morphology, corresponding to global information;

[0142] Stacking the same scale feature maps of the continuous image frames in the time dimension to form a spatio-temporal feature map;

[0143] (3) Calculating the intra-frame spatial attention weight; the calculation of the intra-frame spatial attention weight focuses on single-frame images, enhances face key area features, suppresses background interference, and improves face detection accuracy, specifically including: first step, spatial feature saliency analysis: for each single-frame spatial feature map of the spatio-temporal feature map, the feature response value of each spatial position, i.e., pixel point, is calculated, the channel dimension is compressed to 1 through global average pooling, and the obtained feature response map is obtained. The higher the response value, the more likely it is that the position belongs to the face area; the second step is to construct a spatial attention weight matrix: input the feature response map into the sigmoid activation function, map the response value to the interval of 0 to 1, and obtain a weight matrix consistent with the size of the spatial feature map. The higher the weight value, the higher the attention degree of the corresponding spatial position;

[0144] (4) Introducing face prior knowledge to constrain the spatial attention weight matrix, including: first step, loading a pre-trained face heat map model; the face heat map model is trained by a large number of face images and can output a probability distribution heat map of face key parts such as eyes, nose, and mouth; the second step is to multiply the spatial attention weight matrix and the heat map pixel by pixel, to strengthen the weight of the high-probability area in the heat map, such as further improving the weight of the eye area, and to weaken the weight of the background area, such as reducing the weight of the background texture area; the third step is to perform weight normalization, calculate the sum of all elements of the weight matrix, divide each element by the sum, and ensure that the sum of the elements of the weight matrix is 1, to avoid affecting the subsequent feature fusion due to the difference in weight scale, and to optimize the spatial attention weight matrix. The optimized spatial attention weight matrix can accurately highlight the features of the face key area in the single-frame image;

[0145] (5) Inter-frame time attention weight calculation; the inter-frame time attention weight calculation focuses on the correlation of adjacent frames, solves the problem of unstable detection caused by face blur and occlusion in dynamic scenes by capturing face motion features, and specifically includes:

[0146] The first step is to extract the inter-frame motion feature: for the same scale spatial feature map of adjacent two frames in the space-time feature map, such as the high scale feature map of the t-th frame and the t+1-th frame, the feature difference of the corresponding spatial position of the two frames is calculated to obtain a motion feature map. The greater the difference, the more intense the motion at that position, such as the feature change caused by face movement;

[0147] The second step is to calculate the time attention basic weight: the motion feature map is globally maximum pooled to compress the channel dimension to 1 to obtain a motion intensity map, and then the intensity value is mapped to the interval of 0 to 1 through a sigmoid activation function to obtain a time attention basic weight matrix. The higher the weight value, the more important the motion feature of the corresponding spatial position;

[0148] (6) Weight optimization of the time attention basic weight matrix combined with motion consistency, including: the first step is to analyze the motion feature maps of adjacent multiple frames, calculate the motion direction and speed of the same spatial position in consecutive frames, if the motion direction of any position remains consistent for 3 consecutive frames and the speed is stable, it is determined that the position is an effective motion area, and its time weight is multiplied by 1.2 for strengthening; if the motion direction frequently changes and the speed fluctuates greatly, it is determined as an invalid motion area, and its time weight is multiplied by 0.8 for weakening; the second step is to perform time dimension normalization, sum the time weights of the same spatial position in consecutive frames, and divide the weight of each frame by the sum to ensure that the weight of the position in the time dimension is 1, avoiding the imbalance of motion features caused by the weight of any frame being too high. The optimized time attention weight matrix can effectively associate the face motion features of adjacent frames, and improve the continuity and stability of face detection in dynamic scenes;

[0149] (7) Through a multi-dimensional fusion strategy, the spatial attention weight matrix and the time attention weight matrix are fused into the multi-scale fusion lightweight face detection network to realize attention-guided face detection, including: first, scale-in weight fusion: for each scale of the spatial feature map, the corresponding spatial attention weight matrix and the time attention weight matrix are weighted and summed to obtain the spatio-temporal attention fusion weight matrix of the scale; second, the spatio-temporal attention fusion weight matrix is multiplied with the spatial feature map of the corresponding scale element by element to obtain an attention-enhanced feature map; the attention-enhanced feature map retains the spatial features of the single-frame face key region and integrates the motion features of adjacent frames, thereby improving the recognition degree of the face features; third, the attention-enhanced feature maps of different scales are adjusted to the same spatial size through upsampling or downsampling, and then the adjusted feature maps are spliced in the channel dimension to form a multi-scale fusion feature map; fourth, the multi-scale fusion feature map is input into the detection head of the lightweight detection network, and the detection head can output the face detection result through anchor box generation, classification prediction and regression prediction. In the detection process, the spatio-temporal attention-enhanced feature map can help the detection head to more accurately identify the face region, reduce the influence of background interference and dynamic blur, and improve the accuracy and recall rate of face detection.

[0150] S3.3: applying the multi-scale fusion lightweight face detection network with spatio-temporal attention-guided detection mechanism to each extracted image for face detection, and locating all face regions contained in each image to generate corresponding bounding box coordinates and confidence scores for each detected face;

[0151] Further, a preset number of anchor boxes are generated for each spatial position of each multi-scale fusion feature map to cover faces of different sizes and shapes.

[0152] Further, after the multi-scale fusion feature map and the anchor boxes are input into the detection head of the multi-scale fusion lightweight face detection network with spatio-temporal attention-guided detection mechanism, the face classification and bounding box regression tasks are performed in parallel, wherein the classification branch outputs the probability that each anchor box belongs to a face through two convolution layers, and uses a sigmoid activation function to map the probability to the interval of 0 to 1 to obtain a confidence score; the regression branch also passes through two convolution layers to predict the offset of the anchor box from the real face box, which is used to correct the anchor box position to a more accurate face region, and finally outputs the face bounding box coordinates and the confidence score of each frame to complete single-frame face detection.

[0153] S3.4: screening the generated face detection result according to a preset confidence threshold, retaining the face region and its corresponding bounding box coordinates whose confidence score is higher than the confidence threshold to obtain the face detection result.

[0154] The determination process of the matching threshold includes:

[0155] S4.1: Establishing a material copy type-matching threshold mapping table to store initial matching thresholds corresponding to different material copy types;

[0156] Further, the specific steps of S4.1 include: first, the system constructs a three-level material copy type system according to the acquisition device-scene type-format feature, formulates classification standards and encodes; second, sample data is collected for each type, the matching relationship is labeled and high-quality samples are selected; third, the similarity of samples is calculated, the initial threshold is determined in combination with the error rate index and is calibrated according to the sample quality; fourth, the mapping table structure containing type identification, classification information, threshold, etc. is designed, the memory database and the relational database are combined for storage, and the index is established; fifth, the threshold data is written into the mapping table, the data reliability is ensured through integrity, accuracy and format verification; finally, the query and update interfaces are developed, the permission control and performance optimization mechanism are set, and the establishment and activation of the material copy type-matching threshold mapping table are completed.

[0157] S4.2: Reading the material copy type identification in the audit task, and calling the corresponding initial matching threshold from the material copy type-matching threshold mapping table;

[0158] S4.3: Dynamically adjusting the initial matching threshold in combination with the scene features of the extracted frames of the material copy to obtain the finally adapted matching threshold.

[0159] Further, the specific steps of S4.3 include:

[0160] (1) Preprocessing and dimension division of scene feature data, the system calls the complete scene feature data of the extracted frames of the material copy from the temporary storage area; the scene feature data includes static features and dynamic features;

[0161] (2) Effectiveness screening of scene feature data, invalid data caused by frame image damage and feature extraction abnormality is eliminated, and then dimension classification is performed according to static features and dynamic features;

[0162] (3) Perform scene feature dimension weight allocation, including: the system sets a basic weight for each feature dimension according to the influence degree of different scene features on the face feature comparison result, wherein the illumination feature has the most direct influence on the face image quality, and the highlight overexposure and shadow shielding will significantly change the presentation of the face feature, therefore the basic weight of the illumination feature is set to the highest, such as the basic weights of the highlight area proportion, the shadow area proportion and the illumination uniformity are set to 0.2, 0.2 and 0.15 respectively; the contour number and distribution density in the structure feature will affect the accuracy of face detection, and the object dense scene is easy to cause face misrecognition, therefore the basic weights of the contour number, length and distribution density are set to 0.1, 0.05 and 0.05 respectively; the change amplitude and change type in the dynamic scene feature will affect the definition of the face image, and the fast lens movement is easy to cause image blur, therefore the basic weights of the change amplitude and change type are set to 0.15 and 0.1 respectively; the texture feature has relatively small interference on the face feature, and the basic weights of the contrast, correlation, energy and homogeneity are set to 0.02, 0.02, 0.01 and 0.01 respectively; the system fine-tunes the basic weight according to the actual scene attribute of the material copy, such as indoor monitoring material, outdoor snapshot material, high-definition video material and the like, for example, the outdoor material is more affected by the illumination change, and the total weight of the illumination feature can be increased by 0.05, and the structure feature weight is reduced by 0.05, to ensure that the weight allocation is adapted to the actual characteristics of the material scene;

[0163] (4) For the scene feature of each effective frame, the system calculates the deviation degree of the actual value of each feature dimension from the standard threshold value one by one, and determines the adjustment coefficient of the dimension accordingly, taking the illumination feature as an example, the standard threshold value of the illumination uniformity is preset to 0.6, if the illumination uniformity of any frame is 0.8, which exceeds the standard threshold value by 0.2, it means that the illumination condition is good, and the face feature is clear, at this time, the adjustment coefficient is calculated as 0.9, and the adjustment coefficient less than 1 indicates that the matching threshold value needs to be reduced, and the specific calculation logic is that the adjustment coefficient is equal to the difference value obtained by subtracting the first difference value from 1, and the first difference value is the average value of the difference between the actual value and the standard threshold value; if the illumination uniformity is 0.4, which is lower than the standard threshold value by 0.2, it means that the illumination condition is poor, and the face feature may be distorted, and the adjustment coefficient is equal to the sum value obtained by adding the first difference value to 1, that is, the adjustment coefficient of the illumination feature of the frame is 1.1;

[0164] (5) For the contour number in the structure feature, the standard threshold value is preset to 30, if the contour number of any frame is 40, which exceeds the standard threshold value by 10, it means that the scene is object dense, which is easy to interfere with face recognition, and the adjustment coefficient is equal to the sum value obtained by adding the second difference value to 1, and the second difference value is the difference between the actual value and the standard threshold value, and is multiplied by 0.01; if the contour number is 20, which is lower than the standard threshold value by 10, the scene is simple, and the adjustment coefficient is equal to the difference value obtained by subtracting the second difference value from 1;

[0165] (6) In the dynamic scene feature, the standard threshold of the change amplitude is 0.3. If the change amplitude of any frame is 0.5, which exceeds the standard threshold 0.2, it means that the lens moves fast and the image may be blurred. Therefore, the adjustment coefficient is equal to the sum of 1 and the third difference value, and the third difference value is the difference between the actual value and the standard threshold, multiplied by 0.6. If the change amplitude is 0.1, which is lower than the standard threshold 0.2, the scene is stable. Therefore, the adjustment coefficient is equal to the difference between 1 and the third difference value;

[0166] (7) The adjustment coefficient of each feature dimension is calculated according to the logic of (4)-(6);

[0167] (8) The system multiplies the adjustment coefficient of each scene feature dimension of each effective frame with its corresponding weight to obtain the weighted adjustment value of each dimension. Then, the weighted adjustment values of all dimensions are summed up to obtain the comprehensive adjustment coefficient of the frame;

[0168] (9) The initial matching threshold corresponding to the material copy type is retrieved from the material copy type-matching threshold mapping table. The initial matching threshold is multiplied by the comprehensive adjustment coefficient to obtain the preliminary adjusted threshold. Then, the system will make a second fine adjustment combined with the scene feature trend of the material copy. If the scene feature of the material frame presents a gradually optimized trend, such as gradually increasing uniformity of illumination and gradually decreasing change amplitude, it means that the face feature of the subsequent frame will be clearer. Therefore, the preliminary adjusted threshold can be reduced by 0.02. If the scene feature presents a gradually deteriorating trend, such as gradually increasing highlight area ratio and gradually increasing contour number, the preliminary adjusted threshold needs to be increased by 0.02. Finally, the system checks whether the final threshold is within the preset reasonable range, such as 0.6-0.85. If it is within the range, the value is determined as the final matching threshold. If it is out of the range, such as the final threshold calculation result is 0.58, which is lower than the lowest reasonable value 0.6, the final threshold is set to 0.6. If the calculation result is 0.88, which is higher than the highest reasonable value 0.85, the final threshold is set to 0.85. This ensures that the final threshold meets the actual needs of the scene feature and guarantees the accuracy and reliability of face comparison, completing the dynamic adjustment of the initial matching threshold.

[0169] The obtained material face feature vector and the enhanced person feature vector are compared in layers, including:

[0170] A1: An inverted index is constructed for all enhanced person feature vectors in advance. The inverted index takes the dimension value of the enhanced person feature vector as the index key, and each index key is associated with a set of enhanced person feature vectors containing the corresponding dimension value;

[0171] A2: After obtaining a multi-modal fusion feature vector, extract the top N dimension numbers with the highest values in the material face feature vector as key dimension values, search and filter out the enhanced person feature vector set containing the key dimension values based on the constructed inverted index, and take the enhanced person feature vector set as the preliminary candidate set;

[0172] A3: Calculate the cosine similarity and Manhattan distance between the material face feature vector and each enhanced person feature vector in the preliminary candidate set, sort the weighted calculation results in descending order, select the top K enhanced person feature vectors, and form a candidate feature vector subset, wherein the cosine similarity and Manhattan distance calculation formulas are existing technical contents in the art and are not the creative schemes of the present application, and are not described here;

[0173] A4: Calculate the exact similarity between the material face feature vector and each candidate vector by weighting Euclidean distance, Mahalanobis distance, and Bhattacharyya distance, and sort them, and at the same time, combine the scene features in the multi-modal fusion feature vector to assist in judgment, and obtain the final similarity ranking result, wherein the Euclidean distance, Mahalanobis distance, and Bhattacharyya distance calculation formulas are existing technical contents in the art and are not the creative schemes of the present application, and are not described here.

[0174] If the first exact similarity value in the final similarity ranking result is greater than or equal to the preset matching threshold, it is determined that the material face matches the corresponding important person successfully, and an comparison result containing the person identifier and the exact similarity value is generated, and the corresponding person identifier is labeled on the material copy of the frame;

[0175] If the first exact similarity value in the final similarity ranking result is less than the preset matching threshold, it is determined that the matching fails, and an unknown or reserved special identification label is labeled;

[0176] A5: After associating the comparison result and the labeling information, encapsulate and store them in the audit task database; the labeling information includes timestamp, material file ID, frame number, and label content; the audit task database adopts a strategy combining time sequence partitioning and hash table partitioning.

[0177] The audit task database adopts a strategy combining time sequence partitioning and hash table partitioning, comprising:

[0178] A5.1: Pre-set the number of table partitions of the audit task database as M;

[0179] A5.2: When a new audit task is generated, the system obtains the unique identifier of the audit task as the hash table key, and performs hash calculation on the audit task identifier using the hash table algorithm, to obtain a hash value, wherein the hash table algorithm is a prior art in the field, and is not the creative scheme of the present application, and will not be described here;

[0180] A5.3: The obtained hash value is subjected to modulo operation with the preset number of tables M, and the audit task storage table is determined according to the modulo result;

[0181] A5.4: A time partitioning strategy is used in the determined table, and the comparison results and marking information generated each month are stored in the partition table corresponding to the month;

[0182] A5.5: A table routing table is established to record the correspondence between the audit task identifier range and the table and the partition table, and the table routing table and the audit task database after table storage are output.

[0183] Embodiment 2

[0184] Please refer to Figure 3 Another embodiment provided by the present application: a face auditing system based on feature vector comparison, comprising:

[0185] a management center module, a feature fusion module, a material detection module, a marking module, and a report generation module;

[0186] The management center module is used for receiving important person images uploaded by users and constructing a database, and is also used for initial parameter configuration and storage of audit tasks, to provide basic data and task basis for the face auditing process;

[0187] The feature fusion module is used for preprocessing, model calculation and feature fusion of important person images, to generate enhanced person feature vectors;

[0188] The material detection module is used for downloading a copy of the material to be audited and performing frame-by-frame analysis, to complete face detection through a multi-scale fusion lightweight face detection network of a fusion spatiotemporal attention guide detection mechanism, to synchronously extract scene features and generate material face feature vectors, to provide material end data for subsequent multi-modal fusion and comparison;

[0189] The marking module is used for determining an adaptive matching threshold, to complete similarity calculation of the material face feature vectors and the enhanced person feature vectors through hierarchical comparison, to combine scene features for auxiliary judgment and complete automatic marking, to store the results in the audit task database, and to provide data support for task report generation;

[0190] A report generation module is configured to generate a task report with a check code based on the audit task database data, so as to ensure data security and traceability of the task result.

[0191] The management center module comprises an important person library unit and a person tagging unit.

[0192] The important person library unit is configured to receive important person images uploaded by a user, assign a unique identifier to each image, and store the important person images and corresponding identifiers to form a structured database, so as to ensure that a target image can be accurately retrieved through the identifier.

[0193] The person tagging unit is configured to provide an audit task parameter input interface, receive parameters such as a task name, a to-be-audited time period range, and important person image identifiers input by a user, and perform legality verification on the input to-be-audited time period range, such as judging whether the time period range has logical contradictions or is within the system processable range, generating a unique audit task identifier, associating the audit task parameters with the identifier, and storing the parameters and identifier in a to-be-processed task table of an audit task database.

[0194] The feature fusion module comprises a preprocessing unit, a feature calculation unit, and a heuristic fusion unit.

[0195] The preprocessing unit is configured to retrieve corresponding important person images from the important person library unit according to important person image identifiers in the audit task parameters, perform face region detection and alignment, image normalization, and illumination compensation operations on the retrieved images, and output preprocessed face images.

[0196] The feature calculation unit is configured to input the preprocessed face images into a RetinaFace model and a CurricularFace model respectively to obtain enhanced person feature vectors.

[0197] The heuristic fusion unit is configured to take a biological heuristic pulse neural network as a core, construct a sandwich structure comprising a time coding subunit, a pulse firing subunit, and a synaptic plasticity adjustment subunit, complete weighted fusion of the two features, and output the enhanced person feature vectors.

[0198] The material detection module comprises a material analysis unit and a feature generation unit.

[0199] The material analysis unit is configured to download material copies in a corresponding time period from a system data source according to a to-be-audited time period range in the audit task parameters, store the material copies in a temporary area, and perform frame-by-frame analysis on the material copies in the temporary area to extract continuous image frame sequences.

[0200] The feature generation unit is configured to perform encoding conversion on the output face region, generate a material face feature vector, synchronously extract a scene feature of the material copy, fuse the material face feature vector and the scene feature, and form a multi-modal fusion feature vector.

[0201] The embodiments of the present application are described above with reference to the drawings; however, the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, not restrictive. A person of ordinary skill in the art can make changes, modifications, replacements and variations to the above-described embodiments without departing from the spirit and scope of the present application, and these are all within the scope of the present application.

[0202] If the technical solutions of the present disclosure involve personal information, the product applying the technical solutions of the present disclosure has explicitly informed the personal information processing rules before processing the personal information and has obtained the personal independent consent. If the technical solutions of the present disclosure involve sensitive personal information, the product applying the technical solutions of the present disclosure has obtained the personal independent consent before processing the sensitive personal information and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as a camera, a clear and prominent sign is set to inform that the personal information collection range has been entered and the personal information will be collected. If the individual voluntarily enters the collection range, it is considered to agree to collect the personal information. Or, on the device for processing personal information, the personal information processing rules are informed by using obvious signs / information, and the personal authorization is obtained by means of pop-up information or asking the individual to upload the personal information by himself / herself. The personal information processing rules can include the personal information processor, the purpose of processing personal information, the processing method, and the type of processed personal information, etc.

Claims

1. A face verification method based on feature vector comparison, characterized in that, include: Users can upload images of important figures through the important figures database in the management center. At the same time, they can create review tasks through the tagging module, select the time period to be reviewed and the uploaded images of important figures. Multimodal feature extraction and fusion are performed on selected images of important figures to generate enhanced feature vectors of the figures; Based on the review task set by the user, download copies of the materials within the time period to be reviewed to the temporary area. Use a lightweight face detection network with multi-scale fusion to perform frame-by-frame face detection on the material copies in the temporary area, extract scene features of the material copies at the same time, and encode and convert each detected face image to obtain the material face feature vector. Then, fuse the material face feature vector with the scene features to form a multimodal fusion feature vector. The matching threshold is determined based on the type of material copy. The face feature vector of the material in the obtained multimodal fusion feature vector is compared with the enhanced human feature vector in layers. The scene features are combined to assist in the judgment. The automatic labeling is performed based on the comparison results, and the comparison results and labeling information are stored in the review task database. The temporary area material copy is deleted and the inode is released using the standard file system deletion command. The process is iterated until all materials have been processed, and a task report with a checksum is generated.

2. The face verification method based on feature vector comparison as described in claim 1, characterized in that, The process of establishing an audit task through the tagging module includes: The system calls the tagging module to receive the review task parameters input by the user. The review task parameters include the task name, the time period range to be reviewed, and one or more important person image identifiers selected from the important person database. The important person database is a database formed after the user uploads important person images through the management center, which stores important person images and their corresponding unique identifiers. After verifying the legality of the received time period range to be reviewed, the system generates a review task identifier, associates the review task parameters with the review task identifier, and stores them in the pending task table of the review task database.

3. The face verification method based on feature vector comparison as described in claim 2, characterized in that, The process of generating the enhanced character feature vector includes: The system selects the image identifier of the important person corresponding to the important person based on the audit task parameters, and reads the image of the important person that uniquely corresponds to the image identifier from the important person database; The received images of important figures to be reviewed are input into the RetinaFace model. The RetinaFace model identifies and crops the facial regions in the images of important figures to be reviewed, and obtains the cropped facial images. The cropped face image is input into the CurricularFace model. The CurricularFace model extracts features and transforms vectors from the cropped face image to obtain the face feature vector of the important person to be reviewed. A bio-inspired spiking neural network is used to enhance the facial feature vectors of important individuals to be reviewed, generating enhanced feature vectors of individuals, which are then associated with the image identifiers of important individuals and stored in the feature database of important individuals.

4. The face verification method based on feature vector comparison as described in claim 3, characterized in that, The enhancement process of the bio-inspired spiking neural network includes: Construct a sandwich structure comprising a time-coding layer, a pulse firing layer, and a synaptic plasticity modulation layer; The facial feature vector of the important person to be reviewed is input into the time coding layer. The time coding layer assigns one time coding neuron to each dimension of the facial feature vector of the important person to be reviewed, converts the feature values ​​of each dimension into time series pulse signals, and weights all time series pulse signals through an exponential decay function to form a pulse sequence. Then, the pulse sequences of each dimension are time-aligned to form a standardized pulse sequence. The time-coding layer transmits the generated standardized pulse sequence to the pulse firing layer. The pulse firing layer calculates the membrane potential for each dimension's standardized pulse sequence, with the initial value of the membrane potential being 0. When the membrane potential exceeds a preset threshold, the corresponding dimension's neuron fires a pulse and resets the membrane potential to the resting potential. The total number of pulse firings for each dimension is counted, and the ratio of the number of firings in a single dimension to the total number of firings in all dimensions is used as the weight of that dimension. Through weight filtering, the filtered pulse sequence and the weight parameters for each dimension are obtained. The pulse firing layer inputs the filtered pulse sequence and weight parameters of each dimension into the synaptic plasticity adjustment layer. The synaptic plasticity adjustment layer uses the pulse time-dependent plasticity rule to adjust the synaptic weights according to the time difference between adjacent pulses to obtain the fused feature vector. The synaptic plasticity modulation layer performs normalization on the generated fused feature vector, ultimately generating an enhanced character feature vector.

5. The face verification method based on feature vector comparison as described in claim 4, characterized in that, The lightweight face detection network employing multi-scale fusion performs frame-by-frame face detection on the material copies in the temporary region, including: The system reads a copy of the material to be processed from the temporary area and parses the copy frame by frame to extract a continuous sequence of image frames. A spatiotemporal attention-guided detection mechanism is introduced to construct spatiotemporal feature maps for continuous image frame sequences. Intra-frame spatial attention weights and inter-frame temporal attention weights are calculated using a spatiotemporal attention approach, and these weights are integrated into a lightweight face detection network with multi-scale fusion. The spatial attention weights are used to highlight key features of the face region in a single frame image, while the temporal attention weights are used to associate face motion features between adjacent frames. A lightweight face detection network with multi-scale fusion incorporating a spatiotemporal attention-guided detection mechanism is applied to each extracted image frame for face detection. By locating all face regions contained in each image frame, corresponding bounding box coordinates and confidence scores are generated for each detected face region. The generated face detection results are filtered according to a preset confidence threshold, and face regions with confidence scores higher than the confidence threshold and their corresponding bounding box coordinates are retained to obtain the face detection results.

6. The face verification method based on feature vector comparison as described in claim 5, characterized in that, The process of determining the matching threshold includes: Establish a mapping table between material copy type and matching threshold to store the initial matching threshold corresponding to different material copy types; Read the material copy type identifier in the review task and retrieve the corresponding initial matching threshold from the material copy type-matching threshold mapping table; By combining the scene features of the extracted frames from the material copy, the initial matching threshold is dynamically adjusted to obtain the final adapted matching threshold.

7. The face verification method based on feature vector comparison as described in claim 6, characterized in that, The obtained facial feature vector of the source material is compared with the enhanced human feature vector in a hierarchical manner, including: An inverted index is pre-constructed for all enhanced character feature vectors; the inverted index uses the dimension value of the enhanced character feature vector as the index key, and each index key is associated with a set of enhanced character feature vectors containing the corresponding dimension value; Once a multimodal fusion feature vector is obtained, the top N dimensions with the highest values ​​in the face feature vector of the source material are extracted as key dimension values. Based on the constructed inverted index, a search is performed to select an enhanced human feature vector set containing the key dimension values ​​as a preliminary candidate set. Calculate the cosine similarity and Manhattan distance between the face feature vector of the source material and each enhanced person feature vector in the preliminary candidate set. Sort the results in descending order according to the weighted calculation of the two, and select the top K enhanced person feature vectors to form a subset of candidate feature vectors. The obtained candidate feature vector subsets are weighted using Euclidean distance, Mahalanobis distance and Bachian distance to calculate the precise similarity between the source face feature vector and each candidate vector and sort them. At the same time, scene features in the multimodal fusion feature vector are combined for auxiliary judgment to obtain the final similarity ranking result. If the exact similarity value ranked first in the final similarity ranking results is greater than or equal to the preset matching threshold, it is determined that the face in the material is successfully matched with the corresponding important person, and a comparison result containing the person's identifier and exact similarity value is generated. At the same time, the corresponding person's identifier is tagged for the copy of the frame material. If the exact similarity value of the first-ranked result in the final similarity ranking is less than the preset matching threshold, it is judged as a matching failure and marked with an unknown or reserved special label. The comparison results and tagging information are associated, encapsulated, and stored in the review task database; the tagging information includes timestamp, material file ID, frame number, and tag content; the review task database adopts a strategy combining time-series partitioning and hash table partitioning.

8. The face verification method based on feature vector comparison as described in claim 7, characterized in that, The audit task database employs a strategy combining time-series partitioning and hash-based table partitioning, including: The number of tables in the audit task database is preset to M; When a new review task is generated, the system obtains the unique identifier of the review task as the partition key of the hash partition table, and uses the hash partition algorithm to perform hash calculation on the review task identifier to obtain the hash value. The obtained hash value is moduloed with the preset number of table partitions M, and the table partition where the audit task is stored is determined based on the modulo result. Within the defined partitioned tables, a time-based partitioning strategy is adopted, with each month serving as the partitioning unit to store the comparison results and labeling information generated each month into the corresponding month's partitioned table. Create a table partitioning routing table to record the correspondence between the audit task identifier range and the partitioned table and the sub-table. Output the table partitioning routing table and the audit task database after partitioning.

9. The face verification method based on feature vector comparison as described in claim 8, characterized in that, The step of deleting the temporary area material copy and releasing the inode using the standard file system deletion command includes: After completing all frame processing and face comparison for a media copy, the system calls the storage management service to send a standard file system deletion command to the media copy in the temporary area, notifying the storage device to perform a file deletion operation. At the same time, it releases the file system inode, updates the temporary area storage status table, and marks the media copy as processed. The temporary area storage status table is a table that records the processing status of each media copy in the temporary area. The file system inode is a data structure used to record the storage location, size attributes, and access permissions of the media copy files.

10. A face verification system based on feature vector comparison, used to implement the face verification method based on feature vector comparison as described in any one of claims 1-9, characterized in that, include: The module includes a management center module, a feature fusion module, a material detection module, a tagging module, and a report generation module. The management center module is used to receive images of important figures uploaded by users and build a database, and is also used to review the initial parameter configuration and storage of tasks. The feature fusion module is used to preprocess, perform model calculations and feature fusion on images of important figures to generate enhanced feature vectors of the figures. The material detection module is used to download copies of the materials to be reviewed and parse them frame by frame. It completes face detection through a lightweight face detection network that integrates a spatiotemporal attention-guided detection mechanism and extracts scene features and generates face feature vectors for the materials. The tagging module is used to determine the appropriate matching threshold, calculate the similarity between the face feature vector of the material and the enhanced human feature vector through hierarchical comparison, combine scene features to assist in judgment and complete automatic tagging, and store the results in the review task database. The report generation module is used to generate task reports with verification codes based on data from the audit task database.

Images