Face management method and system based on face recognition

Abstract

The invention discloses a face recognition-based meeting management method and system. The method comprises the steps of face data acquisition, bidirectional identity authentication, meeting sequence management, real-time bidirectional monitoring and meeting data archiving. The invention relates to the technical field of agricultural machinery intelligent management, systematically introduces a two-way identity authentication and real-time two-way monitoring mechanism, and realizes a closed-loop meeting management process from pre-meeting identity verification, in-meeting anomaly recognition to post-meeting data archiving. A high-credibility face template is constructed through dynamic consistency scoring, attitude sensitivity sorting and lightweight distillation network compression, so that the stability and accuracy of identity authentication are improved; meanwhile, a three-dimensional convolution anomaly detection network based on optimal feature matching is designed, abnormal behaviors such as face shielding and personnel replacement in the meeting process are recognized in real time, and a three-level response mechanism is triggered; according to the method, the meeting safety, controllability and intelligent level of prison personnel are remarkably enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition video surveillance, and in particular to a meeting management method and system based on face recognition. Background Art

[0002] The face recognition-based meeting management method and system is a technical solution that collects, compares, and verifies facial biometric information to achieve automated identity verification and intelligent process management for both parties. Its core function is to leverage computer vision and pattern recognition technologies to upgrade the traditional manually verified meeting process to efficient, accurate, and traceable digital management. Specifically, it ensures "identity" through real-time face matching to prevent identity theft; maintains identity authenticity during meetings through dynamic monitoring to eliminate mid-personnel substitutions; and forms a complete electronic chain of evidence for meetings through data archiving. This system significantly improves meeting management efficiency in security scenarios, ensuring safety while reducing manual intervention costs. It is a typical application of biometric technology in the field of identity authentication.

[0003] However, the existing methods for managing meetings with visited persons generally have technical problems such as a static identity authentication process, a one-way verification object, and an inability to dynamically identify abnormal behavior during the monitoring process; the existing identity authentication systems for meeting persons often have technical problems in actual scenarios, such as large differences in sampled image quality, diverse posture changes, and unstable authentication results; and the existing real-time meeting monitoring systems only have basic video acquisition functions and lack automatic abnormality identification and response mechanisms. Summary of the Invention

[0004] The technical solution adopted by the present invention is as follows: The present invention provides a meeting management method based on face recognition, which includes the following steps:

[0005] Step S1: face data collection;

[0006] Step S2: two-way identity authentication;

[0007] Step S3: meeting order management;

[0008] Step S4: real-time two-way monitoring;

[0009] Step S5: Archiving the meeting data.

[0010] Furthermore, in step S1, the facial data collection is used to collect and verify the identity document information and facial images of the visitor and the visited person, specifically by collecting the identity information of the visited person to obtain facial feature template data of the visited person, and by real-time collection of the visitor's face, performing facial feature extraction of the visitor to obtain facial template data of the visitor, and by fusing the facial feature template data of the visited person and the facial template data of the visitor to obtain facial feature template data for two-way identity authentication;

[0011] The data fusion is specifically to perform structural encoding on the facial feature template data of the person being visited and the facial template data of the visiting person after standardization, to construct an identity matching pair data object, and to retain the continuous frame facial feature templates, acquisition timestamps, acquisition device numbers, image quality scores and identity association identifiers, to obtain two-way identity authentication facial feature template data.

[0012] Furthermore, in step S2, the two-way identity authentication is used to simultaneously perform local and remote facial image comparison on the visiting end and the visited end. Specifically, based on the two-way identity authentication facial feature template data, a two-way identity authentication method improved by dynamic consistency scoring is used to perform two-way identity authentication to obtain two-way identity consistency authentication data, including the following steps:

[0013] Step S21: dynamic consistency score calculation, specifically, constructing an improved multi-dimensional similarity information calculation function based on the two-way identity authentication facial feature template data, performing dynamic consistency score calculation, and obtaining facial feature dynamic consistency score parameters;

[0014] The improved multi-dimensional similarity information calculation function specifically calculates the dynamic consistency score by calculating the feature space similarity, the acquisition interval time attenuation similarity and the clarity compensation similarity;

[0015] Step S22: selecting a stable face template, specifically performing transformation gradient calculations at different posture angles for each face feature template in the two-way identity authentication face feature template data to obtain a posture change sensitivity parameter, and using the posture change sensitivity parameter as a stability score to rank the feature stability of the two-way identity authentication face feature template data, and selecting the face feature template data with the highest feature stability as the stable face template data to obtain a stable face feature template;

[0016] Step S23: Dynamic credibility fusion scoring, specifically performing dynamic credibility comprehensive scoring fusion calculation based on the facial feature dynamic consistency scoring parameter, the stable facial feature template, and the image quality score in the two-way identity authentication facial feature template data to obtain a facial template credibility comprehensive scoring parameter;

[0017] Step S24: optimal template construction, specifically by splicing and fusing the visitor and the visited person features to the stable face feature template, and constructing a light distillation network to compress the feature dimension according to the spliced and fused stable face feature template data, to obtain the bidirectional identity authentication template data;

[0018] Step S25: bidirectional identity authentication, specifically performing bidirectional identity authentication according to the face template credibility comprehensive score parameter in the bidirectional identity authentication template data, and performing score classification processing according to the specific score of the credibility comprehensive score parameter, to obtain bidirectional identity consistency authentication data;

[0019] The bidirectional identity consistency authentication data specifically includes bidirectional identity authentication template data and bidirectional identity consistency authentication results;

[0020] The score classification processing specifically sets the bidirectional identity consistency authentication result with the face template credibility comprehensive score parameter greater than or equal to 0.8 as passing the verification, and sets the bidirectional identity consistency authentication result less than 0.8 as failing the verification.

[0021] Further, in step S3, the meeting order management is used to manage the meeting queuing order of the visited person and the visitor, specifically by sequentially performing meeting order queue priority judgment, meeting room resource matching scheduling and call number notification according to the meeting data passing the verification in the bidirectional identity consistency authentication data, to perform meeting order management, obtain meeting queuing scheduling results, and perform meeting call number notification according to the meeting queuing scheduling results, to perform the meeting of the visited person;

[0022] The meeting order queue priority judgment specifically sets the lawyer visit as the highest priority, and sequentially queues the meeting order queue according to the reservation time;

[0023] The call number notification specifically pushes the call number information and performs voice broadcast through the MQTT Internet of Things protocol.

[0024] Further, in step S4, the real-time bidirectional monitoring is used to continuously monitor the dynamic face recognition of both parties during the meeting, specifically by collecting real-time face image data according to the local camera of the meeting room after the meeting parties enter the meeting room according to the bidirectional identity consistency authentication data, and performing real-time bidirectional monitoring through the improved feature comparison double-path anomaly detection method, to obtain bidirectional identity monitoring state data, including the following steps:

[0025] Step S41: feature comparison optimization, specifically, by using a lightweight image recognition model, extracting features based on the real-time facial image data to obtain real-time facial feature data, and constructing an improved cosine similarity with intervention of a posture change sensitivity parameter to calculate feature comparison similarity between the real-time facial feature data and the bidirectional identity consistency authentication data to obtain a feature comparison similarity parameter, and using the real-time facial image with the highest feature comparison similarity parameter as monitoring data input to obtain the best matching real-time feature data;

[0026] Step S42: dual-path anomaly monitoring, specifically constructing a lightweight three-dimensional convolutional neural network, performing dual-path anomaly monitoring based on the optimal matching feature data, and obtaining anomaly monitoring score data;

[0027] The dual-path anomaly monitoring specifically includes facial occlusion anomalies and personnel change anomalies;

[0028] Step S43: Real-time fusion decision making, specifically, performing three-level abnormality monitoring decisions based on the abnormality monitoring score data to obtain abnormal status data, and performing two-way detection on both parties of the meeting to obtain two-way abnormal status data;

[0029] Step S44: abnormal monitoring feedback, specifically, based on the abnormal state in the two-way abnormal state data, when the abnormal state is a warning state, a prompt sound is sent to the meeting room; when the abnormal state is an abnormal alarm state, the current abnormal frame data is automatically intercepted and stored, and a suggestion to terminate the meeting is sent via MQTT to obtain the two-way identity monitoring status data;

[0030] The two-way identity monitoring status data specifically includes two-way abnormal status data, abnormal frame data and meeting termination instruction sending record data.

[0031] Furthermore, in step S5, the meeting data archiving is used to structure the audio and video data and identity verification information generated during the meeting. Specifically, after the meeting is completed, the multimodal data generated during the meeting is classified and stored, and bound to the corresponding meeting record to obtain a meeting archive data set, including the following steps:

[0032] Step S51: Archiving audio and video data, specifically starting synchronous audio and video recording at the beginning of the meeting, compressing and encoding using a standard encoding protocol, and logically segmenting the data into 5-minute segments to obtain meeting audio archive data and meeting video archive data;

[0033] Step S52: Archiving the identity authentication log, specifically, collecting the current two-way identity consistency authentication data, the real-time identification frame log during the meeting, and the abnormal alarm record, by timestamp and unique meeting number, to obtain the identity authentication log archive data and abnormal identification alarm information;

[0034] Step S53: Generate a structured data index, specifically using the meeting unique number as the primary index to generate a metadata index table to obtain structured data; the structured data specifically includes the meeting unique number, information about the meeting participants, meeting start and end times, meeting video file path, two-way identity consistency authentication data, and monitoring anomaly data;

[0035] Step S54: Archiving data storage and backup, specifically, synchronously writing the structured data into the local database and the central archiving server to obtain the meeting archive data set;

[0036] The meeting archive data set specifically includes meeting audio archive data, meeting video archive data, identity authentication log archive data and abnormality identification alarm information.

[0037] The present invention provides a face recognition-based meeting management system, which includes a face data acquisition module, a two-way identity authentication module, a meeting sequence management module, a real-time two-way monitoring module, and a meeting data archiving module;

[0038] The face data collection module is used to collect face data, obtain face feature template data for two-way identity authentication through face data collection, and send the face feature template data for two-way identity authentication to the two-way identity authentication module and the meeting order management module;

[0039] The two-way identity authentication module is used for two-way identity authentication, obtains two-way identity consistency authentication data through two-way identity authentication, and sends the two-way identity consistency authentication data to the meeting order management module and the real-time two-way monitoring module;

[0040] The meeting order management module is used for meeting order management, through which meeting order management, meeting queue scheduling results are obtained and meetings with the visited persons are conducted;

[0041] The real-time two-way monitoring module is used for real-time two-way monitoring, obtains two-way identity monitoring status data through real-time two-way monitoring, and sends the two-way identity monitoring status data to the meeting data archiving module;

[0042] The meeting data archiving module is used for archiving meeting data, and obtains a meeting archive data set by archiving the meeting data.

[0043] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0044] (1) In response to the technical problems commonly found in existing visitor management methods, such as static identity authentication processes, one-way authentication objects, and the inability to dynamically identify abnormal behaviors during the monitoring process, the present invention systematically integrates a "two-way identity authentication + real-time two-way monitoring" mechanism, performs two-path authentication on the identities of the visitor and the visitee before the meeting, and continuously performs two-way facial feature matching and abnormal status monitoring during the meeting, thus constructing a closed-loop, full-process dynamic identity management architecture;

[0045] (2) To address the technical issues often encountered in existing meeting person identity authentication systems in actual scenarios, such as large differences in sampled image quality, diverse posture changes, and unstable authentication results, the present invention introduces an authentication process that integrates dynamic consistency scoring mechanism + posture change sensitivity ranking + lightweight distillation network fusion optimization, significantly improving the accuracy and credibility of identity matching under complex acquisition conditions;

[0046] (3) In response to the technical problem that the existing real-time meeting monitoring system only has basic video acquisition functions and lacks automatic anomaly recognition and response mechanisms, the present invention designs a dual-path three-dimensional convolutional anomaly detection network based on optimal feature matching, which realizes real-time detection and a three-level response feedback mechanism for risky behaviors such as people blocking people (such as wearing masks and sunglasses) and putting people in the camera during the meeting. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flow chart of a face recognition-based meeting management method provided by the present invention;

[0048] Figure 2 A schematic diagram of a face recognition-based meeting management system provided by the present invention;

[0049] Figure 3 This is a flow chart of the two-way identity authentication process in step S2;

[0050] Figure 4 This is a flow chart of real-time bidirectional monitoring in step S4.

[0051] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0053] In the description of the present invention, it should be understood that terms such as "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they should not be understood as limiting the present invention.

[0054] Example 1, see Figure 1 The present invention provides a face recognition-based meeting management method, which includes the following steps:

[0055] Step S1: face data collection;

[0056] Step S2: two-way identity authentication;

[0057] Step S3: meeting order management;

[0058] Step S4: real-time two-way monitoring;

[0059] Step S5: Archiving the meeting data.

[0060] By performing the above operations, the present invention systematically integrates the "two-way identity authentication + real-time two-way monitoring" mechanism to address the technical problems commonly existing in existing meeting management methods for visited persons, such as static identity authentication process, one-way verification object, and inability to dynamically identify abnormal behavior during the monitoring process. It performs two-way identity authentication + real-time two-way monitoring before the meeting, and continuously performs two-way facial feature matching and abnormal status monitoring during the meeting, thus constructing a closed-loop full-process dynamic identity management architecture. Specifically, traditional systems usually only collect faces at the visiting end and compare them with the database. The visited person only relies on the management desk's dispatch list or identity certificate, and there is no actual identity verification process, which is prone to loopholes such as impersonation and registration errors. Even if equipped with video surveillance, it is impossible to determine whether the current image is of the authenticated person. Monitoring is only used as a means of post-examination and lacks active recognition capabilities. The present invention constructs facial templates on both the visited and visiting ends and binds them to authentication records. During the meeting, continuous comparison is performed based on the template, and the matching stability is improved by combining posture intervention cosine similarity. This realizes an integrated closed loop of identity authentication and behavior supervision of "pre-meeting confirmation, in-meeting supervision, and post-meeting archiving", breaking through the existing structural limitations of only static verification.

[0061] Example 2, see Figure 1 and Figure 2In step S1, the facial data collection is used to collect and verify the identity document information and facial images of the visitor and the visited person. Specifically, the facial feature template data of the visited person is obtained by collecting the identity information of the visited person, and the facial features of the visitor are extracted in real time during the face collection of the visitor to obtain the facial template data of the visitor. The facial feature template data of the visited person and the facial template data of the visitor are fused to obtain the facial feature template data for two-way identity authentication.

[0062] Preferably, the identity information of the person being visited is collected, specifically by binding the unique identity number of the person being visited with multi-angle facial image data, and collecting facial images of the front face, left side, right side and expression changes (such as opening the mouth, blinking) of the person being visited through the identity collection terminal, and obtaining optimized facial image data through image clarity evaluation, lighting balance verification and facial integrity detection, and extracting key feature vectors of the face of the person being visited and constructing facial feature template data of the person being visited based on the optimized facial image data, and writing the facial feature template data and identity information of the person being visited into the database for use as long-term static identity authentication benchmark data;

[0063] Preferably, the visitor's face is collected in real time. Specifically, after the visitor arrives at the registration area, the visitor's identity information is collected through the registration client, and the visitor's frontal face image data is simultaneously collected; based on the frontal face image data, the facial feature code is extracted, and the visitor's face template data is constructed; if the image quality of the visitor's frontal face image data does not meet the image quality setting standard, a prompt is given to retake the image, and a maximum of three collection retries are allowed; after the quality conditions are met, the visitor's face template data is stored in a temporary cache database and bound to the current meeting record, and is only valid during this meeting process;

[0064] Preferably, Table 1 is an example table of parameter settings for real-time collection of visitor faces. As shown in the table, the image quality setting standards specifically include clarity, illumination balance, posture angle, occlusion detection, and retry limit number;

[0065] The clarity is specifically evaluated by using a Tenengrad value based on an image gradient clarity evaluation algorithm. The illumination balance is specifically evaluated by calculating the grayscale mean of the image pixels to evaluate the overall brightness level. The posture angles specifically include the left and right head turning angle yaw, the up and down pitch angle pitch, and the head roll angle roll. The occlusion detection is specifically performed using the landmark recognition model in OpenCV to detect key facial areas such as the eyes, nose, and mouth. The retry limit is used to set the maximum number of retries.

[0066] Table 1 Parameter setting example of real-time collection of visitor face

[0067]

[0068] The data fusion, in particular, the face feature template data of the visited person and the face template data of the visitor are respectively processed by standardization and then structure coding is performed, identity matching pair data objects are constructed, and continuous frame face feature templates, collection time stamps, collection device numbers, image quality scores, and identity association identifiers are respectively retained to obtain the bidirectional identity authentication face feature template data.

[0069] Embodiment three, refer to Figure 1 , Figure 2 and Figure 3 , this embodiment is based on the above-mentioned embodiments, in step S2, the bidirectional identity authentication is used to simultaneously perform local and remote face image comparison at the visitor end and the visited end, in particular, according to the bidirectional identity authentication face feature template data, a bidirectional identity authentication method improved by dynamic consistency score is adopted to perform bidirectional identity authentication, and bidirectional identity consistency authentication data is obtained, including the following steps:

[0070] Step S21: dynamic consistency score calculation, in particular, according to the bidirectional identity authentication face feature template data, an improved multi-dimensional similarity information calculation function is constructed to perform dynamic consistency score calculation, and face feature dynamic consistency score parameters are obtained;

[0071] The improved multi-dimensional similarity information calculation function, in particular, performs dynamic consistency score calculation by calculating feature space similarity, collection interval time decay similarity, and definition compensation similarity, and the calculation formula is:

[0072] S dynamic = α·S cos + β·S temporal + γ·S cross ;

[0073] In the formula, S dynamic is the face feature dynamic consistency score parameter, α is the feature similarity weight, S cos is the feature space similarity parameter, β is the collection interval time decay similarity weight, S temporal is the collection interval time decay similarity parameter, γ is the definition compensation similarity weight, and S cross is the definition compensation similarity parameter.

[0074] Preferably, the default value of the feature similarity weight α is 0.6, the default value of the collection interval time decay similarity weight β is 0.2, and the default value of the definition compensation similarity weight γ is 0.2.

[0075] The feature space similarity parameter is specifically calculated based on the feature cosine similarity of the visitor's face template data and the visited person's face feature template data in the two-way identity authentication face feature template data;

[0076] The acquisition interval time decay similarity parameter is specifically calculated by calculating the acquisition timestamp in the two-way identity authentication facial feature template data to perform time decay rate calculation;

[0077] The clarity compensation similarity parameter is specifically calculated by improving the clarity value similarity calculation formula, which is:

[0078]

[0079] Where, is the clarity Tenengrad value in the visitor's face template data, is the clarity Tenengrad value of the face feature template data of the person being visited;

[0080] Step S22: selecting a stable face template, specifically performing transformation gradient calculations at different posture angles for each face feature template in the two-way identity authentication face feature template data to obtain a posture change sensitivity parameter, and using the posture change sensitivity parameter as a stability score to rank the feature stability of the two-way identity authentication face feature template data, and selecting the face feature template data with the highest feature stability as the stable face template data to obtain a stable face feature template;

[0081] The calculation formula of the posture change sensitivity parameter is:

[0082]

[0083] Where R(f) is the posture change sensitivity parameter calculation function, f is the face feature template data for two-way identity authentication, K is the total number of face feature templates, k is the face feature template index, is the posture angle direction of the k-th facial feature template;

[0084] Step S23: Dynamic credibility fusion scoring, specifically performing dynamic credibility comprehensive scoring fusion calculation based on the facial feature dynamic consistency scoring parameter, the stable facial feature template, and the image quality score in the two-way identity authentication facial feature template data to obtain a facial template credibility comprehensive scoring parameter;

[0085] The calculation formula for the dynamic credibility comprehensive score fusion calculation is:

[0086] S trust=σ(w1·S dynamic +w2·C(f v , f d )+w3·Q(q));

[0087] Where S trust is the comprehensive scoring parameter of the face template credibility, σ is the Sigmoid activation function, w1 is the dynamic consistency weight, S dynamic is the facial feature dynamic consistency scoring parameter, w2 is the stable face template weight, C(f v , f d ) is a stable facial feature template, w3 is the image quality weight, Q(q) is the normalized image quality score data, specifically calculated by the hyperbolic tangent function, and q is the image quality score in the facial feature template data of the two-way identity authentication;

[0088] Step S24: constructing an optimal template, specifically by concatenating and fusing the features of the visitor and the person being visited on the stable facial feature template, and constructing a lightweight distillation network to compress the feature dimensions based on the concatenated and fused stable facial feature template data to obtain bidirectional identity authentication template data;

[0089] The two-way identity authentication template data includes compression features, sampling timestamp, face template credibility comprehensive scoring parameter, posture change sensitivity parameter and image quality score, and the calculation formula is:

[0090] M= <f optimal ,t,S trust , R(f),q>

[0091] Where M is the two-way authentication template data, f optimal is the compressed feature, t is the sampling timestamp;

[0092] Preferably, Table 2 is an example table of model parameters of the lightweight distillation network. As shown in the table, the lightweight distillation network specifically adopts a two-layer fully connected network and introduces a GELU activation function for activation;

[0093] Table 2 Example of model parameters of lightweight distillation network

[0094]

[0095] Step S25: Bidirectional identity authentication, specifically performing bidirectional identity authentication based on the face template credibility comprehensive scoring parameter in the bidirectional identity authentication template data, and performing score grading processing based on the specific scores of the credibility comprehensive scoring parameter to obtain bidirectional identity consistency authentication data;

[0096] The two-way identity consistency authentication data specifically includes two-way identity authentication template data and two-way identity consistency authentication results;

[0097] The score grading process specifically sets the two-way identity consistency authentication result with the face template credibility comprehensive score parameter greater than or equal to 0.8 as passed verification, and sets the two-way identity consistency authentication result with the score parameter less than 0.8 as failed verification.

[0098] By performing the above operations, in order to solve the technical problems of large sample image quality differences, diverse posture changes, and unstable authentication results that often occur in existing meeting person identity authentication systems in actual scenarios, the present invention introduces a dynamic consistency scoring mechanism + posture change sensitivity ranking + lightweight distillation network fusion optimization authentication process, which significantly improves the identity matching accuracy and credibility under complex acquisition conditions; specifically, traditional face recognition methods are often based on feature comparison based on a single image or a real-time frame image, which is easily affected by the actual acquisition environment. For example, situations such as backlighting in the visited area and the person being visited lowering his head will lead to increased feature errors, resulting in authentication results. The result deviates; although some systems support multi-frame acquisition, they do not dynamically model the image quality and time consistency, and lack stability screening; the present invention constructs a dynamic scoring function based on three factors: feature space similarity, acquisition interval attenuation coefficient, and image clarity compensation, and selects the most stable frame in the template as the final authentication feature based on the sensitivity to posture changes. It is compressed by a double-layer distillation network activated by GELU and stored as a standard authentication template; this mechanism can not only provide hierarchical scoring and improve fault tolerance in the authentication stage, but also serve as a high-quality reference in the monitoring stage, making up for the practical problems of "unstable sampling and easy misjudgment of authentication" in the existing scheme.

[0099] Example 4, see Figure 1 、 Figure 2 This embodiment is based on the above embodiment. In step S3, the meeting order management is used to manage the meeting queue sequence between the visited person and the visiting person. Specifically, based on the meeting data verified in the two-way identity consistency authentication data, the meeting order queue priority judgment, meeting room resource matching scheduling and call notification are performed in sequence to perform meeting order management, obtain the meeting queue scheduling result, and perform meeting call notification based on the meeting queue scheduling result to meet the visited person.

[0100] The priority of the meeting order queue is specifically to set the lawyer visit as the highest priority, and to queue the meeting order queue according to the appointment time;

[0101] The call notification specifically pushes the call information and performs voice broadcasting through the MQTT Internet of Things protocol.

[0102] Example 5, seeFigure 1 、 Figure 2 and Figure 4 This embodiment is based on the above embodiment. In step S4, the real-time two-way monitoring is used to continuously perform dynamic facial recognition monitoring on both parties during the meeting. Specifically, based on the two-way identity consistency authentication data, after the two parties enter the meeting room, real-time facial image data is collected using the local camera in the meeting room. The real-time two-way monitoring is performed using a dual-path anomaly detection method based on an improved feature comparison method to obtain two-way identity monitoring status data. The method includes the following steps:

[0103] Step S41: feature comparison optimization, specifically, by using a lightweight image recognition model, extracting features based on the real-time facial image data to obtain real-time facial feature data, and constructing an improved cosine similarity with intervention of a posture change sensitivity parameter to calculate feature comparison similarity between the real-time facial feature data and the bidirectional identity consistency authentication data to obtain a feature comparison similarity parameter, and using the real-time facial image with the highest feature comparison similarity parameter as monitoring data input to obtain the best matching real-time feature data;

[0104] The lightweight image recognition model specifically adopts a compressed version of the MobileFaceNet model with 1M parameters;

[0105] The calculation formula of the improved cosine similarity with the intervention of the posture change sensitivity parameter is:

[0106]

[0107] Where S match is the feature contrast similarity parameter, f real-time It is the real-time facial feature data output by the lightweight image recognition model;

[0108] Step S42: dual-path anomaly monitoring, specifically constructing a lightweight three-dimensional convolutional neural network, performing dual-path anomaly monitoring based on the optimal matching feature data, and obtaining anomaly monitoring score data;

[0109] The dual-path anomaly monitoring specifically includes facial occlusion anomalies and personnel change anomalies;

[0110] The calculation formula of the abnormal monitoring score data is:

[0111] A=0.6P occl +0.4P swap ;

[0112] Where A is the abnormal monitoring score data, P occl is the facial occlusion anomaly detection result, P swap It is the abnormal monitoring result of personnel change;

[0113] Preferably, Table 3 is a parameter example table of the lightweight three-dimensional convolutional neural network. As shown in the table, the lightweight three-dimensional convolutional neural network specifically takes the last five frames of continuous facial image data of the optimal matching real-time feature data as input, sequentially constructs a backbone feature extraction channel through three groups of three-dimensional convolutional layers and pooling layers, obtains inter-frame stable feature distribution through time-dimensional average pooling operation, and realizes dual-path parallel prediction output of abnormal state through a two-layer fully connected structure;

[0114] Table 3 Parameter example of lightweight three-dimensional convolutional neural network

[0115]

[0116] Step S43: Real-time fusion decision making, specifically, performing three-level abnormality monitoring decisions based on the abnormality monitoring score data to obtain abnormal status data, and performing two-way detection on both parties of the meeting to obtain two-way abnormal status data;

[0117] The calculation formula for the three-level abnormality monitoring decision is:

[0118]

[0119] In the formula, Decision is the abnormal state data, Normal is the normal state, Warning is the warning state, and Alert is the abnormal alarm state;

[0120] Step S44: abnormal monitoring feedback, specifically, based on the abnormal state in the two-way abnormal state data, when the abnormal state is a warning state, a prompt sound is sent to the meeting room; when the abnormal state is an abnormal alarm state, the current abnormal frame data is automatically intercepted and stored, and a suggestion to terminate the meeting is sent via MQTT to obtain the two-way identity monitoring status data;

[0121] The two-way identity monitoring status data specifically includes two-way abnormal status data, abnormal frame data and meeting termination instruction sending record data.

[0122] By performing the above operations, in order to solve the technical problem that the existing real-time meeting monitoring system only has basic video acquisition functions and lacks automatic abnormality recognition and response mechanisms, the present invention designs a dual-path three-dimensional convolutional anomaly detection network based on optimal feature matching, thereby realizing real-time detection and a three-level response feedback mechanism for risky behaviors such as people occluding people (such as wearing masks and sunglasses) and putting people in the camera during the meeting; specifically, in traditional systems, supervisors mainly rely on visual judgment or post-recording video playback to identify abnormal behaviors, which are easily affected by factors such as blind spots in monitoring and distraction, resulting in missed reports of abnormal behaviors; even with basic human The facial recognition capability is insufficient, and it cannot cope with behaviors such as occlusion, rapid replacement, or identity switching. The present invention uses the stable feature vector screened out from the two-way authentication template as the core benchmark, extracts continuous frame image sequences in real-time meeting acquisition, and constructs high-order dynamic features of facial behavior changes through three sets of 3D convolution + temporal average pooling. The facial occlusion score and identity change score are then predicted in parallel by a dual-output fully connected structure. Finally, based on the comprehensive score, a three-level control of "prompt sound-warning-meeting interruption" is executed, which significantly improves the system's ability to actively identify and respond to potential violations, and solves the structural pain point of "visible but slow to respond".

[0123] Example 6, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S5, the meeting data is archived to perform structured archiving of the audio and video data and identity verification information during the meeting. Specifically, after the meeting is completed, the multimodal data generated during the meeting is classified and stored, and bound to the corresponding meeting record to obtain a meeting archive data set, including the following steps:

[0124] Step S51: Archiving audio and video data, specifically starting synchronous audio and video recording at the beginning of the meeting, compressing and encoding using a standard encoding protocol, and logically segmenting the data into 5-minute segments to obtain meeting audio archive data and meeting video archive data;

[0125] Step S52: Archiving the identity authentication log, specifically, collecting the current two-way identity consistency authentication data, the real-time identification frame log during the meeting, and the abnormal alarm record, by timestamp and unique meeting number, to obtain the identity authentication log archive data and abnormal identification alarm information;

[0126] Step S53: Generate a structured data index, specifically using the meeting unique number as the primary index to generate a metadata index table to obtain structured data; the structured data specifically includes the meeting unique number, information about the meeting participants, meeting start and end times, meeting video file path, two-way identity consistency authentication data, and monitoring anomaly data;

[0127] Step S54: Archiving data storage and backup, specifically, synchronously writing the structured data into the local database and the central archiving server to obtain the meeting archive data set;

[0128] The meeting archive data set specifically includes meeting audio archive data, meeting video archive data, identity authentication log archive data and abnormality identification alarm information.

[0129] Example 7, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. The present invention provides a face recognition-based meeting management system, which includes a face data acquisition module, a two-way identity authentication module, a meeting sequence management module, a real-time two-way monitoring module, and a meeting data archiving module.

[0130] The face data collection module is used to collect face data, obtain face feature template data for two-way identity authentication through face data collection, and send the face feature template data for two-way identity authentication to the two-way identity authentication module and the meeting order management module;

[0131] The two-way identity authentication module is used for two-way identity authentication, obtains two-way identity consistency authentication data through two-way identity authentication, and sends the two-way identity consistency authentication data to the meeting order management module and the real-time two-way monitoring module;

[0132] The meeting order management module is used for meeting order management, through which meeting order management, meeting queue scheduling results are obtained and meetings with the visited persons are conducted;

[0133] The real-time two-way monitoring module is used for real-time two-way monitoring, obtains two-way identity monitoring status data through real-time two-way monitoring, and sends the two-way identity monitoring status data to the meeting data archiving module;

[0134] The meeting data archiving module is used for archiving meeting data, and obtains a meeting archive data set by archiving the meeting data.

[0135] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0136] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary and that changes can be made in the embodiments without departing from the spirit and scope of the application.

[0137] The above description of the application and its embodiments is not restrictive, and the embodiments shown in the drawings are only one of the embodiments of the application, and the actual structure is not limited thereto. In general, if a person skilled in the art is inspired by it, without departing from the purpose of the application, without creative design, similar structure and embodiments of the technical solution should belong to the protection scope of the application.

Claims

1. A meeting management method based on face recognition, characterized by: The method comprises the following steps: Step S1: facial data collection, used to collect and verify the identity document information and facial images of the visitor and the person being visited, and obtain facial feature template data for two-way identity authentication; Step S2: Bidirectional identity authentication, based on the bidirectional identity authentication facial feature template data, adopts a bidirectional identity authentication method improved by dynamic consistency scoring to perform bidirectional identity authentication and obtain bidirectional identity consistency authentication data, including the following steps: Step S21: Dynamic consistency score calculation, constructing an improved multi-dimensional similarity information calculation function, and performing dynamic consistency score calculation; the improved multi-dimensional similarity information calculation function specifically calculates feature space similarity, acquisition interval time attenuation similarity and clarity compensation similarity to perform dynamic consistency score calculation; Step S22: Stable face template selection; Step S23: Dynamic credibility fusion scoring; Step S24: Optimal template construction; Step S25: Bidirectional identity authentication; Step S3: Meeting order management: Based on the meeting data verified in the two-way identity consistency authentication data, meeting order queue priority determination, meeting room resource matching scheduling and call notification are performed in sequence to perform meeting order management and obtain the meeting queue scheduling result; Step S4: Real-time two-way monitoring. Based on the two-way identity consistency authentication data, after the two parties enter the meeting room, real-time facial image data is collected based on the local camera in the meeting room. Real-time two-way monitoring is performed using an improved feature comparison dual-path anomaly detection method to obtain two-way identity monitoring status data. The method includes the following steps: Step S41: Feature comparison optimization; Step S42: Dual-path anomaly monitoring; Step S43: Real-time fusion decision-making; Step S44: Anomaly monitoring feedback. Step S5: Archiving the meeting data. After the meeting is completed, the multimodal data generated during the meeting are classified and stored, and bound to the corresponding meeting records to obtain a meeting archive data set.

2. The face recognition-based meeting management method according to claim 1, characterized in that: In step S1, the facial data collection is used to collect and verify the identity document information and facial images of the visitor and the visited person. Specifically, the facial feature template data of the visited person is obtained by collecting the identity information of the visited person, and the facial features of the visitor are extracted in real time during the face collection of the visitor to obtain the facial template data of the visitor. The facial feature template data of the visited person and the facial template data of the visitor are fused to obtain the facial feature template data for two-way identity authentication. The data fusion is specifically to perform structural encoding on the facial feature template data of the person being visited and the facial template data of the visiting person after standardization, to construct an identity matching pair data object, and to retain the continuous frame facial feature templates, acquisition timestamps, acquisition device numbers, image quality scores and identity association identifiers, to obtain two-way identity authentication facial feature template data.

3. The face recognition-based meeting management method according to claim 2, characterized in that: In step S2, the two-way identity authentication is used to perform local and remote facial image comparison at the visiting end and the visited end at the same time. Specifically, based on the two-way identity authentication facial feature template data, a two-way identity authentication method improved by dynamic consistency scoring is used to perform two-way identity authentication to obtain two-way identity consistency authentication data, including the following steps: Step S21: dynamic consistency score calculation, specifically, constructing an improved multi-dimensional similarity information calculation function based on the two-way identity authentication facial feature template data, performing dynamic consistency score calculation, and obtaining facial feature dynamic consistency score parameters; Step S22: selecting a stable face template, specifically performing transformation gradient calculations at different posture angles for each face feature template in the two-way identity authentication face feature template data to obtain a posture change sensitivity parameter, and using the posture change sensitivity parameter as a stability score to rank the feature stability of the two-way identity authentication face feature template data, and selecting the face feature template data with the highest feature stability as the stable face template data to obtain a stable face feature template; Step S23: Dynamic credibility fusion scoring, specifically performing dynamic credibility comprehensive scoring fusion calculation based on the facial feature dynamic consistency scoring parameter, the stable facial feature template, and the image quality score in the two-way identity authentication facial feature template data to obtain a facial template credibility comprehensive scoring parameter; Step S24: constructing an optimal template, specifically by concatenating and fusing the features of the visitor and the person being visited on the stable facial feature template, and constructing a lightweight distillation network to compress the feature dimensions based on the concatenated and fused stable facial feature template data to obtain bidirectional identity authentication template data; Step S25: Bidirectional identity authentication, specifically performing bidirectional identity authentication based on the face template credibility comprehensive scoring parameter in the bidirectional identity authentication template data, and performing score grading processing based on the specific scores of the credibility comprehensive scoring parameter to obtain bidirectional identity consistency authentication data; The two-way identity consistency authentication data specifically includes two-way identity authentication template data and two-way identity consistency authentication results; The score grading process specifically sets the two-way identity consistency authentication result with the face template credibility comprehensive score parameter greater than or equal to 0.8 as passed verification, and sets the two-way identity consistency authentication result with the score parameter less than 0.8 as failed verification.

4. The face recognition-based meeting management method according to claim 3, characterized in that: In step S3, the meeting order management is used to manage the meeting queue sequence between the visited person and the visitor. Specifically, based on the meeting data verified in the two-way identity consistency authentication data, the meeting order queue priority judgment, meeting room resource matching scheduling and call notification are performed in sequence to perform meeting order management, obtain the meeting queue scheduling result, and perform meeting call notification based on the meeting queue scheduling result to meet the visited person; The priority of the meeting order queue is specifically to set the lawyer visit as the highest priority, and to queue the meeting order queue according to the appointment time; The call notification specifically pushes the call information and performs voice broadcasting through the MQTT Internet of Things protocol.

5. The face recognition-based meeting management method according to claim 4, characterized in that: In step S4, the real-time two-way monitoring is used to continuously perform dynamic facial recognition monitoring on both parties during the meeting. Specifically, based on the two-way identity consistency authentication data, after the two parties enter the meeting room, real-time facial image data is collected using the local camera in the meeting room, and real-time two-way monitoring is performed using a dual-path anomaly detection method with improved feature comparison to obtain two-way identity monitoring status data, including the following steps: Step S41: feature comparison optimization, specifically, by using a lightweight image recognition model, extracting features based on the real-time facial image data to obtain real-time facial feature data, and constructing an improved cosine similarity with intervention of a posture change sensitivity parameter to calculate feature comparison similarity between the real-time facial feature data and the bidirectional identity consistency authentication data to obtain a feature comparison similarity parameter, and using the real-time facial image with the highest feature comparison similarity parameter as monitoring data input to obtain the best matching real-time feature data; Step S42: dual-path anomaly monitoring, specifically constructing a lightweight three-dimensional convolutional neural network, performing dual-path anomaly monitoring based on the optimal matching feature data, and obtaining anomaly monitoring score data; The dual-path anomaly monitoring specifically includes facial occlusion anomalies and personnel change anomalies; Step S43: Real-time fusion decision making, specifically, performing three-level abnormality monitoring decisions based on the abnormality monitoring score data to obtain abnormal status data, and performing two-way detection on both parties of the meeting to obtain two-way abnormal status data; Step S44: abnormal monitoring feedback, specifically, based on the abnormal state in the two-way abnormal state data, when the abnormal state is a warning state, a prompt sound is sent to the meeting room; when the abnormal state is an abnormal alarm state, the current abnormal frame data is automatically intercepted and stored, and a suggestion to terminate the meeting is sent via MQTT to obtain the two-way identity monitoring status data; The two-way identity monitoring status data specifically includes two-way abnormal status data, abnormal frame data and meeting termination instruction sending record data.

6. The face recognition-based meeting management method according to claim 5, characterized in that: In step S5, the meeting data is archived to perform structured archiving of the audio and video data and identity verification information during the meeting. Specifically, after the meeting is completed, the multimodal data generated during the meeting is classified and stored, and bound to the corresponding meeting record to obtain a meeting archive data set.

7. The face recognition-based meeting management method according to claim 6, characterized in that: The meeting data archiving includes the following steps: Step S51: Archiving audio and video data, specifically starting synchronous audio and video recording at the beginning of the meeting, compressing and encoding using a standard encoding protocol, and logically segmenting the data into 5-minute segments to obtain meeting audio archive data and meeting video archive data; Step S52: Archiving the identity authentication log, specifically, collecting the current two-way identity consistency authentication data, the real-time identification frame log during the meeting, and the abnormal alarm record, by timestamp and unique meeting number, to obtain the identity authentication log archive data and abnormal identification alarm information; Step S53: Generate a structured data index, specifically using the meeting unique number as the primary index to generate a metadata index table to obtain structured data; the structured data specifically includes the meeting unique number, information about the meeting participants, meeting start and end times, meeting video file path, two-way identity consistency authentication data, and monitoring anomaly data; Step S54: Archiving data storage and backup, specifically, synchronously writing the structured data into the local database and the central archiving server to obtain the meeting archive data set; The meeting archive data set specifically includes meeting audio archive data, meeting video archive data, identity authentication log archive data and abnormality identification alarm information.

8. A face recognition-based meeting management system, for implementing a face recognition-based meeting management method as claimed in any one of claims 1 to 7, characterized in that: It includes face data collection module, two-way identity authentication module, meeting sequence management module, real-time two-way monitoring module and meeting data archiving module.

9. The face recognition-based meeting management system according to claim 8, characterized in that: The face data collection module is used to collect face data, obtain face feature template data for two-way identity authentication through face data collection, and send the face feature template data for two-way identity authentication to the two-way identity authentication module and the meeting order management module; The two-way identity authentication module is used for two-way identity authentication, obtains two-way identity consistency authentication data through two-way identity authentication, and sends the two-way identity consistency authentication data to the meeting order management module and the real-time two-way monitoring module; The meeting order management module is used for meeting order management, through which meeting order management, meeting queue scheduling results are obtained and meetings with the visited persons are conducted; The real-time two-way monitoring module is used for real-time two-way monitoring, obtains two-way identity monitoring status data through real-time two-way monitoring, and sends the two-way identity monitoring status data to the meeting data archiving module; The meeting data archiving module is used for archiving meeting data, and obtains a meeting archive data set by archiving the meeting data.

Images