A meeting management method and system based on face recognition

Abstract

The application discloses a meeting management method and system based on face recognition, which comprises face data collection, two-way identity authentication, meeting order management, real-time two-way monitoring and meeting data archiving. The application relates to the technical field of intelligent management of agricultural machinery, systematically introduces the mechanism of "two-way identity authentication + real-time two-way monitoring", realizes the closed-loop meeting management process from pre-meeting identity verification, meeting abnormality identification to post-meeting data archiving, constructs a high-credibility face template through dynamic consistency scoring, posture sensitivity sorting and lightweight distillation network compression, and improves the stability and accuracy of identity authentication. Meanwhile, a three-dimensional convolutional abnormality detection network based on optimal feature matching is designed to identify abnormal behaviors such as face covering and personnel replacement in the meeting process in real time and trigger a three-level response mechanism. The method significantly enhances the safety, controllability and intelligent level of the meeting of prison personnel.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face recognition video monitoring, in particular to a meeting management method and system based on face recognition. BACKGROUND

[0002] The meeting management method and system based on face recognition is a technical solution for realizing automatic identity verification and intelligent process management of meeting parties by collecting, comparing and verifying face biometric information. Its core function is to upgrade the traditional manual meeting process to efficient, accurate and traceable digital management by using computer vision and pattern recognition technology. Specifically, it ensures "identity consistency" through real-time face comparison to prevent identity fraud, maintains the authenticity of the identity during the meeting process through dynamic monitoring to prevent mid-way substitution, and forms a complete meeting electronic evidence chain through data archiving. This system significantly improves the meeting management efficiency in security scenarios, reduces the cost of manual intervention while ensuring security, and is a typical application of biometric technology in the field of identity authentication.

[0003] However, the existing meeting management method for the visited personnel has the technical problems of static identity verification process, one-way verification object and inability to dynamically identify behavior abnormalities during the monitoring process. The existing meeting personnel identity authentication system often has the technical problems of large differences in image quality, diversified posture changes and unstable authentication results in actual scenarios. The existing real-time meeting monitoring system only has basic video collection function and lacks automatic abnormality identification and response mechanism. SUMMARY

[0004] The technical solution adopted by the present application is as follows: The present application provides a meeting management method based on face recognition, which comprises the following steps:

[0005] Step S1: face data collection;

[0006] Step S2: two-way identity authentication;

[0007] Step S3: meeting order management;

[0008] Step S4: real-time two-way monitoring;

[0009] Step S5: meeting data archiving.

[0010] Further, in step S1, the face data is collected to collect and verify the identity information and face image of the visitor and the visited person. Specifically, the face feature template data of the visited person is obtained through the collection of the identity information of the visited person, and the face feature of the visitor is extracted through real-time collection of the face of the visitor to obtain the face template data of the visitor. The face feature template data of the visited person and the face template data of the visitor are fused to obtain the face feature template data of the two-way identity authentication.

[0011] The data fusion is specifically the structural coding of the face feature template data of the visited person and the face template data of the visitor after standardization processing, the construction of identity matching data objects, and the reservation of continuous frame face feature template, collection time stamp, collection device number, image quality score and identity association identifier to obtain the face feature template data of the two-way identity authentication.

[0012] Further, in step S2, the two-way identity authentication is performed to simultaneously perform local and remote face image comparison at the visitor end and the visited end. Specifically, the two-way identity authentication method improved by dynamic consistency score is used to perform two-way identity authentication according to the face feature template data of the two-way identity authentication to obtain two-way identity consistency authentication data, including the following steps:

[0013] Step S21: Dynamic consistency score calculation, specifically, the improved multi-dimensional similarity information calculation function is constructed according to the face feature template data of the two-way identity authentication to perform dynamic consistency score calculation to obtain the face feature dynamic consistency score parameter.

[0014] The improved multi-dimensional similarity information calculation function specifically calculates the feature space similarity, collection interval time decay similarity and definition compensation similarity to perform dynamic consistency score calculation.

[0015] Step S22: Stable face template selection, specifically, the transformation gradient calculation under different posture angles is performed on each face feature template in the face feature template data of the two-way identity authentication to obtain the posture change sensitivity parameter, and the posture change sensitivity parameter is used as the stability score to rank the feature stability of the face feature template data of the two-way identity authentication, and the face feature template data with the highest feature stability is selected as the stable face template data to obtain the stable face feature template.

[0016] Step S23: Dynamic credibility fusion score, specifically, the dynamic credibility comprehensive score fusion calculation is performed according to the face feature dynamic consistency score parameter, the stable face feature template and the image quality score in the face feature template data of the two-way identity authentication to obtain the face template credibility comprehensive score parameter.

[0017] Step S24: optimal template construction, specifically by splicing and fusing the visitor and the visited person features to the stable face feature template, and constructing a light distillation network to compress the feature dimension according to the spliced and fused stable face feature template data, to obtain the bidirectional identity authentication template data;

[0018] Step S25: bidirectional identity authentication, specifically performing bidirectional identity authentication according to the face template credibility comprehensive score parameter in the bidirectional identity authentication template data, and performing score classification processing according to the specific score of the credibility comprehensive score parameter, to obtain bidirectional identity consistency authentication data;

[0019] The bidirectional identity consistency authentication data specifically includes bidirectional identity authentication template data and bidirectional identity consistency authentication results;

[0020] The score classification processing specifically sets the bidirectional identity consistency authentication result with the face template credibility comprehensive score parameter greater than or equal to 0.8 as passing the verification, and sets the bidirectional identity consistency authentication result less than 0.8 as failing the verification.

[0021] Further, in step S3, the meeting order management is used to manage the meeting queuing order of the visited person and the visitor, specifically by sequentially performing meeting order queue priority judgment, meeting room resource matching scheduling and call number notification according to the meeting data passing the verification in the bidirectional identity consistency authentication data, to perform meeting order management, obtain meeting queuing scheduling results, and perform meeting call number notification according to the meeting queuing scheduling results, to perform the meeting of the visited person;

[0022] The meeting order queue priority judgment specifically sets the lawyer visit as the highest priority, and sequentially queues the meeting order queue according to the reservation time;

[0023] The call number notification specifically pushes the call number information and performs voice broadcast through the MQTT Internet of Things protocol.

[0024] Further, in step S4, the real-time bidirectional monitoring is used to continuously monitor the dynamic face recognition of both parties during the meeting, specifically by collecting real-time face image data according to the local camera of the meeting room after the meeting parties enter the meeting room according to the bidirectional identity consistency authentication data, and performing real-time bidirectional monitoring through the improved feature comparison double-path anomaly detection method, to obtain bidirectional identity monitoring state data, including the following steps:

[0025] Step S41: feature comparison optimization, specifically, by using a lightweight image recognition model, feature extraction is performed based on the real-time face image data to obtain real-time face feature data, and an improved cosine similarity with posture change sensitivity parameter intervention is constructed to perform feature comparison similarity calculation on the real-time face feature data and the bidirectional identity consistency authentication data, to obtain a feature comparison similarity parameter, and the real-time face image with the highest feature comparison similarity parameter is input as monitoring data to obtain optimal matching degree real-time feature data;

[0026] Step S42: dual-path anomaly monitoring, specifically, a lightweight three-dimensional convolutional neural network is constructed, and based on the optimal matching degree feature data, dual-path anomaly monitoring is performed to obtain anomaly monitoring score data;

[0027] The dual-path anomaly monitoring specifically includes face occlusion anomaly and personnel change anomaly.

[0028] Step S43: real-time fusion decision, specifically, based on the anomaly monitoring score data, three-level anomaly monitoring decision is performed to obtain anomaly state data, and bidirectional detection is performed on the meeting parties to obtain bidirectional anomaly state data.

[0029] Step S44: anomaly monitoring feedback, specifically, based on the anomaly state in the bidirectional anomaly state data, when the anomaly state is a warning state, a prompt sound is sent to the meeting room, and when the anomaly state is an anomaly alarm state, the current anomaly frame data is automatically intercepted for storage, and a suggestion to terminate the meeting instruction is sent through MQTT, to obtain bidirectional identity monitoring state data.

[0030] The bidirectional identity monitoring state data specifically includes bidirectional anomaly state data, anomaly frame data, and suggestion to terminate the meeting instruction sending record data.

[0031] Further, in step S5, the meeting data is archived, which is used to structure the audio and video data in the meeting process and the identity verification information, specifically, after the meeting is completed, the multi-modal data generated in the meeting process is classified and stored, and is bound with the corresponding meeting record to obtain a meeting archive data set, including the following steps:

[0032] Step S51: audio and video data archiving, specifically, audio and video synchronization recording is started at the beginning of the meeting, compressed and encoded using a standard encoding protocol, and logically segmented every 5 minutes to obtain meeting audio archive data and meeting video archive data.

[0033] Step S52: identity authentication log archiving, specifically, the two-way identity consistency authentication data in front of the meeting, the real-time identification frame log in the meeting process and the abnormal alarm record are uniformly collected according to the time stamp and the meeting unique number, identity authentication log archiving data and abnormal identification alarm information are obtained;

[0034] Step S53: structured data index generation, specifically, the meeting unique number is taken as the main index, the metadata index table is generated, and the structured data is obtained; the structured data specifically includes the meeting unique number, the meeting personnel information, the meeting start and end time, the meeting video segment file path, the two-way identity consistency authentication data and the monitoring abnormal data;

[0035] Step S54: archiving data storage backup, specifically, the structured data is synchronously written into the local database and the center archiving server, and the meeting archiving data set is obtained;

[0036] The meeting archiving data set specifically includes meeting audio archiving data, meeting video archiving data, identity authentication log archiving data and abnormal identification alarm information.

[0037] The meeting management system based on face recognition provided by the application includes a face data acquisition module, a two-way identity authentication module, a meeting order management module, a real-time two-way monitoring module and a meeting data archiving module.

[0038] The face data acquisition module is used for face data acquisition, and through face data acquisition, two-way identity authentication face feature template data is obtained, and the two-way identity authentication face feature template data is sent to the two-way identity authentication module and the meeting order management module.

[0039] The two-way identity authentication module is used for two-way identity authentication, and through two-way identity authentication, two-way identity consistency authentication data is obtained, and the two-way identity consistency authentication data is sent to the meeting order management module and the real-time two-way monitoring module.

[0040] The meeting order management module is used for meeting order management, and through meeting order management, a meeting queuing scheduling result is obtained, and a visited person is met.

[0041] The real-time two-way monitoring module is used for real-time two-way monitoring, and through real-time two-way monitoring, two-way identity monitoring state data is obtained, and the two-way identity monitoring state data is sent to the meeting data archiving module.

[0042] The meeting data archiving module is used for meeting data archiving, and through meeting data archiving, a meeting archiving data set is obtained.

[0043] The application has the following beneficial effects by adopting the above scheme:

[0044] (1) In view of the technical problems that the identity verification process is static, the verification object is unidirectional, and the monitoring process cannot dynamically identify behavior abnormalities in the existing visited personnel meeting management method, the application systematically integrates the "two-way identity authentication + real-time two-way monitoring" mechanism, performs two-path verification on the identity of the visiting end and the visited end before the meeting, and continuously performs two-way face feature matching and abnormal state monitoring during the meeting process, thereby constructing a closed-loop full-process dynamic identity management architecture;

[0045] (2) In view of the technical problems that the existing meeting personnel identity authentication system often has large differences in the quality of sampled images, diversified posture changes, and unstable authentication results in actual scenes, the application introduces a dynamic consistency scoring mechanism + posture change sensitivity sorting + lightweight distillation network fusion optimization authentication process, which significantly improves the identity matching accuracy and reliability under complex collection conditions;

[0046] (3) In view of the technical problem that the existing real-time meeting monitoring system only has basic video collection function and lacks automatic abnormality identification and response mechanism, the application designs a two-path three-dimensional convolution anomaly detection network based on optimal feature matching, realizes real-time detection and three-level response feedback mechanism for risk behaviors such as personnel occlusion (such as wearing masks and sunglasses) and replacing people in the meeting process. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A flowchart of a meeting management method based on face recognition provided by the application;

[0048] Figure 2 A schematic diagram of a meeting management system based on face recognition provided by the application;

[0049] Figure 3 A flowchart of step S2 two-way identity authentication;

[0050] Figure 4 A flowchart of step S4 real-time two-way monitoring.

[0051] The accompanying drawings are used to provide a further understanding of the application, and constitute a part of the specification, together with the embodiments of the application, to explain the application, and do not constitute a limitation on the application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the application.

[0053] In the description of the present application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0054] Embodiment one, refer to Figure 1 The present application provides a meeting management method based on face recognition, which comprises the following steps:

[0055] Step S1: face data acquisition;

[0056] Step S2: two-way identity authentication;

[0057] Step S3: meeting order management;

[0058] Step S4: real-time two-way monitoring;

[0059] Step S5: meeting data archiving.

[0060] By performing the above operation, in view of the technical problems of static identity verification process, one-way verification object and inability to dynamically identify behavior abnormalities in the monitoring process in the existing meeting management method of the visited personnel, the present application systematically integrates the "two-way identity authentication + real-time two-way monitoring" mechanism, verifies the identity of the visiting end and the visited end in two paths before the meeting, and continuously performs two-way face feature matching and abnormal state monitoring during the meeting process, and constructs a closed-loop dynamic identity management architecture throughout the process. Specifically, the traditional system usually only collects face data at the visiting end and compares it with the database, and the visited personnel only rely on the dispatch list or identity card of the management desk, without actual identity verification process, which is easy to appear as a substitute, registration error and other vulnerabilities. Even if equipped with video monitoring, it is impossible to determine whether the current image is the authenticated person, and the monitoring is only used as a post-examination means, lacking active identification ability. The present application simultaneously constructs a face template on the visited end and the visiting end and binds the authentication record, continuously compares with the template during the meeting process, improves the matching stability by combining the posture intervention cosine similarity, realizes the "pre-meeting confirmation, meeting supervision, post-meeting archiving" integrated identity verification and behavior supervision closed loop, and breaks through the structural limitation of the existing static verification.

[0061] Embodiment two, refer to Figure 1 and Figure 2In step S1, the face data is collected, which is used to collect and verify the identity information and face image of the visitor and the visited person. Specifically, the face feature template data of the visited person is obtained through the collection of the identity information of the visited person, and the face feature of the visitor is extracted through real-time collection of the face of the visitor to obtain the face template data of the visitor. The face feature template data of the visited person and the face template data of the visitor are fused to obtain the bidirectional identity authentication face feature template data.

[0062] Preferably, the identity information of the visited person is collected, and specifically, the unique identity number of the visited person is bound with the multi-angle face image data. The face image of the visited person in front, left side, right side and expression change (such as opening mouth and blinking) is collected through the identity collection terminal, and the optimized face image data is obtained through image definition evaluation, light balance verification and face integrity detection. The face key feature vector of the visited person is extracted according to the optimized face image data, and the face feature template data of the visited person is constructed. The face feature template data of the visited person and the identity information are written into the database as long-term static identity authentication reference data.

[0063] Preferably, the real-time collection of the face of the visitor includes the following steps. After the visitor arrives at the registration area, the identity information of the visitor is collected through the registration client, and the front face image data of the visitor is collected synchronously. The face feature code is extracted according to the front face image data, and the face template data of the visitor is constructed. If the image quality of the front face image data of the visitor does not meet the image quality setting standard, it is prompted to re-shoot, and a maximum of three collection retries are allowed. After meeting the quality condition, the face template data of the visitor is stored in the temporary cache database and is bound with the current meeting record, which is only valid in this meeting process.

[0064] Preferably, Table 1 is a parameter setting example table for real-time collection of the face of the visitor. The image quality setting standard specifically includes definition, light balance, posture angle, shielding detection and retry limit number.

[0065] The definition is specifically evaluated by Tenengrad value evaluation algorithm based on image gradient. The light balance is specifically evaluated by calculating the gray mean value of image pixels to evaluate the overall brightness level. The posture angle specifically includes left and right turning angle yaw, up and down tilting angle pitch and head lateral tilting angle roll. The shielding detection is specifically detected by the landmark recognition model in OpenCV to detect the face key area of eyes, nose and mouth. The retry limit is used to set the maximum retry number.

[0066] Table 1 Parameter setting example of real-time collection of visitor face

[0067]

[0068] The data fusion, in particular, the face feature template data of the visited person and the face template data of the visitor are respectively processed by standardization and then structure coding is performed, identity matching pair data objects are constructed, and continuous frame face feature templates, collection time stamps, collection device numbers, image quality scores, and identity association identifiers are respectively retained to obtain the bidirectional identity authentication face feature template data.

[0069] Embodiment three, refer to Figure 1 , Figure 2 and Figure 3 , this embodiment is based on the above-mentioned embodiments, in step S2, the bidirectional identity authentication is used to simultaneously perform local and remote face image comparison at the visitor end and the visited end, in particular, according to the bidirectional identity authentication face feature template data, a bidirectional identity authentication method improved by dynamic consistency score is adopted to perform bidirectional identity authentication, and bidirectional identity consistency authentication data is obtained, including the following steps:

[0070] Step S21: dynamic consistency score calculation, in particular, according to the bidirectional identity authentication face feature template data, an improved multi-dimensional similarity information calculation function is constructed to perform dynamic consistency score calculation, and face feature dynamic consistency score parameters are obtained;

[0071] The improved multi-dimensional similarity information calculation function, in particular, performs dynamic consistency score calculation by calculating feature space similarity, collection interval time decay similarity, and definition compensation similarity, and the calculation formula is:

[0072] S dynamic = α·S cos + β·S temporal + γ·S cross ;

[0073] In the formula, S dynamic is the face feature dynamic consistency score parameter, α is the feature similarity weight, S cos is the feature space similarity parameter, β is the collection interval time decay similarity weight, S temporal is the collection interval time decay similarity parameter, γ is the definition compensation similarity weight, and S cross is the definition compensation similarity parameter.

[0074] Preferably, the default value of the feature similarity weight α is 0.6, the default value of the collection interval time decay similarity weight β is 0.2, and the default value of the definition compensation similarity weight γ is 0.2.

[0075] The feature space similarity parameter is calculated according to the feature cosine similarity of the visitor face template data and the visited face feature template data in the two-way identity authentication face feature template data.

[0076] The acquisition interval time decay similarity parameter is calculated by calculating the acquisition time stamp in the two-way identity authentication face feature template data.

[0077] The definition compensation similarity parameter is calculated by improving the definition value similarity calculation formula, and the calculation formula is:

[0078]

[0079] In the formula, is the definition Tenengrad value in the visitor face template data, is the definition Tenengrad value in the visited face feature template data.

[0080] Step S22: Selecting a stable face template, specifically, for each face feature template in the two-way identity authentication face feature template data, a transformation gradient calculation under different pose angles is performed to obtain a pose change sensitivity parameter, and the pose change sensitivity parameter is used as a stability score for ranking the feature stability of the two-way identity authentication face feature template data. The highest face feature template data is selected as the stable face template data, and a stable face feature template is obtained.

[0081] The calculation formula of the pose change sensitivity parameter is:

[0082]

[0083] In the formula, R(f) is the pose change sensitivity parameter calculation function, f is the two-way identity authentication face feature template data, K is the total number of face feature templates, k is the face feature template index, is the pose angle direction of the kth face feature template.

[0084] Step S23: Dynamic credibility fusion score, specifically, according to the face feature dynamic consistency score parameter, the stable face feature template and the image quality score in the two-way identity authentication face feature template data, a dynamic credibility comprehensive score fusion calculation is performed to obtain a face template credibility comprehensive score parameter.

[0085] The calculation formula of the dynamic credibility comprehensive score fusion calculation is:

[0086] S trust= σ(w1·S dynamic + w2·C(f v , f d ) + w3·Q(q));

[0087] In the formula, S trust is a face template credibility comprehensive score parameter, σ is a Sigmoid activation function, w1 is a dynamic consistency weight, S dynamic is a face feature dynamic consistency score parameter, w2 is a stable face template weight, C(f v , f d ) is a stable face feature template, w3 is an image quality weight, and Q(q) is normalized image quality score data, which is specifically calculated by a hyperbolic tangent function, and q is an image quality score in the face feature template data of the two-way identity authentication.

[0088] Step S24: optimal template construction, specifically, the stable face feature template is spliced and fused with the features of the visited personnel and the visited personnel, and a light distillation network is constructed according to the spliced and fused stable face feature template data to perform feature dimension compression, thereby obtaining two-way identity authentication template data.

[0089] The two-way identity authentication template data includes compressed features, a sampling timestamp, a face template credibility comprehensive score parameter, a posture change sensitivity parameter, and an image quality score, and the calculation formula is:

[0090] M = <f optimal , t, S trust , R(f), q>;

[0091] In the formula, M is two-way identity authentication template data, f optimal is compressed features, and t is a sampling timestamp.

[0092] Preferably, Table 2 is a model parameter instance table of the light distillation network, as shown in the table, the light distillation network specifically adopts a double-layer fully connected network and introduces a GELU activation function for activation.

[0093] Table 2 Model parameter instance table of light distillation network

[0094]

[0095] Step S25: two-way identity authentication, specifically, two-way identity authentication is performed according to the face template credibility comprehensive score parameter in the two-way identity authentication template data, and score classification processing is performed according to the specific score of the credibility comprehensive score parameter, thereby obtaining two-way identity consistency authentication data.

[0096] The bidirectional identity consistency authentication data specifically includes bidirectional identity authentication template data and bidirectional identity consistency authentication results.

[0097] The score grading processing specifically sets the bidirectional identity consistency authentication result with the human face template credibility comprehensive score parameter greater than or equal to 0.8 as passing verification, and sets the bidirectional identity consistency authentication result less than 0.8 as failing verification.

[0098] By performing the above operation, for the technical problems of large sampling image quality difference, diversified posture change and unstable authentication result of the existing meeting personnel identity authentication system in the actual scene, the application introduces a dynamic consistency scoring mechanism + posture change sensitivity sorting + lightweight distillation network fusion optimization authentication process, which significantly improves the identity matching accuracy and credibility under complex collection conditions. Specifically, the traditional face recognition method is often based on single image or real-time image for feature comparison, which is easily affected by the actual collection environment, such as the backlight of the visiting area, the situation of the visited personnel lowering their heads, etc. which will increase the feature error and cause the authentication result to deviate. Although some systems support multi-frame collection, they do not dynamically model the image quality and time consistency, and lack stability screening. The application constructs a dynamic scoring function based on feature space similarity, collection interval attenuation coefficient and image clarity compensation, and selects the most stable frame in the template as the final authentication feature according to the posture change sensitivity, and stores the compressed standard authentication template through the double-layer distillation network activated by GELU. This mechanism not only provides grading scores and improves fault tolerance during authentication, but also serves as a high-quality reference during monitoring, addressing the practical problems of "unstable sampling and easy authentication errors" in existing solutions.

[0099] Embodiment four, refer to Figure 1 、 Figure 2 This embodiment is based on the above-mentioned embodiments. In step S3, the meeting order management is used to manage the meeting queue order of the visited personnel and the visiting personnel. Specifically, by means of the meeting data passing verification in the bidirectional identity consistency authentication data, the meeting order queue priority judgment, the meeting room resource matching scheduling and the number calling notification are sequentially performed to manage the meeting order, obtain the meeting queue scheduling result, and perform the meeting number calling notification for the visited personnel meeting according to the meeting queue scheduling result.

[0100] The meeting order queue priority judgment specifically sets the lawyer visit as the highest priority, and sequentially queues the meeting order queue according to the reservation time.

[0101] The number calling notification specifically pushes the number calling information and performs voice broadcast through the MQTT Internet of Things protocol.

[0102] Embodiment five, refer toFigure 1 、 Figure 2 and Figure 4 , based on the above embodiment, in step S4, the real-time bidirectional monitoring is used for continuously monitoring the dynamic face recognition of both parties during the meeting, specifically, according to the bidirectional identity consistency authentication data, after the meeting parties enter the meeting room, real-time face image data is collected according to the local camera of the meeting room, and through the improved feature comparison double-path anomaly detection method, real-time bidirectional monitoring is carried out to obtain bidirectional identity monitoring state data, including the following steps:

[0103] Step S41: feature comparison optimization, specifically, by using a lightweight image recognition model, real-time face feature data is obtained by performing feature extraction on the real-time face image data, and an improved cosine similarity with posture change sensitivity parameter intervention is constructed to perform feature comparison similarity calculation on the real-time face feature data and the feature in the bidirectional identity consistency authentication data. Similarity parameter, and the real-time face image with the highest feature comparison similarity parameter is input as monitoring data to obtain the optimal matching degree real-time feature data;

[0104] The lightweight image recognition model specifically adopts a 1M parameter MobileFaceNet compressed version model;

[0105] The calculation formula of the improved cosine similarity with posture change sensitivity parameter intervention is:

[0106]

[0107] In the formula, S match is the feature comparison similarity parameter, f real-time is the real-time face feature data output by the lightweight image recognition model;

[0108] Step S42: double-path anomaly monitoring, specifically, a lightweight three-dimensional convolutional neural network is constructed, and according to the optimal matching degree feature data, double-path anomaly monitoring is performed to obtain anomaly monitoring score data;

[0109] The double-path anomaly monitoring specifically includes face occlusion anomaly and personnel change anomaly;

[0110] The calculation formula of the anomaly monitoring score data is:

[0111] A = 0.6P occl + 0.4P swap ;

[0112] In the formula, A is the anomaly monitoring score data, P occl is the face occlusion anomaly monitoring result, and P swap is the personnel change anomaly monitoring result;

[0113] Preferably, Table 3 is a parameter instance table of the lightweight three-dimensional convolutional neural network, as a table, which takes the last five frames of continuous face image data with the optimal matching degree of real-time feature data as input, sequentially passes through three groups of three-dimensional convolutional layers and pooling layers to construct a main feature extraction channel, obtains inter-frame stable feature distribution through time dimension average pooling operation, and realizes double-path parallel prediction output of abnormal state through two-layer full connection structure.

[0114] Table 3 Parameter instance table of lightweight three-dimensional convolutional neural network

[0115]

[0116] Step S43: Real-time fusion decision, specifically, according to the abnormal monitoring score data, three-level abnormal monitoring decision is made to obtain abnormal state data, and through bidirectional detection of the meeting parties, bidirectional abnormal state data is obtained.

[0117] The calculation formula of the three-level abnormal monitoring decision is:

[0118]

[0119] In the formula, Decision is the abnormal state data, Normal is the normal state, Warning is the pre-warning state, and Alert is the abnormal alarm state.

[0120] Step S44: Abnormal monitoring feedback, specifically, according to the abnormal state in the bidirectional abnormal state data, when the abnormal state is the pre-warning state, a prompt sound is sent to the meeting room, and when the abnormal state is the abnormal alarm state, the current abnormal frame data is automatically intercepted and stored, and a suggestion to terminate the meeting instruction is sent through MQTT, to obtain bidirectional identity monitoring state data.

[0121] The bidirectional identity monitoring state data specifically includes bidirectional abnormal state data, abnormal frame data, and suggestion to terminate the meeting instruction sending record data.

[0122] By performing the above operation, for the existing real-time meeting monitoring system only has basic video acquisition function, lack of automatic abnormal identification and response mechanism, the application realizes the real-time detection and three-level response feedback mechanism of the risk behaviors such as personnel shielding (such as wearing a mask, sunglasses) and replacing a person in the meeting process through the design of the double-path three-dimensional convolution abnormal detection network based on optimal feature matching. Specifically, in the traditional system, the supervisors mainly rely on visual judgment or post-recording review to identify abnormal behaviors, which is easily affected by factors such as monitoring blind area and distraction of attention, resulting in missed reporting of abnormal behaviors. Even if it has basic face recognition ability, it cannot deal with behaviors such as shielding, rapid replacement, or identity switching. The stable feature vector screened out in the two-way authentication template is used as the core reference in the meeting real-time acquisition, the continuous frame image sequence is extracted, the high-order dynamic features of the face behavior change are constructed through three groups of 3D convolution + time sequence average pooling, and then the face shielding score and identity change score are predicted in parallel by the double-output full connection structure. Finally, based on the comprehensive score, the three-level control of "prompt sound-warning-meeting interruption" is executed, which significantly improves the active identification and reaction ability of the system to potential illegal behaviors and solves the structural pain point of "seeing but reacting slowly".

[0123] In an embodiment six, referring to Figure 1 and Figure 2 , this embodiment is based on the above-mentioned embodiments, in step S5, the meeting data is archived, which is used for structuring and archiving the audio and video data and identity authentication information in the meeting process. Specifically, after the meeting is completed, the multi-modal data generated in the meeting process is classified and stored, and is bound with the corresponding meeting record to obtain a meeting archive data set, including the following steps:

[0124] Step S51: audio and video data archiving, specifically starting audio and video synchronous recording at the beginning of the meeting, using a standard encoding protocol for compression and encoding, and logically segmenting every 5 minutes to obtain meeting audio archive data and meeting video archive data;

[0125] Step S52: identity authentication log archiving, specifically, the two-way identity consistency authentication data before the meeting, the real-time identification frame log and the abnormal alarm record in the meeting process are uniformly collected according to the timestamp and the unique meeting number to obtain the identity authentication log archiving data and the abnormal identification alarm information;

[0126] Step S53: structured data index generation, specifically, the unique meeting number is taken as the main index to generate a metadata index table to obtain structured data; the structured data specifically includes the unique meeting number, the meeting personnel information, the meeting start and end time, the meeting video segment file path, the two-way identity consistency authentication data and the monitoring abnormal data;

[0127] Step S54: archiving data storage backup, specifically, synchronously writing the structured data into a local database and a central archiving server to obtain a meeting archiving data set;

[0128] The meeting archiving data set specifically includes meeting audio archiving data, meeting video archiving data, identity authentication log archiving data and abnormal identification alarm information.

[0129] Embodiment seven, referring to Figure 1 and Figure 2 The embodiment is based on the above-mentioned embodiments, and the application provides a meeting management system based on face recognition, which includes a face data acquisition module, a two-way identity authentication module, a meeting order management module, a real-time two-way monitoring module and a meeting data archiving module.

[0130] The face data acquisition module is used for face data acquisition, and through face data acquisition, two-way identity authentication face feature template data is obtained, and the two-way identity authentication face feature template data is sent to the two-way identity authentication module and the meeting order management module.

[0131] The two-way identity authentication module is used for two-way identity authentication, and through two-way identity authentication, two-way identity consistency authentication data is obtained, and the two-way identity consistency authentication data is sent to the meeting order management module and the real-time two-way monitoring module.

[0132] The meeting order management module is used for meeting order management, and through meeting order management, a meeting queuing scheduling result is obtained, and a visited person is met.

[0133] The real-time two-way monitoring module is used for real-time two-way monitoring, and through real-time two-way monitoring, two-way identity monitoring state data is obtained, and the two-way identity monitoring state data is sent to the meeting data archiving module.

[0134] The meeting data archiving module is used for meeting data archiving, and through meeting data archiving, a meeting archiving data set is obtained.

[0135] It should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the term "includes", "contains" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment.

[0136] While embodiments of the application have been shown and described, it is to be understood that the embodiments described are merely exemplary and that changes can be made in the embodiments without departing from the spirit and scope of the application.

[0137] The above description of the application and its embodiments is not restrictive, and the embodiments shown in the drawings are only one of the embodiments of the application, and the actual structure is not limited thereto. In general, if a person skilled in the art is inspired by it, without departing from the purpose of the application, without creative design, similar structure and embodiments of the technical solution should belong to the protection scope of the application.

Claims

1. A meeting management method based on face recognition, characterized in that: The method comprises the following steps: Step S1: face data acquisition, used for collecting and verifying the identity card information and face image of the visitor and the visited person, obtaining a two-way identity authentication face feature template data; Step S2: two-way identity authentication, according to the two-way identity authentication face feature template data, using a dynamic consistency score improved two-way identity authentication method, performing two-way identity authentication, obtaining two-way identity consistency authentication data, comprising the following steps: Step S21: dynamic consistency score calculation, constructing an improved multi-dimensional similarity information calculation function, and performing dynamic consistency score calculation; the improved multi-dimensional similarity information calculation function specifically calculates the feature space similarity, the collection interval time decay similarity and the definition compensation similarity to perform dynamic consistency score calculation; Step S22: stable face template selection; Step S23: dynamic credibility fusion score; Step S24: optimal template construction; Step S25: two-way identity authentication; Step S3: meeting order management, according to the meeting data verified in the two-way identity consistency authentication data, sequentially performing meeting order queue priority judgment, meeting room resource matching scheduling and call number notification, and performing meeting order management, obtaining a meeting queuing scheduling result; Step S4: real-time two-way monitoring, according to the two-way identity consistency authentication data, after the meeting parties enter the meeting room, collecting real-time face image data according to the local camera of the meeting room, and performing real-time two-way monitoring through an improved feature comparison double-path abnormality detection method, obtaining two-way identity monitoring state data, comprising the following steps: Step S41: feature comparison optimization; Step S42: double-path abnormality monitoring; Step S43: real-time fusion decision; Step S44: abnormality monitoring feedback; Step S5: meeting data archiving, after the meeting is completed, the multi-modal data generated in the meeting process is classified and stored, and is bound with the corresponding meeting record, obtaining a meeting archiving data set. 2.The meeting management method based on face recognition according to claim 1, characterized in that: In step S1, the face data acquisition is used for collecting and verifying the identity card information and face image of the visitor and the visited person, specifically through the visited person identity information acquisition to obtain the visited person face feature template data, and through the visitor face real-time acquisition to obtain the visitor face feature extraction, obtaining the visitor face template data, and through the data fusion of the visited person face feature template data and the visitor face template data, obtaining the two-way identity authentication face feature template data; The data fusion specifically structures the encoding of the visited person face feature template data and the visitor face template data after standardization processing, constructs an identity matching data object, and respectively retains the continuous frame face feature template, the collection timestamp, the collection device number, the image quality score and the identity association identifier, to obtain the two-way identity authentication face feature template data. 3.The meeting management method based on face recognition of claim 2, characterized in that: In step S2, the bidirectional identity authentication is used to perform local and remote face image comparison at the visiting end and the visited end simultaneously. Specifically, the bidirectional identity authentication face feature template data is used to perform bidirectional identity authentication by using a dynamic consistency score improved bidirectional identity authentication method to obtain bidirectional identity consistency authentication data, including the following steps: Step S21: dynamic consistency score calculation. Specifically, the bidirectional identity authentication face feature template data is used to construct an improved multi-dimensional similarity information calculation function to perform dynamic consistency score calculation to obtain face feature dynamic consistency score parameters. Step S22: stable face template selection. Specifically, the transformation gradient calculation under different pose angles is performed on each face feature template in the bidirectional identity authentication face feature template data to obtain pose change sensitivity parameters, which are used as stability scores for feature stability ranking of the bidirectional identity authentication face feature template data. The face feature template data with the highest feature stability is selected as the stable face template data to obtain a stable face feature template. Step S23: dynamic credibility fusion score. Specifically, the face feature dynamic consistency score parameters, the stable face feature template, and the image quality score in the bidirectional identity authentication face feature template data are used to perform dynamic credibility comprehensive score fusion calculation to obtain face template credibility comprehensive score parameters. Step S24: optimal template construction. Specifically, the stable face feature template is fused by splicing the features of the visiting personnel and the visited personnel, and a light distillation network is constructed based on the fused stable face feature template data to perform feature dimension compression to obtain bidirectional identity authentication template data. Step S25: bidirectional identity authentication. Specifically, the face template credibility comprehensive score parameters in the bidirectional identity authentication template data are used to perform bidirectional identity authentication, and score classification processing is performed according to the specific score value of the credibility comprehensive score parameters to obtain bidirectional identity consistency authentication data. The bidirectional identity consistency authentication data specifically includes bidirectional identity authentication template data and bidirectional identity consistency authentication results. The score classification processing specifically sets the bidirectional identity consistency authentication results with face template credibility comprehensive score parameters greater than or equal to 0.8 as passing verification, and sets the bidirectional identity consistency authentication results with face template credibility comprehensive score parameters less than 0.8 as failing verification. 4.The meeting management method based on face recognition of claim 3, wherein: In step S3, the meeting order management is used to manage the meeting queuing order of the visited personnel and the visitors. Specifically, the meeting data passing verification in the bidirectional identity consistency authentication data is used to perform meeting order queue priority judgment, meeting room resource matching scheduling, and call notification in sequence to perform meeting order management to obtain meeting queuing scheduling results. The meeting call notification is performed according to the meeting queuing scheduling results to perform the meeting of the visited personnel. The meeting order queue priority judgment specifically sets the lawyer visit as the highest priority and queues the meeting order queue according to the reservation time in sequence. The call notification is specifically performed by pushing call information through an MQTT Internet of Things protocol and performing voice broadcasting.

5. The meeting management method based on face recognition according to claim 4, characterized in that: In step S4, the real-time bidirectional monitoring is used for continuously monitoring the faces of the two parties during the meeting, specifically, according to the bidirectional identity consistency authentication data, after the two parties enter the meeting room, real-time face image data is collected according to the local camera of the meeting room, and real-time bidirectional monitoring is performed through an improved feature comparison double-path anomaly detection method to obtain bidirectional identity monitoring state data, including the following steps: Step S41: feature comparison optimization, specifically, through the use of a lightweight image recognition model, feature extraction is performed according to the real-time face image data to obtain real-time face feature data, and an improved cosine similarity with posture change sensitivity parameter intervention is constructed to perform feature comparison similarity calculation on the real-time face feature data and the feature in the bidirectional identity consistency authentication data, to obtain a feature comparison similarity parameter, and the real-time face image with the highest feature comparison similarity parameter is input as monitoring data to obtain optimal matching degree real-time feature data; Step S42: double-path anomaly monitoring, specifically, a lightweight three-dimensional convolutional neural network is constructed, and double-path anomaly monitoring is performed according to the optimal matching degree real-time feature data to obtain anomaly monitoring score data; The double-path anomaly monitoring specifically includes face occlusion anomaly and personnel change anomaly; Step S43: real-time fusion decision, specifically, according to the anomaly monitoring score data, a three-level anomaly monitoring decision is made to obtain anomaly state data, and bidirectional detection is performed on the two parties to obtain bidirectional anomaly state data; Step S44: anomaly monitoring feedback, specifically, according to the anomaly state in the bidirectional anomaly state data, when the anomaly state is a warning state, a prompt sound is sent to the meeting room, and when the anomaly state is an anomaly alarm state, the current anomaly frame data is automatically intercepted and stored, and an instruction to terminate the meeting is sent through MQTT, to obtain bidirectional identity monitoring state data; The bidirectional identity monitoring state data specifically includes bidirectional anomaly state data, anomaly frame data, and suggestion termination meeting instruction sending record data. 6.The meeting management method based on face recognition of claim 5, characterized in that: In step S5, the meeting data archiving is used for structuring and archiving the audio and video data and identity authentication information in the meeting process, specifically, after the meeting is completed, the multi-modal data generated in the meeting process is classified and stored, and is bound with the corresponding meeting record to obtain a meeting archiving data set.

7. The meeting management method based on face recognition according to claim 6, characterized in that: The meeting data archiving includes the following steps: Step S51: audio and video data archiving, specifically, audio and video synchronous recording is started at the beginning of the meeting, compressed encoding is performed using a standard encoding protocol, and logical segmentation is performed every 5 minutes to obtain meeting audio archiving data and meeting video archiving data; Step S52: identity authentication log archiving, specifically, the bidirectional identity consistency authentication data before the meeting, the real-time recognition frame log during the meeting, and the anomaly alarm record are collected according to the time stamp and the meeting unique number to obtain identity authentication log archiving data and anomaly recognition alarm information; Step S53: structured data index generation, specifically, taking the meeting unique number as the main index, generating a metadata index table, and obtaining structured data; the structured data specifically includes a meeting unique number, participant information, meeting start and end times, meeting video segment file paths, two-way identity consistency authentication data, and monitoring exception data; Step S54: archive data storage backup, specifically, synchronously writing the structured data into a local database and a central archive server, and obtaining a meeting archive data set; The meeting archive data set specifically includes meeting audio archive data, meeting video archive data, identity authentication log archive data, and exception identification alarm information.

8. A face recognition based meeting management system for implementing a face recognition based meeting management method according to any one of claims 1-7, characterized in that: The meeting system includes a face data collection module, a two-way identity authentication module, a meeting order management module, a real-time two-way monitoring module, and a meeting data archiving module. 9.The meeting management system based on face recognition of claim 8, wherein: The face data collection module is configured to collect face data, obtain two-way identity authentication face feature template data through face data collection, and send the two-way identity authentication face feature template data to the two-way identity authentication module and the meeting order management module; The two-way identity authentication module is configured to perform two-way identity authentication, obtain two-way identity consistency authentication data through two-way identity authentication, and send the two-way identity consistency authentication data to the meeting order management module and the real-time two-way monitoring module; The meeting order management module is configured to manage meeting orders, obtain a meeting queuing scheduling result through meeting order management, and perform a meeting with a person to be visited; The real-time two-way monitoring module is configured to perform real-time two-way monitoring, obtain two-way identity monitoring state data through real-time two-way monitoring, and send the two-way identity monitoring state data to the meeting data archiving module; The meeting data archiving module is configured to archive meeting data, and obtain a meeting archive data set through meeting data archiving.

Images