Automatic Labeling Method for Intelligent Conference Participants Based on Image Recognition

Abstract

The present invention relates to the technical fields of image recognition and conference management, and particularly relates to an automatic marking method for intelligent conference participants based on image recognition, which includes the following steps: S1: Real-time acquisition of the facial images of the participants to generate structured participant image data; S2: Extracting the facial features of the participants from the structured participant image data to generate an anti-occlusion feature vector; S3: Matching the anti-occlusion feature vector with a pre-constructed participant identity database; S4: Associating preset permission rules in the conference management system to generate composite marking information; S5: Performing homomorphic encryption processing on the composite marking information; S6: Decrypting the encrypted marking data packet and superimposing it on the real-time video stream of the participants. With the present invention, through the anti-occlusion feature extraction, automatic permission association, and encrypted dynamic annotation mechanism, the accuracy of participant identity recognition, the real-time nature of permission management, and the security of data transmission in complex conference scenarios are achieved.

Description

Technical Field

[0001] The present invention relates to the technical fields of image recognition and conference management, and particularly to an automatic marking method for intelligent conference participants based on image recognition. Background Art

[0002] With the popularization of multi-party remote collaborative work and large-scale offline conferences, conference organizers have put forward higher requirements for the real-time identification of participants' identities and permission management; traditional conference sign-in and identity verification mainly rely on manual verification or radio frequency card identification, which not only has low efficiency and is error-prone, but also cannot achieve accurate and stable identity recognition and permission control in the face of complex scenarios such as participants wearing masks, glasses or being blocked.

[0003] Most of the existing image recognition technologies are based on full-face feature extraction, lacking robust processing of local features and unable to adapt to non-ideal shooting conditions such as partial facial occlusion, uneven illumination or pose deviation; in addition, the generation and transmission of marked data in the current conference system mostly do not adopt an encryption mechanism, there is a security risk of data leakage. Therefore, there is an urgent need for an automatic marking method for intelligent conference participants based on image recognition to solve the above problems. Summary of the Invention

[0004] Based on the above purpose, the present invention provides an automatic marking method for intelligent conference participants based on image recognition.

[0005] The automatic marking method for intelligent conference participants based on image recognition includes the following steps:

[0006] S1: Through multi-angle image acquisition devices deployed in the conference scene, the facial images of the participants are obtained in real time, and the facial images are subjected to illumination compensation, pose correction and occlusion area segmentation processing to generate structured participant image data;

[0007] S2: Extract the facial features of the participants from the structured participant image data. For the scenes of wearing masks and glasses occlusion, a local feature enhancement algorithm is adopted to extract the geometric distribution features of the eye area and the texture features of the nose bridge area to generate an anti-occlusion feature vector;

[0008] S3: Match the anti-occlusion feature vector with a pre-constructed participant identity database, and the participant identity database includes encrypted biometric codes, position information and permission labels of registered participants;

[0009] If the match is successful, the corresponding identity label data is output;

[0010] If the match fails, an exception marking for unregistered personnel is triggered to generate temporary identity label data including a temporary number and an admission time;

[0011] S4: According to the identity tag data or temporary identity tag data, associate the preset permission rules in the conference management system to generate composite tag information including name, position, permission level, and abnormal status identifier;

[0012] S5: Perform homomorphic encryption processing on the composite tag information to generate an encrypted tag data packet;

[0013] S6: In the interaction interface of the conference management terminal, decrypt the encrypted tag data packet and superimpose it on the real-time video stream of the participants, and trigger dynamic refresh and repositioning of the tag information based on events such as the movement of the participants' positions or permission changes.

[0014] Optionally, the S1 specifically includes:

[0015] S11: Through the camera group preset at the four corners and the central position in the conference scene, synchronously trigger shooting to cover the full-angle facial area of the participants, and generate an original multi-view image set;

[0016] S12: Perform light intensity analysis on the original multi-view image set. For overexposed or underexposed areas in a single-frame image, adopt a multi-frame fusion compensation algorithm to extract the optimal light pixels in each area and generate a light-balanced image;

[0017] S13: Based on the light-balanced image, detect the coordinates of facial key points, and determine the head deflection angle by calculating the spatial vector angle between the tip of the nose point and the points of the two earlobes; when the deflection angle exceeds the preset threshold, use affine transformation to correct the facial image to the front view angle and generate a pose-normalized image;

[0018] S14: Perform edge detection on the pose-normalized image, identify the contour of the mask, glasses, or hand occlusion area, and generate a repaired and filled image of the occlusion area according to the facial skin texture features that are not occluded;

[0019] S15: Bind the repaired and filled image with the associated acquisition timestamp, camera position code, light compensation parameter, and pose correction parameter, and encapsulate it into structured participant image data containing multi-dimensional metadata.

[0020] Optionally, the S2 specifically includes:

[0021] S21: Receive the structured participant image data generated in S1, locate the facial area in the image, and use a preset facial key point detection model to mark key points such as the corners of the eyes, the ends of the eyebrows, the center of the eyebrows, the tip of the nose, and the root of the nose, and delimit the boundary range of the periorbital area and the nasal bridge area;

[0022] S22: In the delimited periorbital area, calculate the geometric relationships between the key points, including the Euclidean distance and relative angle between the end of the eyebrow and the center of the eyebrow, and between the corners of the eyes, and construct the local geometric distribution features of the face;

[0023] S23: In the nasal bridge area, adopt the local texture gradient analysis method to analyze the gray-scale change distribution in this area, extract the gray-scale gradient information characterizing the skin texture difference, and extract multi-scale local texture features in the way of fixed window sliding;

[0024] S24: Stitch and fuse the geometric distribution features and texture features obtained in S22 and S23 to generate a one-dimensional anti-occlusion feature vector.

[0025] Optionally, the specific steps of S21 include:

[0026] S211: Based on the structured participant image data, obtain the image coordinates of the eye corner points, eyebrow tail points, eyebrow center points, nose tip points, and nose root points through a pre-trained face key point detection model;

[0027] S212: According to the detected key point coordinates, define the bounding box of the periorbital region. Taking the outer eye corner points of the left and right eyes as the center, the width of the box body is taken as 1.5 times the Euclidean distance between the two outer eye corner points, the height is taken as 0.8 times the vertical distance from the eyebrow center point to the nose tip point, and the center line of the box body coincides with the line connecting the two outer eye corners;

[0028] S213: Define the bounding box of the nasal bridge region. Taking the midpoint of the nose root point and the nose tip point as the center of the box body, the height of the box body is set as 1.2 times the Euclidean distance between the nose root point and the nose tip point, the width of the box body is 0.6 times the horizontal distance between the two inner eye corner points, and the vertical center line of the box body coincides with the line connecting the nose root point and the nose tip point.

[0029] Optionally, the specific steps of S22 include:

[0030] S221: Using the facial image coordinates obtained in S21, taking the eyebrow center point as the reference, calculate the Euclidean distances between the eyebrow center point and the left eyebrow tail point, the eyebrow center point and the right eyebrow tail point, and the left outer eye corner point and the right outer eye corner point respectively to obtain the characteristic lengths of the periorbital region;

[0031] S222: Calculate the angles formed by connecting the lines with the left and right eyebrow tail points with the eyebrow center point as the vertex respectively, and the angles formed by connecting the lines with the left and right outer eye corner points with the eyebrow center point as the vertex respectively to obtain the characteristic angles of the periorbital region;

[0032] S223: Normalize the Euclidean distances obtained in S221 and the angles obtained in S222;

[0033] S224: Sequentially stitch the normalized characteristic lengths and characteristic angles to form the facial local geometric distribution features.

[0034] Optionally, the specific steps of S23 include:

[0035] S231: Extract the grayscale image sub-region from the nose bridge region bounding box delimited by S21, denoted as the region to be analyzed;

[0036] S232: Taking each pixel point in the region to be analyzed as the center, calculate the gradient magnitude of the grayscale value along the horizontal and vertical directions of the image respectively, to obtain the local gradient intensity information;

[0037] S233: Based on the gradient magnitude, with a square window of a preset size, traverse the region to be analyzed by sliding in a fixed step length, and statistically analyze the histogram distribution of the gradient magnitude within each window to form a local texture feature histogram;

[0038] S234: Change the window size, repeat S233, extract the gradient histograms with different window sizes respectively, and splice the histogram features under different window sizes in sequence to form local texture features with multi-scale information.

[0039] Optionally, the specific steps of S3 include:

[0040] S31: Extract the encrypted biometric code of the registered attendee from the pre-constructed attendee identity database, and decrypt the encrypted biometric code and convert it into an identity feature vector in a standard format;

[0041] S32: Calculate the similarity between the anti-occlusion feature vector obtained in S2 and each identity feature vector one by one, and calculate the matching similarity value between each identity feature vector and the anti-occlusion feature vector through the cosine similarity method;

[0042] S33: Sort the matching similarity values obtained in S32, select the identity feature vector with the highest matching similarity, and determine whether the similarity value is higher than the preset identity matching threshold;

[0043] S34: If the highest matching similarity value in S33 exceeds the identity matching threshold, it is determined that the match is successful, and output the identity label data associated with the corresponding identity feature vector; if the highest matching similarity value does not exceed the identity matching threshold, it is determined that the match fails, trigger the unregistered person exception marking program, and output the temporary identity label data including the temporary number and the admission time.

[0044] Optionally, the specific steps of S4 include:

[0045] S41: Receive the identity label data or the temporary identity label data output by S3, and retrieve the corresponding identity attribute information according to the identity number in the identity label data or the temporary number in the temporary identity label data;

[0046] S42: Based on the preset permission rule table in the conference management system, using the position information in the identity attribute information retrieved in S41 as an index, automatically determine the permission level of the attendee;

[0047] S43: Determine whether the current attendee identity data comes from temporary identity tag data. If the attendee data comes from temporary identity tag data, add an exception status flag for the attendee; if it comes from registered identity tag data, set the exception status flag to the default normal status;

[0048] S44: Merge the name and position information obtained in S41, the permission level obtained in S42, and the exception status flag determined in S43 to form a structured data record and generate composite tag information.

[0049] Optionally, the specific steps of S5 are as follows:

[0050] S51: Receive the composite tag information generated in S4, perform homomorphic encryption processing using the preset public key, and encrypt each data field including name, position, permission level, and exception status flag item by item to generate corresponding ciphertext;

[0051] S52: After splicing the ciphertext of each field generated in S51 in a predetermined order, add a predefined data packet header and a checksum tail marker, and encapsulate it into a complete encrypted tag data packet that conforms to the parsing protocol of the conference management terminal;

[0052] S53: Through a preset independent communication channel, send the encrypted tag data packet encapsulated in S52 to the conference management terminal using a secure communication protocol. This communication channel is physically isolated from the video stream communication channel and is only used to transmit encrypted identity tag information data;

[0053] S54: After receiving the encrypted tag data packet, the conference management terminal checks the header and tail of the data packet according to the agreed parsing protocol.

[0054] Optionally, the specific steps of S6 are as follows:

[0055] S61: After the conference management terminal receives the encrypted tag data packet transmitted in S5, perform homomorphic decryption on the ciphertext of each field in the data packet item by item using the pre-stored private key, and decrypt to obtain the plaintext data of the composite tag information;

[0056] S62: According to the attendee identity number or temporary number, automatically associate the decrypted composite tag information with the corresponding attendee image position coordinates in the real-time video stream of the attendee. After performing image coordinate matching, superimpose and display the name, position, permission level, and exception status flag near the corresponding attendee position;

[0057] S63: Monitor the location information of the participants in real time. If it is detected that the location coordinates of a participant have changed compared to the previous moment, trigger a location movement event, recalculate the display coordinates of the marker information in the video frame, and immediately update the overlay location.

[0058] S64: Monitor the status information of the participants' permission levels in the conference management system in real time. If the system detects a change in a participant's permission level, update the data content of the composite marker information according to the latest permission level, and synchronously refresh the overlay marker displayed on the video frame.

[0059] S65: Continuously execute S63 and S64 to ensure that the marker information of the participants displayed in the real-time video stream always remains consistent with the actual locations and real-time permission statuses of the participants.

[0060] Advantages of the present invention:

[0061] In the present invention, through multi-angle image acquisition and preprocessing means, combined with a local feature enhancement algorithm, it is possible to stably extract the key structures and texture information of the eye area and the bridge of the nose in the case of a participant wearing a mask, glasses, or hand occlusion, etc., generate a robust anti-occlusion feature vector, and improve the accuracy and adaptability of face recognition; at the same time, by adopting a multi-scale texture analysis and geometric distribution feature fusion method, the expression ability of the features under complex lighting and pose change conditions is further enhanced, ensuring reliable and stable identity recognition results in the actual conference environment.

[0062] In the present invention, by automatically associating the recognized identity information with the permission rules in the conference management system, constructing composite marker information including name, position, permission level, and exception status identifier, and completing data encapsulation and transmission through homomorphic encryption and an independent communication channel, the secure release and real-time update of identity data are realized; the content of the overlay marker can be dynamically repositioned and refreshed according to the location movement and permission change of the participants, thereby significantly improving the automation level of identity annotation and the efficiency of permission management at the conference site. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only those of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0064] Figure 1 Schematic diagram of the intelligent conference participant automatic marking method according to an embodiment of the present invention;

[0065] Figure 2 Schematic diagram of the method for generating an anti-occlusion feature vector according to an embodiment of the present invention. Detailed Implementation Modes

[0066] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. At the same time, it should be noted here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments. For some well-known technologies, those skilled in the art can also adopt other alternative methods for implementation; moreover, the accompanying drawings are only for more specifically describing the embodiments and are not intended to specifically limit the present invention.

[0067] It should be pointed out that in the specification, when referring to "an embodiment", "embodiment", "exemplary embodiment", "some embodiments", etc., it indicates that the described embodiment may include specific features, structures or characteristics, but not necessarily every embodiment includes such specific features, structures or characteristics. In addition, when combining an embodiment to describe a specific feature, structure or characteristic, implementing such feature, structure or characteristic in combination with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the relevant art.

[0068] Generally, terms can be understood at least in part from their use in context. For example, at least in part depending on the context, the term "one or more" used herein can be used to describe any feature, structure or characteristic in a singular sense, or can be used to describe a combination of features, structures or characteristics in a plural sense. In addition, the term "based on" can be understood as not necessarily intended to convey a set of exclusive factors, but instead, at least in part depending on the context, allowing for the existence of other factors that may not be explicitly described.

[0069] As Figure 1 - Figure 2 shown, the intelligent conference participant automatic marking method based on image recognition includes the following steps:

[0070] S1: Through multi-angle image acquisition devices deployed in the conference scene, the facial images of the participants are obtained in real time, and the facial images are subjected to light compensation, pose correction and occlusion area segmentation processing to generate structured participant image data;

[0071] S2: Extract the facial features of the participants from the structured participant image data. For the scenarios of wearing masks and glasses occlusion, a local feature enhancement algorithm is used to extract the geometric distribution features of the eye area and the texture features of the nose bridge area to generate an anti-occlusion feature vector;

[0072] S3: Match the anti-occlusion feature vector with a pre-constructed participant identity database, which includes the encrypted biometric codes, position information and permission tags of the registered participants;

[0073] If the match is successful, the corresponding identity tag data is output;

[0074] If the matching fails, an unregistered person exception flag is triggered, and temporary identity tag data including a temporary number and an entry time is generated;

[0075] S4: According to the identity tag data or the temporary identity tag data, associate with the preset permission rules in the conference management system, and generate composite tag information including name, position, permission level, and exception status identifier;

[0076] S5: Perform homomorphic encryption processing on the composite tag information to generate an encrypted tag data packet;

[0077] S6: In the interactive interface of the conference management terminal, decrypt the encrypted tag data packet and overlay it on the real-time video stream of the participant, and trigger the dynamic refresh and repositioning of the tag information based on the participant's position movement or permission change event.

[0078] S1 specifically includes:

[0079] S11: Through the camera group preset at the four corners and the central position in the conference scene, synchronously trigger shooting to cover the full-angle facial area of the participant, and generate an original multi-view image set;

[0080] S12: Perform light intensity analysis on the original multi-view image set. For overexposed or underexposed areas in a single-frame image, adopt a multi-frame fusion compensation algorithm to extract the optimal light pixel values of each area, and generate a light-balanced image; Specifically, let the gray value of the pixel point in the frame image be , and its local light confidence level be , then the pixel value calculation formula of the light-balanced image is:

[0081] , where represents the pixel value of the fused image at the position , is the total number of frames; this formula can achieve the smoothing processing of local light extreme pixels and improve the overall brightness balance of the image;

[0082] S13: Based on the light-balanced image, detect the coordinates of facial key points, and determine the head deflection angle by calculating the spatial vector angle between the tip of the nose point and the points of the two earlobes; when the deflection angle exceeds the preset threshold, use affine transformation to correct the facial image to a frontal view, and generate a pose-normalized image; Specifically, let the coordinates of the tip of the nose point be , and the left ear point and the right ear point be , then the head deflection angle is calculated by the following formula:

[0083] , if (where If the set threshold is met), an affine transformation matrix is executed The image is corrected, and the calculation formula is:

[0084] ; where is the rotation angle, is the translation amount to ensure that the face in the corrected image faces the camera directly;

[0085] S14: Perform edge detection on the pose-normalized image, identify the contours of the mask, glasses, or hand occlusion areas, and generate a repaired and filled image for the occlusion areas based on the facial skin texture features of the unoccluded areas;

[0086] S15: Bind the repaired and filled image to the associated acquisition timestamp, camera position encoding, light compensation parameters, and pose correction parameters, and package it into structured participant image data containing multi-dimensional metadata. The above steps improve the image light balance through multi-frame fusion, supplemented by pose correction and occlusion repair based on key point calculations, which can ensure that even in low light, side face, or occlusion situations, unified, complete, and recognizable structured participant image data can be generated, significantly enhancing the accuracy and robustness of subsequent face recognition and identity marking.

[0087] S2 specifically includes:

[0088] S21: Receive the structured participant image data generated in S1, locate the facial area in the image, and use a preset face key point detection model to mark key points such as the outer corners of the eyes, the ends of the eyebrows, the center of the eyebrows, the tip of the nose, and the root of the nose, and delimit the boundary ranges of the periorbital area and the nasal bridge area;

[0089] S22: Calculate the geometric relationships between the key points within the delimited periorbital area, including the Euclidean distances and relative angles between the ends of the eyebrows to the center of the eyebrows and between the outer corners of the eyes to the outer corners of the eyes, and construct the facial local geometric distribution features for identifying the stable structures of individual features;

[0090] S23: In the nasal bridge area, use the local texture gradient analysis method to analyze the gray level change distribution in this area, extract the gray level gradient information representing the skin texture differences, and extract multi-scale local texture features in a fixed window sliding manner;

[0091] S24: Concatenate and fuse the geometric distribution features and texture features obtained in S22 and S23 to generate a one-dimensional anti-occlusion feature vector. This feature vector has recognition robustness against occlusion areas such as glasses and masks, and retains the identity feature information of the areas on the face that are not easily occluded. Through the above steps, by extracting regional features from the periorbital and nasal bridge areas in the structured image and fusing the local geometric structure and texture difference information, the interference of the occlusion area on the overall recognition can be effectively bypassed, significantly improving the integrity and recognition accuracy of facial feature expression under occlusion conditions, and providing a stable input basis for anti-occlusion identity recognition.

[0092] S21 specifically includes:

[0093] S211: Based on the structured attendee image data, obtain the image coordinates of the outer corner points of the eyes, the end points of the eyebrows, the center point of the eyebrows, the tip point of the nose, and the root point of the nose through a pre-trained face key point detection model.

[0094] S212: According to the detected key point coordinates, define the bounding box of the periorbital area. Taking the outer corner points of the left and right eyes as the center, the width of the box is taken as 1.5 times the Euclidean distance between the two outer corner points, and the height is taken as 0.8 times the vertical distance from the center point of the eyebrows to the tip point of the nose. The center line of the box coincides with the line connecting the two outer corner points.

[0095] S213: Define the bounding box of the nasal bridge area. Taking the midpoint of the root point of the nose and the tip point of the nose as the center of the box, the height of the box is set to 1.2 times the Euclidean distance between the root point of the nose and the tip point of the nose, and the width of the box is 0.6 times the horizontal distance between the two inner corner points. The vertical center line of the box coincides with the line connecting the root point of the nose and the tip point of the nose.

[0096] Specifically, the width and height of the periorbital area bounding box are calculated by the following formulas respectively:

[0097] ;

[0098] ;

[0099] The width and height of the nasal bridge area bounding box are calculated by the following formulas respectively:

[0100] ;

[0101] ; where, respectively represent the coordinates of the left and right outer corner points of the eyes, respectively represent the coordinates of the left and right inner corner points of the eyes, and respectively represent the coordinates of the glabella point, the tip of the nose point, and the nasion point; the above steps delimit the boundaries of the periorbital and nasal bridge regions through explicit geometric formulas, can accurately locate stable and reliable local facial regions, provide an accurate spatial basis for subsequent local feature extraction, and further improve the consistency and anti-occlusion robustness of feature extraction.

[0102] Obtaining the image coordinates of the outer corner of the eye point, the end of the eyebrow point, the glabella point, the tip of the nose point, and the nasion point through a pre-trained face key point detection model includes the following steps:

[0103] S2111: Input the structured participant image data into the pre-trained face key point detection model, which is based on a convolutional neural network structure and has been trained through a large number of labeled face training sets, and has the ability to predict high-precision face key points;

[0104] S2112: The pre-trained model first extracts features from the input image, outputs multi-scale facial feature maps, and then uses a fully connected regression layer to perform regression prediction on the feature maps, outputting the two-dimensional coordinate values of each key point;

[0105] S2113: According to the standardized coordinate values output by the model, combined with the actual size of the input image, perform scale mapping on the standardized coordinates of the outer corner of the eye point, the end of the eyebrow point, the glabella point, the tip of the nose point, and the nasion point to obtain the accurate coordinate positions of each key point in the image coordinate system;

[0106] The scale mapping calculation formula for the actual image coordinates of the key points is:

[0107] , where, is the actual coordinate of the key point in the image, is the standardized coordinate output by the model, and are the actual width and height of the input image respectively.

[0108] S22 specifically includes:

[0109] S221: Using the facial image coordinates obtained in S21, with the glabella point as the reference, calculate the Euclidean distances between the glabella point and the left end of the eyebrow point, the glabella point and the right end of the eyebrow point, and the left outer corner of the eye point and the right outer corner of the eye point respectively to obtain the characteristic lengths of the periorbital region; specifically, let the Euclidean distance be , and its calculation formula is:

[0110] , in the formula, represents the image coordinates of the starting point based on the glabella point in the structured image; represents the image coordinates of the target point corresponding to the glabella point in the structured image;

[0111] S222: Calculate the angles formed by connecting the midpoint of the eyebrows with the left and right eyebrow ends, and the angles formed by connecting the midpoint of the eyebrows with the left and right outer canthi respectively, to obtain the characteristic angles in the eye region;

[0112] Let the angle be , and its calculation formula is: , where represents the image coordinates of the midpoint of the eyebrows; represents the coordinates of the left key point corresponding to the midpoint of the eyebrows as the vertex; represents the coordinates of the right key point corresponding to the midpoint of the eyebrows as the vertex; the symbol • represents the dot product operation of two vectors; the symbol represents the modulus of the vector;

[0113] S223: Normalize the Euclidean distance obtained in S221 and the angles obtained in S222 to eliminate the influence of the image size differences of different participants on the feature stability;

[0114] The length normalization formula is: ;

[0115] The angle normalization formula is: ; where and are the normalized length value and angle value respectively; and represent the upper limit value and lower limit value of the preset length normalization respectively; and represent the upper limit value and lower limit value of the preset angle normalization respectively;

[0116] S224: Concatenate the normalized feature length and feature angle in sequence to form a facial local geometric distribution feature descriptor, which provides geometric structure feature input for the subsequent anti-occlusion feature vector; through the above steps, the stability of the local geometric features against occlusion interference can be significantly enhanced, and the consistency and robustness of the feature expression can be guaranteed, thereby improving the accuracy of identity recognition.

[0117] S23 specifically includes:

[0118] S231: Extract the grayscale image sub-region from the nose bridge region bounding box delimited in S21, denoted as the region to be analyzed;

[0119] S232: Taking each pixel point in the region to be analyzed as the center, calculate the gradient magnitude of the grayscale value along the horizontal and vertical directions of the image respectively to obtain the local gradient intensity information;

[0120] The gradient magnitude The calculation formula is: , where represents the gradient value of the grayscale change along the horizontal direction of the image, represents the gradient value of the grayscale change along the vertical direction of the image; and are calculated respectively through the following Sobel operators:

[0121] ;

[0122] ;

[0123] wherein, is the grayscale value of the pixel point at the area to be analyzed;

[0124] S233: Based on the gradient magnitude, with a square window of a preset size, slide and traverse the area to be analyzed at a fixed step length, and statistically analyze the histogram distribution of the gradient magnitude within each window to form a local texture feature histogram;

[0125] The statistical method of the gradient magnitude histogram is: , wherein, is the histogram frequency when the gradient magnitude is an integer value ; is the current sliding window area; is an indicator function, which takes the value of 1 when the condition in the parentheses holds, otherwise 0; round represents the rounding operation on the gradient magnitude;

[0126] S234: Change the window size, repeat S233, extract the gradient histograms respectively with different window sizes, and splice the histogram features under different window sizes in sequence to form a local texture feature descriptor with multi-scale information for the subsequent construction of the anti-occlusion feature vector; The above steps can capture rich local texture details through the accurate calculation of the local grayscale gradient in the bridge of the nose area and the multi-scale window analysis, effectively improve the discrimination ability and stability of the texture features for the recognition of the occlusion area, and provide a highly reliable local texture feature basis for the identification of the participants' identities.

[0127] S3 specifically includes:

[0128] S31: Extract the encrypted biometric feature codes of the registered participants from the pre-constructed participants' identity database, and decrypt the encrypted biometric feature codes and convert them into identity feature vectors in a standard format;

[0129] S32: Calculate the similarity between the anti-occlusion feature vector obtained in S2 and each identity feature vector one by one, and calculate the matching similarity values between each identity feature vector and the anti-occlusion feature vector through the cosine similarity method;

[0130] S33: Sort the matching similarity values obtained in S32, select the identity feature vector with the highest matching similarity, and determine whether the similarity value is higher than a preset identity matching threshold;

[0131] S34: If the highest matching similarity value in S33 exceeds the identity matching threshold, it is determined that the match is successful, and the identity label data associated with the corresponding identity feature vector is output; if the highest matching similarity value does not exceed the identity matching threshold, it is determined that the match fails, and an unregistered person exception marking program is triggered to output temporary identity label data including a temporary number and an entry time; through the clear feature vector similarity calculation and threshold matching process, the above steps can effectively ensure the objectivity and accuracy of the identity recognition process, improve the accurate recognition efficiency of the system for the identities of conference participants under occluded face conditions, and avoid misjudgment and missed judgment phenomena.

[0132] Table 1 Example of the pre-constructed identity database of participants

[0133]

[0134] In Table 1 above, the identity number is the unique identity identifier assigned by the system for indexing database records; the name represents the real name of the participant; the position corresponds to the name of the position to which the participant belongs and is used for subsequent permission association; the feature vector encoding is the standard face feature vector extracted from the collected images during the registration stage, and the format is a 128-dimensional or 256-dimensional real number vector; the feature encryption value is the ciphertext obtained by encrypting the feature vector encoding F to ensure data security and is used for privacy protection during storage and transmission; the registration time represents the timestamp when the participant's identity information is completed and written into the database.

[0135] S4 specifically includes:

[0136] S41: Receive the identity label data or temporary identity label data output by S3, and retrieve the corresponding identity attribute information according to the identity number in the identity label data or the temporary number in the temporary identity label data;

[0137] S42: Based on the preset permission rule table in the conference management system, use the position information in the identity attribute information retrieved in S41 as an index to automatically determine the permission level of the participant;

[0138] S43: Determine whether the current participant identity data comes from temporary identity label data. If the participant data comes from temporary identity label data, an exception status flag is added to the participant; if it comes from the registered identity label data, the exception status flag is set to the default normal state;

[0139] S44: Merge the name and position information obtained in S41, the permission level obtained in S42, and the exception status identifier determined in S43 to form a structured data record, and generate composite tag information for subsequent display; the above steps ensure the accuracy and clear structure of the composite tag information of the attendees generated by the meeting management system through a clear identity information retrieval, automatic mapping of permission rules, and exception status judgment process, can identify unregistered attendees in real time, and enhance the security and reliability of on-site identity management for the meeting.

[0140] Table 2 Example of Preset Permission Rules

[0141]

[0142] In Table 2 above, the position category represents the position information marked by the attendee during registration and is used as the basis for permission grading judgment; the permission level is the permission control code defined within the meeting system, and the levels are L5 to L1 from high to low; the permission description is a detailed description of the operation capabilities of each level of permission and is used for the permission execution logic mapping of the meeting management system.

[0143] When the position of the attendee retrieved in S41 is the department head, the system will search for a matching item in this permission rule table and assign the attendee the L3 permission level, with the permission description being the permission to speak normally and participate in the topic; if it is a temporary numbered identity tag, it will default to matching the category of temporary visitor / unregistered person, be assigned the L1 permission level, and an exception status identifier will be set in the subsequent steps.

[0144] S5 specifically includes:

[0145] S51: Receive the composite tag information generated in S4, perform homomorphic encryption processing using a preset public key, encrypt each data field containing name, position, permission level, and exception status identifier item by item, and generate corresponding ciphertext;

[0146] S52: After splicing the ciphertext of each field generated in S51 in a predetermined order, add a predefined data packet header and checksum tail marker, and encapsulate it into a complete encrypted tag data packet that conforms to the parsing protocol of the meeting management terminal;

[0147] S53: Send the encrypted tag data packet encapsulated in S52 to the meeting management terminal through a preset independent communication channel. This communication channel is physically isolated from the video stream communication channel and is only used to transmit encrypted identity tag information data;

[0148] S54: After receiving the encrypted tag data packet, the conference management terminal verifies the header and tail of the data packet according to the agreed parsing protocol. After passing the verification, it enters the subsequent decryption steps. The above steps significantly improve the security and privacy protection level during the data transmission process by implementing homomorphic encryption on the composite tag information and transmitting it independently and securely, preventing information leakage and tampering, and ensuring the security and integrity of the identity data in the conference scenario.

[0149] S6 specifically includes:

[0150] S61: After receiving the encrypted tag data packet transmitted by S5, the conference management terminal homomorphically decrypts the ciphertext of each field in the data packet item by item using the pre-stored private key, and decrypts to obtain the plaintext data of the composite tag information.

[0151] S62: For the decrypted composite tag information, according to the participant's identity number or temporary number, it is automatically associated with the corresponding participant's image position coordinates in the real-time video stream of the participant. After performing image coordinate matching, the name, position, permission level, and exception status identifier are superimposed and displayed near the corresponding participant's position.

[0152] S63: Continuously monitor the position information of the participants. If it is detected that the position coordinates of the participants have changed compared to the previous moment, a position movement event is triggered, the display coordinates of the tag information in the video frame are recalculated, and the superimposed position is immediately updated.

[0153] S64: Continuously monitor the status information of the permission levels of the participants in the conference management system. If the system detects that the permission level of a participant has changed, the data content of the composite tag information is updated according to the latest permission level, and the superimposed tag displayed on the video frame is synchronously refreshed.

[0154] S65: Continuously execute S63 and S64 to ensure that the participant tag information displayed in the real-time video stream is always consistent with the actual positions and real-time permission statuses of the participants. The above steps realize the real-time, accurate, and dynamic update of the participant identity tag information through the real-time decryption of the encrypted tag data, precise video position matching, and dynamic monitoring of position movement and permission changes, improve the real-time performance and accuracy of the participant management process, avoid information delay or misaligned display, and ensure efficient and orderly on-site management.

[0155] The present invention covers any alternatives, modifications, equivalent methods, and solutions made within the essence and scope of the present invention. To enable the public to have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention. However, those skilled in the art can fully understand the present invention without these detailed descriptions. Additionally, well-known methods, processes, procedures, components, and circuits are not described in detail to avoid unnecessary confusion to the essence of the present invention.

[0156] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. An automatic marking method for intelligent conference participants based on image recognition, characterized in that, It includes the following steps: S1: Real-time obtain the facial images of the participants through multi-angle image acquisition devices deployed in the meeting scenario, and perform light compensation, pose correction, and occlusion area segmentation processing on the facial images to generate structured participant image data; S2: Extract the facial features of the participants from the structured participant image data. For the scenarios of wearing masks and glasses occlusion, adopt a local feature enhancement algorithm to extract the geometric distribution features of the eye area and the texture features of the nose bridge area, and generate an anti-occlusion feature vector; S3: Match the anti-occlusion feature vector with a pre-constructed participant identity database, which includes the encrypted biometric codes, position information, and permission tags of the registered participants; If the match is successful, output the corresponding identity tag data; If the match fails, trigger an unregistered person exception flag and generate temporary identity tag data including a temporary number and the entry time; S4: According to the identity tag data or the temporary identity tag data, associate the preset permission rules in the conference management system to generate composite tag information including name, position, permission level, and exception status identifier; S5: Perform homomorphic encryption processing on the composite tag information to generate an encrypted tag data packet; S6: In the interactive interface of the conference management terminal, decrypt the encrypted tag data packet and overlay it on the real-time video stream of the participants, and trigger the dynamic refresh and repositioning of the tag information based on the participant's position movement or permission change event.

2. The automatic marking method for intelligent conference participants based on image recognition according to claim 1, wherein The specific content of S1 includes: S11: Synchronously trigger shooting through the camera groups preset at the four corners and the central position in the meeting scenario to cover the full-angle facial area of the participants and generate an original multi-view image set; S12: Perform light intensity analysis on the original multi-view image set. For the overexposed or underexposed areas in a single-frame image, adopt a multi-frame fusion compensation algorithm to extract the optimal light pixels of each area and generate a light-balanced image; S13: Based on the light-balanced image, detect the facial key point coordinates, calculate the spatial vector angle between the tip of the nose point and the points of the two earlobes to determine the head deflection angle; when the deflection angle exceeds the preset threshold, use affine transformation to correct the facial image to the front view angle and generate a pose-normalized image; S14: Perform edge detection on the pose-normalized image, identify the contour of the mask, glasses, or hand occlusion area, and generate a repaired and filled image of the occlusion area according to the facial skin texture features that are not occluded; S15: Bind the repaired and filled image with the associated acquisition timestamp, camera position code, light compensation parameter, and pose correction parameter, and encapsulate it as structured participant image data containing multi-dimensional metadata.

3. The automatic marking method for intelligent conference participants based on image recognition according to claim 1, characterized in that The specific content of S2 includes: S21: Receive the structured participant image data generated in S1, locate the facial area in the image, and use a preset face key point detection model to mark the key points such as the corners of the eyes, the ends of the eyebrows, the center of the eyebrows, the tip of the nose, and the root of the nose, and delimit the boundary range of the eye area and the nose bridge area; S22: Calculate the geometric relationships between the key points within the delimited eye area, including the Euclidean distances and relative angles between the end of the eyebrow and the center of the eyebrow, and between the corners of the eyes, and construct the facial local geometric distribution features; S23: In the nasal bridge area, adopt the local texture gradient analysis method to analyze the gray-scale change distribution in this area, extract the gray-scale gradient information representing skin texture differences, and extract multi-scale local texture features in the way of fixed window sliding; S24: Stitch and fuse the geometric distribution features and texture features obtained in S22 and S23 to generate a one-dimensional anti-occlusion feature vector.

4. The automatic marking method for intelligent conference participants based on image recognition according to claim 3, characterized in that, The specific content of S21 includes: S211: Based on the structured participant image data, obtain the image coordinates of the outer corner points of the eyes, the end points of the eyebrows, the center point of the eyebrows, the tip point of the nose, and the root point of the nose through a pre-trained face key point detection model; S212: According to the detected key point coordinates, define the bounding box of the periorbital area. Taking the outer corner points of the left and right eyes as the center, the width of the box body is taken as 1.5 times the Euclidean distance between the two outer corner points, and the height is taken as 0.8 times the vertical distance from the center point of the eyebrows to the tip point of the nose. The center line of the box body coincides with the line connecting the two outer corner points; S213: Define the bounding box of the nasal bridge area. Taking the midpoint between the root point of the nose and the tip point of the nose as the center of the box body, the height of the box body is set as 1.2 times the Euclidean distance between the root point of the nose and the tip point of the nose, and the width of the box body is 0.6 times the horizontal distance between the two inner corner points of the eyes. The vertical center line of the box body coincides with the line connecting the root point of the nose and the tip point of the nose.

5. The automatic marking method for intelligent conference participants based on image recognition according to claim 3, characterized in that, The specific content of S22 includes: S221: Using the facial image coordinates obtained in S21, taking the center point of the eyebrows as the reference, calculate the Euclidean distances between the center point of the eyebrows and the end point of the left eyebrow, the center point of the eyebrows and the end point of the right eyebrow, and the outer corner point of the left eye and the outer corner point of the right eye respectively to obtain the characteristic lengths of the periorbital area; S222: Calculate the angles formed by connecting the center point of the eyebrows with the end points of the left and right eyebrows respectively, and the angles formed by connecting the center point of the eyebrows with the outer corner points of the left and right eyes respectively to obtain the characteristic angles of the periorbital area; S223: Normalize the Euclidean distances obtained in S221 and the angles obtained in S222; S224: Sequentially stitch the normalized characteristic length and characteristic angle to form the facial local geometric distribution feature.

6. The automatic marking method for intelligent conference participants based on image recognition according to claim 3, characterized in that, The specific content of S23 includes: S231: Extract the gray-scale image sub-region from the bounding box of the nasal bridge area delimited in S21, denoted as the area to be analyzed; S232: Taking each pixel point in the area to be analyzed as the center, calculate the gradient amplitude of the gray-scale value along the horizontal and vertical directions of the image respectively to obtain the local gradient intensity information; S233: Based on the gradient amplitude, use a square window of a preset size to slide and traverse the area to be analyzed at a fixed step length. The histogram distribution of the gradient amplitude is statistically calculated within each window to form a local texture feature histogram; S234: Change the window size, repeat S233, extract the gradient histograms respectively with different window sizes, and sequentially stitch the histogram features under different window sizes to form local texture features with multi-scale information.

7. The automatic marking method for intelligent conference participants based on image recognition according to claim 1, characterized in that The specific content of S3 includes: S31: Extract the encrypted biometric feature codes of the registered participants from the pre-constructed participant identity database, and decrypt the encrypted biometric feature codes and convert them into identity feature vectors in standard format; S32: Calculate the similarity between the anti-occlusion feature vector obtained in S2 and each identity feature vector one by one, and calculate the matching similarity values between each identity feature vector and the anti-occlusion feature vector through the cosine similarity method; S33: Sort the matching similarity values obtained in S32, select the identity feature vector with the highest matching similarity, and determine whether the similarity value is higher than the preset identity matching threshold; S34: If the highest matching similarity value in S33 exceeds the identity matching threshold, it is determined that the match is successful, and the identity label data associated with the corresponding identity feature vector is output; if the highest matching similarity value does not exceed the identity matching threshold, it is determined that the match fails, and the unregistered person exception marking program is triggered, and the temporary identity label data including the temporary number and the entry time is output.

8. The automatic marking method for intelligent conference participants based on image recognition according to claim 1, characterized in that The specific steps of S4 are as follows: S41: Receive the identity label data or temporary identity label data output by S3, and retrieve the corresponding identity attribute information according to the identity number in the identity label data or the temporary number in the temporary identity label data; S42: Based on the preset permission rule table in the conference management system, use the position information in the identity attribute information retrieved in S41 as an index to automatically determine the permission level of the attendee; S43: Determine whether the current attendee identity data comes from the temporary identity label data. If the attendee data comes from the temporary identity label data, an exception status flag is added to the attendee; if it comes from the registered identity label data, the exception status flag is set to the default normal status; S44: Merge the name and position information obtained in S41, the permission level obtained in S42, and the exception status flag determined in S43 to form a structured data record and generate composite marking information.

9. The automatic marking method for intelligent conference participants based on image recognition according to claim 1, characterized in that The specific steps of S5 are as follows: S51: Receive the composite marking information generated by S4, perform homomorphic encryption processing using the preset public key, encrypt each data field including name, position, permission level, and exception status flag item by item, and generate corresponding ciphertexts; S52: After splicing the ciphertexts of each field generated in S51 in a predetermined order, add a predefined data packet header and a checksum tail mark, and encapsulate them into a complete encrypted marking data packet that conforms to the parsing protocol of the conference management terminal; S53: Through the preset independent communication channel, send the encrypted marking data packet encapsulated in S52 to the conference management terminal using the secure communication protocol. This communication channel is physically isolated from the video stream communication channel and is only used to transmit encrypted identity marking information data; S54: After receiving the encrypted marking data packet, the conference management terminal checks the packet header and tail according to the agreed parsing protocol.

10. The automatic marking method for intelligent conference participants based on image recognition according to claim 1, characterized in that, The specific steps of S6 are as follows: S61: After receiving the encrypted marking data packet transmitted by S5, the conference management terminal performs homomorphic decryption on the ciphertexts of each field in the data packet item by item using the pre-stored private key, and decrypts to obtain the plaintext data of the composite marking information; S62: Automatically associate the decrypted composite tag information with the corresponding participant image position coordinates in the real-time video stream of the participant according to the participant identity number or temporary number. After performing image coordinate matching, display the name, position, permission level, and anomaly status identifier near the corresponding participant position; S63: Continuously monitor the position information of the participant. If it is detected that the position coordinates of the participant have changed compared to the previous moment, trigger a position movement event, recalculate the display coordinates of the tag information in the video frame, and immediately update the overlay position; S64: Continuously monitor the status information of the participant permission level in the conference management system. If the system detects a change in the participant permission level, update the data content of the composite tag information according to the latest permission level, and synchronously refresh the overlay tag displayed on the video frame; S65: Continuously execute S63 and S64 to ensure that the participant tag information displayed in the real-time video stream always remains consistent with the actual position and real-time permission status of the participant.

Images