A visual recognition-based examination administration identity verification method and system

Abstract

The application discloses a kind of based on visual identification's examination identity verification method and system, specifically related to the field of examination identity verification, for solving the problem that identity track is difficult to continuous verification after face recovery;By obtaining examination process video stream, face visible state recognition is carried out to video frame, the continuous change of stable visible, unusable, recovery visible is intercepted as breakpoint fragment group, and the boundary of breakpoint fragment group is determined;Based on boundary, breakpoint front video frame, face unusable period video frame and recovery rear video frame are divided, respectively extract breakpoint front identity feature, auxiliary visible area feature and recovery rear identity feature, and obtain breakpoint reconnection feature in time sequence association;According to breakpoint reconnection feature, face recovery misplacement and auxiliary area fracture amount are obtained, the same conflict section is found, and the face similarity of recovery rear identity feature and pre-stored examinee image is combined to form identity track discontinuous anomaly or the continuous identity verification result of examination process.

Description

Technical Field

[0001] This invention relates to the field of examination administration identity verification, and more specifically, to a method and system for examination administration identity verification based on visual recognition. Background Technology

[0002] In the current practice of verifying the identity of examinees, facial images are usually collected before the examinee enters the examination room or before the examination begins. The on-site image is then compared with the registration image or the image on the ID card to confirm whether the examinee is the person in question. With the increase in remote examinations, drone examinations, and video proctoring scenarios, identity verification no longer only occurs at the moment of entry. It is also necessary to continuously confirm that the person in the picture is still the same examinee during the examination. However, in actual examinations, examinees may look down to answer questions, briefly cover their faces, experience image lag, or have changes in camera status, which may cause the face to be temporarily unclear in the video and then reappear in the picture. This situation itself does not necessarily indicate an abnormality, but it will interrupt the originally continuous facial verification process.

[0003] Current technology often simply performs a facial similarity comparison again after a face reappears. As long as the recovered facial image is close to the pre-stored candidate image, it is easy to continue to verify the identity. However, this process does not connect the changes in the image before and after the face disappeared, and cannot confirm whether the recovered face naturally follows the same identity trajectory as before it disappeared. In other words, the system sees "whether it looks like the person" but does not determine "whether it is the same person who reappears and continues in the image". When similar people appear in the image, there are pre-prepared images, or replacements after occlusion, relying solely on the similarity of the recovered face can easily give incorrect passing results, causing a gap in identity verification during the examination process, and thus affecting the reliability and security of the examination administration's identity verification.

[0004] To address the aforementioned problems, a technical solution is provided. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a visual recognition-based examination identity verification method and system. This method involves acquiring a video stream of the examination process, identifying the visible state of faces in the video frames, and extracting continuous changes from stable visibility, unavailability, to restored visibility into breakpoint segment groups, defining the boundaries of these breakpoint segment groups. Based on these boundaries, the method divides the video frames before the breakpoint, the video frames during the period when the face is unavailable, and the video frames after restoration. Identity features before the breakpoint, auxiliary visible area features, and identity features after restoration are extracted respectively, and these are correlated chronologically to obtain breakpoint reconnection features. Based on the breakpoint reconnection features, the method obtains the face restoration misalignment amount and the auxiliary area breakage amount, identifies the same conflict segment, and combines the restored identity features with the face similarity of pre-stored candidate images to form an identity trajectory discontinuity anomaly or a continuous identity verification result for the examination process, thereby solving the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for verifying the identity of examination staff based on visual recognition, comprising:

[0008] S1. Obtain the video stream of the examination process, perform face visibility status recognition on the video frames in the video stream of the examination process, and extract the continuous changes of face visibility status in time sequence from stable visibility to unusable to restored visibility as breakpoint segment groups, and use the video frames that change from stable visibility to unusable and the video frames that change from unusable to restored visibility as the boundaries of the breakpoint segment groups.

[0009] S2. Based on the boundary of the breakpoint segment group, the breakpoint segment group is divided into video frames before the breakpoint, video frames during the period when the face is unavailable, and video frames after recovery. The identity features before the breakpoint and the identity features after recovery are extracted from the video frames before the breakpoint and the video frames after recovery. The auxiliary visible area features are extracted from the video frames during the period when the face is unavailable. The breakpoint reconnection features are obtained by associating them in chronological order.

[0010] S3. Based on the breakpoint reconnection features, obtain the face restoration misalignment amount and auxiliary region breakage amount. Search for the same conflict segment formed by the two along the time sequence of the breakpoint segment group. Combine the restored identity features with the face similarity of the pre-stored candidate image to form the identity trajectory discontinuity anomaly or the continuous identity verification result of the examination process.

[0011] Furthermore, in S1, the data processing server extracts the face region location and face region confidence results frame by frame;

[0012] Before calculating the boundary of the face region, candidate continuous face regions are formed by consecutive video frames in which face regions can be extracted in adjacent video frames and the confidence result of the face region reaches the boundary of face detection.

[0013] Then, the face region boundary is determined based on the face region boundary value corresponding to the candidate continuous face region.

[0014] When the face region coverage value of the current video frame reaches the face region coverage boundary, the face region of the current video frame is included in the same face region change sequence.

[0015] If the face region coverage value of the current video frame does not reach the face region coverage boundary, the face region of the current video frame will not be included in the same face region change sequence.

[0016] Furthermore, in S1, the data processing server marks the face visibility status of each video frame based on the face region confidence result, face region acceptance value, and face key point localization result.

[0017] When all three conditions are met, the current video frame is marked as stable and visible.

[0018] If any of the three results fails to meet the corresponding boundary, the current video frame will be marked as unusable.

[0019] When subsequent video frames that are unavailable meet the corresponding boundaries again and are included in the same face region change sequence, the subsequent video frames are marked as restored to visibility.

[0020] When the number of video frames contained in an unavailable state segment does not exceed the boundary of the number of isolated unavailable frames and the face regions on both sides can be accessed by the same face region change sequence, the unavailable state segment will be merged into the stable visible state segment.

[0021] When the above merging conditions are not met, unavailable state segments are retained to participate in the formation of breakpoint segment groups.

[0022] Furthermore, in S2, the data processing server reads the face regions from the video frames before the breakpoint and the video frames after the recovery, respectively.

[0023] Perform facial landmark localization on the read face region and output the positions of the facial landmarks corresponding to the eyes, nose and mouth;

[0024] When the facial key points are fully output, the facial region is aligned in pose and the facial feature vector is extracted.

[0025] When the facial key points are not fully output, the corresponding video frame is not written into the identity features before the breakpoint or the identity features after the recovery.

[0026] The facial feature vectors of adjacent video frames are judged for continuity. Facial feature vectors that reach the identity feature continuity boundary and satisfy the facial region continuity relationship are written into the identity feature before the breakpoint or the identity feature after recovery. Facial feature vectors that do not reach the identity feature continuity boundary are not written into the corresponding identity feature.

[0027] Furthermore, in S2, the data processing server extracts non-face-visible regions from video frames during the period when the face is unavailable. The non-face-visible regions include at least one of the collar, shoulder, desktop edge, and background edge.

[0028] Non-face visible areas are converted into grayscale images, and local image feature points are extracted using a scale-invariant feature transform algorithm.

[0029] The forward tracking position and backward lookup position of local image feature points between adjacent video frames are obtained by using the Pyramid Lucas-Cornard optical flow algorithm.

[0030] When the optical flow lookup error is not greater than the optical flow lookup boundary, the corresponding local image feature points are determined as valid moving feature points;

[0031] When the optical flow lookup error is greater than the optical flow lookup boundary, the corresponding local image feature points are not written into the auxiliary visible area features.

[0032] Furthermore, in S2, the data processing server determines the proximity relationships of image locations based on the position and scale of local image feature points;

[0033] When two local image feature points satisfy the auxiliary region proximity boundary, and both local image feature points satisfy the optical flow lookup error not greater than the optical flow lookup boundary in adjacent video frames, the two local image feature points are assigned to the same auxiliary visible region.

[0034] When two local image feature points do not satisfy the auxiliary region proximity boundary, or when the optical flow lookup error of any local image feature point is greater than the optical flow lookup boundary, the two local image feature points will not be classified into the same auxiliary visible region.

[0035] Auxiliary visible region features can be written to form an auxiliary visible region that continues the image position near the first boundary or the second boundary.

[0036] Furthermore, in S3, when the facial similarity between the restored identity features and the pre-stored candidate image passes, and the same conflict segment is not located near the second boundary, the data processing server will shift the second boundary to the side where the conflict segment first appeared.

[0037] When the facial misalignment change zone appears before the auxiliary break change zone, the second boundary is shifted to the video frame position where the facial misalignment change zone is located.

[0038] When the auxiliary fracture change zone appears before the facial misalignment change zone, the second boundary is shifted to the video frame position where the auxiliary fracture change zone is located.

[0039] S2 is re-executed after each translation of the second boundary.

[0040] When the new second boundary position is the same as the previous second boundary position, or when the new second boundary position results in the absence of a recovered video frame after the second boundary that can form the recovered identity features, the boundary translation stops and a continuous identity verification result is formed.

[0041] Furthermore, in S1, the data processing server outputs the breakpoint segment group, the face visibility status of each video frame within the breakpoint segment group, and the first and second boundaries.

[0042] In S2, the data processing server retains the temporal correspondence between the first boundary, the second boundary, the identity features before the breakpoint, the auxiliary visible area features, and the identity features after recovery in the breakpoint reconnection feature; in S3, when an identity trajectory discontinuity anomaly is formed, the identity trajectory discontinuity anomaly is written into the continuous identity verification result of the examination process.

[0043] When no discontinuity or anomaly in the identity trajectory is formed, the continuous identity verification result is written based on the facial similarity between the recovered identity features and the pre-stored candidate image.

[0044] Furthermore, in S3, the data processing server records the new position of the second boundary after each translation of the second boundary;

[0045] When the same conflict segment no longer exists in the reconnected breakpoint features, the verification result corresponding to the face similarity between the current restored identity features and the pre-stored candidate image is written into the continuous identity verification result.

[0046] When the same conflict segment that re-forms is still located near the second boundary, the face breakpoint is marked as an anomaly of discontinuity in the identity trajectory;

[0047] If the facial similarity between the restored identity features and the pre-stored candidate image fails, the facial similarity verification result will be written into the continuous identity verification result.

[0048] A visual recognition-based examination administration identity verification system includes:

[0049] The status recognition module is used to acquire the video stream of the examination process, perform face visibility status recognition on the video frames in the video stream of the examination process, obtain the stable visible, unavailable, and restored visible face visibility status corresponding to each video frame, and pass the face visibility status to the segment extraction module.

[0050] The segment extraction module is used to extract the continuous changes in the face visibility state in time sequence from stable visibility to unusable to restored visibility, based on the face visibility state transmitted by the state recognition module. These changes are then extracted into breakpoint segment groups, with the video frames from stable visibility to unusable and from unusable to restored visibility serving as the boundaries of the breakpoint segment groups. The breakpoint segment groups and their boundaries are then transmitted to the feature extraction module.

[0051] The feature extraction module is used to divide the breakpoint segment group and its boundaries into video frames before the breakpoint, video frames during the period when the face is unavailable, and video frames after recovery, based on the breakpoint segment group and its boundaries passed by the segment extraction module. It extracts the identity features before the breakpoint and the identity features after recovery from the video frames before the breakpoint and the video frames after recovery, extracts the auxiliary visible area features from the video frames during the period when the face is unavailable, and passes the identity features before the breakpoint, the identity features after recovery, and the auxiliary visible area features to the reconnection module.

[0052] The reconnection module is used to associate the identity features before the breakpoint, the identity features after the recovery, and the auxiliary visible area features transmitted by the feature extraction module in chronological order to obtain the breakpoint reconnection features, and then transmit the breakpoint reconnection features to the conflict handling module.

[0053] The conflict resolution module is used to obtain the face restoration misalignment amount and auxiliary region breakage amount based on the breakpoint reconnection features transmitted by the reconnection association module. It searches for the same conflict segment formed by the two along the time sequence of the breakpoint segment group, and combines the restored identity features with the face similarity of the pre-stored candidate image to form the identity trajectory discontinuity anomaly or the continuous identity verification result of the examination process.

[0054] The technical effects and advantages of the examination administration identity verification method and system based on visual recognition of the present invention are as follows:

[0055] This invention limits the complete video frame range from stable visibility to unusable to restored visibility of a face by using breakpoint segment groups, ensuring that brief facial occlusion, head-down answering, or blurred images are not simply equated with identity anomalies. By using breakpoint reconnection features, it places the identity features before the breakpoint, the auxiliary visible area features, and the identity features after restoration in the same temporal sequence, enabling continued verification of the continuity of the identity trajectory even after the face similarity is passed after restoration. By using the face restoration misalignment amount and the auxiliary area breakage amount to jointly find the same conflict segment, it ensures that abnormal results do not only originate from single-frame face comparisons but are bound to the video frame range where face reconnection deviation and image continuity breakage occur simultaneously. By outputting the correspondence between the same conflict segment and the restored visible position, it outputs the identity trajectory discontinuity anomaly or continuous identity verification result, providing clear video frame basis for handling face breakpoints during the examination process. Attached Figure Description

[0056] Figure 1 This is a flowchart illustrating a visual recognition-based examination administration identity verification method according to the present invention.

[0057] Figure 2 This is a schematic diagram illustrating the same conflict segment search and boundary translation processing of the present invention;

[0058] Figure 3 This is a schematic diagram of the structure of an examination administration identity verification system based on visual recognition according to the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Please see Figure 1 This invention provides a visual recognition-based method for verifying the identity of examination staff, comprising:

[0061] This invention uses the video stream of the examination process as the processing object. First, it identifies the visibility state of the face according to the time sequence of the video frames, and organizes the continuous changes of the face from stable visibility to unavailability and then back to visibility into breakpoint segment groups. At the same time, it determines a first boundary to indicate that the face begins to lose its verification value and a second boundary to indicate that the face regains its verification value. Then, based on the boundaries of the breakpoint segment groups, the breakpoint segment groups are divided into video frames before the breakpoint, video frames during the period when the face is unavailable, and video frames after recovery. The identity features before the breakpoint and the identity features after recovery are extracted from the video frames before the breakpoint and the video frames after recovery. The auxiliary visible area features are extracted from the video frames during the period when the face is unavailable, and they are associated in time sequence to form breakpoint reconnection features. On this basis, the identity features before the breakpoint and the identity features after recovery are organized into a pre-breakpoint connection zone and a post-recovery connection zone, and the auxiliary visible area features are organized into an auxiliary area connection zone. The face recovery misalignment amount and the auxiliary area break amount are further obtained, and the identity trajectory processing result at the face breakpoint is determined through the same conflict segment.

[0062] S1. Obtain the video stream of the examination process, perform face visibility status recognition on the video frames in the video stream, and extract the breakpoint segments by forming a continuous change in face visibility status from stable visibility to unusable to restored visibility in time sequence. The video frames that change from stable visibility to unusable and from unusable to restored visibility are used as the boundaries of the breakpoint segment groups; specifically:

[0063] To determine whether face regions in adjacent video frames change continuously, the data processing server calculates face region continuity values ​​for the face regions in adjacent video frames. These continuity values ​​are calculated using the intersection-over-union ratio (IoU).

[0064]

[0065] in, For the first The video frame and the first The face region continuity value between video frames is used to indicate the degree of positional continuity of face regions in adjacent video frames; For the first The location of the face region in the video frame is derived from the first... Face region extraction results for each video frame; For the first The location of the face region in the video frame is derived from the first... Face region extraction results for each video frame; Let be the overlapping area of ​​the two face regions in the image plane; This represents the total area covered by the two face regions in the image plane. This is a dimensionless result used to indicate whether the positions of face regions in adjacent video frames can be consecutively connected.

[0066] Before calculating the face region acceptance boundary, the data processing server first forms candidate continuous face regions. These candidate continuous face regions consist of consecutive video frames in which face regions can be extracted from adjacent video frames, and the face region confidence result reaches the face detection pass boundary. These candidate continuous face regions are only used to initialize the face region acceptance boundary and are not directly equivalent to a stable visible state. The face region acceptance boundary is determined by the face region acceptance value corresponding to the candidate continuous face regions. The data processing server sorts the face region acceptance values ​​corresponding to the candidate continuous face regions in ascending order and takes the quartile as the face region acceptance boundary; when... When the boundary of the face region is reached, the first The facial regions of each video frame are accessed through the same facial region change sequence; when When the boundary of the face area is not reached, the first The facial regions of each video frame are not included in the same facial region change sequence, and the first facial region change sequence is retained. The original time position of each video frame is determined and face visibility status is then recognized.

[0067] The data processing server uses video frame sequences, face region locations, face region confidence results, and the same face region change sequence as a basis to annotate the face visibility status of each video frame. The face visibility status includes stable visibility, unavailable, and restored visibility. Stable visibility means that the face region in the video frame can support identity verification and can be connected to the same face region change sequence. Unavailable means that the face region in the video frame is missing, occluded, off-center, or blurry and cannot form identity features. Restored visibility means that after being unavailable, a face region that can form identity features reappears and can be connected to the continuous change sequence again.

[0068] Stable visibility is determined based on whether the face region has continuous verification value. The data processing server reads the face region confidence result, face region acceptance value, and face key point localization result of the current video frame. When the face region confidence result reaches the face detection pass boundary, the face region acceptance value reaches the face region acceptance boundary, and the face key point localization result can output the corresponding positions of the eyes, nose, and mouth, the current video frame is marked as stable visibility. The face detection pass boundary is determined by the video frames in the initial verification segment of the examination process video stream where the face region can be extracted. The data processing server sorts the face region confidence results in the initial verification segment in ascending order of value and takes the quartile as the face detection pass boundary. When there is no initial verification segment in the examination process video stream, the data processing server uses the face region confidence result obtained by pre-stored candidate images through the same face region extraction process as the initial source of the face detection pass boundary. The face key point localization result is used to determine whether the face region has the basis for forming identity features. S1 does not extract identity features before the breakpoint or identity features after recovery.

[0069] The unusable label is based on missing face regions and loss of continuous verification value. If no face region is extracted in the current video frame, or the face region confidence result is lower than the face detection pass boundary, or the face region acceptance value does not reach the face region acceptance boundary, or the face key point localization result cannot output the corresponding positions of the eyes, nose, and mouth, the current video frame is labeled as unusable. The labeling of restored visibility occurs after unusable. The data processing server continues to read subsequent video frames along the video frame sequence. When the subsequent video frame again meets the corresponding conditions of face region confidence result, face region acceptance value, and face key point localization result, and the face regions in the subsequent adjacent video frames can be connected to the same face region change sequence, the subsequent video frame is labeled as restored visibility.

[0070] After all video frames have been labeled with the visible face status, the data processing server groups video frames with the same status and appearing consecutively into a continuous state segment according to the time sequence of the video frame sequence. The continuous state segment includes a stable visible state segment, an unusable state segment, and a restored visible state segment. The continuous frontal video frames before the examinee lowers their head to write are grouped into the stable visible state segment. The video frames in which the examinee's face is obscured by the exam paper or arm while the examinee is lowering their head are grouped into the unusable state segment. The video frames in which the examinee raises their head again and the facial area is once again of continuous verification value are grouped into the restored visible state segment.

[0071] The data processing server searches along the video frame sequence for a continuous change in time sequence, from stable visibility to unavailability to restored visibility. Isolated video frames are determined by the isolated unavailability frame count boundary, which is determined according to the frame rate of the video stream during the examination process. The calculation formula is as follows:

[0072]

[0073] in, The isolated unavailable frame number boundary is used to indicate the number of consecutive unavailable video frames that are allowed to be incorporated into a stable visible state segment; The frame rate of the exam process video stream is derived from the encoding information of the exam process video stream; To isolate unavailable verification time, it is pre-configured by the examination administration identity verification process to indicate the allowed duration of a single image shake; The number of video frames corresponding to the isolated unavailability verification time is rounded down. When the number of video frames contained in the unavailability segment does not exceed... Furthermore, if the face regions before and after the unavailable state segment can be included in the same face region change sequence, the data processing server will merge the unavailable state segment into the stable visible state segment, without forming a breakpoint segment group; when the number of video frames contained in the unavailable state segment exceeds If an unavailable state segment forms a continuous segment between a stable visible state segment and a restored visible state segment, and the face regions on both sides of the unavailable state segment cannot directly maintain the same continuous change, then the continuous changes corresponding to the stable visible state segment, the unavailable state segment, and the restored visible state segment are determined as face breakpoints.

[0074] When the data processing server extracts breakpoint segments, it retains consecutive, stable, and visible video frames before the unavailable segment and consecutive, recoverable, and visible video frames after the unavailable segment. The consecutive, stable, and visible video frames before the unavailable segment are used for the video frames before the breakpoint is formed in S2; the unavailable segment itself is used for the video frames during the period when the face is unavailable in S2; and the consecutive, recoverable and visible video frames after the unavailable segment are used for the video frames after recovery in S2. If the number of video frames that can form a continuous face region in the stable or recoverable and visible segment is insufficient, the data processing server continues to read adjacent video frames in the time sequence of the exam process video stream until the breakpoint segment group can simultaneously contain three consecutive parts: stable and visible, unavailable, and recoverable and visible.

[0075] The first boundary is determined based on video frames that transition from stable visibility to unusable status. The data processing server reads backwards from the end of the stable visibility segment. When a video frame is stable visibility and the next adjacent video frame is unusable, the next adjacent video frame is determined as the first boundary. The first boundary represents the position where the face begins to lose its verification value. The second boundary is determined based on video frames that transition from unusable to re-visible status. The data processing server reads backwards from the end of the unusable status segment. When a video frame is unusable and the next adjacent video frame is re-visible, the next adjacent video frame is determined as the second boundary. The second boundary represents the position where the face regains its verification value. The data processing server outputs the breakpoint segment group, the face visibility status of each video frame in the breakpoint segment group, the first boundary, and the second boundary. S2 forms the pre-breakpoint video frame based on the stable visibility video frames before the first boundary, forms the video frames during the face unusable period based on the unusable video frames between the first and second boundaries, and forms the post-recovery video frame based on the re-visible video frames after the second boundary.

[0076] S2. Based on the boundaries of the breakpoint segment group, the breakpoint segment group is divided into video frames before the breakpoint, video frames during the period when the face is unavailable, and video frames after recovery. Identity features before and after the breakpoint are extracted from the video frames before and after the breakpoint. Auxiliary visible area features are extracted from the video frames during the period when the face is unavailable. These are then correlated in chronological order to obtain the breakpoint reconnection features. Specifically:

[0077] In the specific division, the data processing server reads sequentially from the starting video frame of the breakpoint segment group. Video frames located before the first boundary and in a stable visible state are written into the pre-breakpoint video frames. Video frames located between the first and second boundaries and in an unusable visible state are written into the video frames during the period when the face is unusable. Video frames located after the second boundary and in a revived visible state are written into the revived video frames. The first boundary corresponds to the position where the face begins to lose its verification value, and the video frames corresponding to the first boundary are written into the video frames during the period when the face is unusable. The second boundary corresponds to the position where the face regains its verification value, and the video frames corresponding to the second boundary are written into the revived video frames. This ensures that the range of the three types of video frames is consistent with the position of the change in the visible state of the face in S1.

[0078] The data processing server reads the face regions from the video frames before and after the breakpoint, and extracts the identity features before and after the breakpoint using the same face processing method. The identity features before the breakpoint are used to express the identity state before the face becomes unusable, and the identity features after the face is restored to have verification value. Both are composed of face feature vectors in consecutive video frames in chronological order, rather than being determined by a single video frame.

[0079] For each video frame before the breakpoint, the data processing server first reads the face region in each frame, then performs facial landmark localization on the face region, outputting the positions of the facial landmarks corresponding to the eyes, nose, and mouth. When the facial landmark positions can be output completely, the data processing server performs pose alignment on the face region based on the facial landmark positions. The pose alignment uses a similarity transformation, which is performed as follows:

[0080]

[0081] in, For the first The location of individual facial landmarks within the standard face region; For the first The location of individual facial landmarks within the face region of the current video frame; This is a scaling factor used to make the spacing between facial key points in the current video frame consistent with the preset standard key point spacing. This is a rotation matrix used to align the orientation of facial key points in the current video frame with the orientation of preset standard key points. This is a translation vector used to move the facial key points in the current video frame to the preset standard key point positions; , and The facial key points are determined based on the minimum positional deviation between the corresponding facial key points of the eyes, nose, and mouth and the preset standard key points. Subsequently, the data processing server inputs the aligned facial region into the convolutional neural network facial feature extraction algorithm and outputs the facial feature vector of the corresponding video frame. If the complete facial key point position cannot be output in the video frame before a certain breakpoint, the identity feature before the breakpoint is not written into the corresponding video frame.

[0082] Whether facial feature vectors in video frames before adjacent breakpoints have entered the same identity change process is determined by facial feature continuity value, which is calculated using cosine similarity.

[0083]

[0084] in, The first frame in the video before the breakpoint Personal facial feature vector and the first The face feature continuity value between individual face feature vectors is used to indicate whether the identity features in adjacent video frames before the breakpoint are continuous; The first frame in the video before the breakpoint The individual facial feature vector is derived from the convolutional neural network facial feature extraction algorithm; The first frame in the video before the breakpoint Personal facial feature vectors, source and Consistent; It is the inner product of two facial feature vectors; and These are the magnitudes of the two facial feature vectors, respectively; The result is dimensionless.

[0085] The identity feature acceptance boundary is determined by the acceptance values ​​of continuously and stably visible facial features in the video frames before the breakpoint. The data processing server sorts the continuously and stably visible facial feature acceptance values ​​in the video frames before the breakpoint in ascending order and takes the quartiles as the identity feature acceptance boundary; when When the identity feature boundary is reached, and the face region positions in the corresponding adjacent video frames satisfy the face region continuation relationship, the first... Personal facial feature vectors are written into identity features before the breakpoint; when If the boundary for identity feature acceptance is not reached, or the position of the face region in the corresponding adjacent video frame does not satisfy the face region acceptance relationship, then the first... Since the individual facial feature vector is a non-continuous facial feature vector, it is not written into the identity features before the breakpoint.

[0086] For the recovered video frames, the data processing server uses the same facial landmark localization, pose alignment, and convolutional neural network facial feature extraction algorithms as the video frames before the breakpoint to obtain the facial feature vectors corresponding to the recovered video frames, and performs continuation judgment according to the chronological order of the recovered video frames; the continuation values ​​of facial features in the recovered video frames are represented by independent symbols. It is expressed and calculated according to the following formula:

[0087]

[0088] in, For the recovered video frame Personal facial feature vector and the first The face feature continuity value between individual face feature vectors is used to indicate whether the identity features in adjacent video frames are continuous after restoration; For the recovered video frame The individual facial feature vector is derived from the convolutional neural network facial feature extraction algorithm; For the recovered video frame Personal facial feature vectors, source and Consistent; It is the inner product of two facial feature vectors; and These are the magnitudes of two facial feature vectors; when When the identity feature boundary is reached and the face region connection relationship is satisfied, the corresponding face feature vector is written into the restored identity feature. If exposure recovery, focus change or brief blur occurs after the second boundary, the data processing server retains the original time position of these video frames, but only writes the face feature vectors that can continuously output the complete face key point position, can complete the face region pose alignment, and can reach the identity feature boundary into the restored identity feature. The restored identity feature thus represents the continuous identity state that can be verified after the face reappears.

[0089] The data processing server continues to read video frames during the period when the face is unavailable and extracts auxiliary visible area features from the video frames during the period when the face is unavailable. The auxiliary visible area is the non-face visible area in the picture that can still be tracked in chronological order during the period when the face is unavailable. The non-face visible area includes areas that appear continuously in the picture, such as collar, shoulder, table edge, and background edge. The auxiliary visible area features are used to describe the picture transition process during the period when the face is unavailable and are sorted by S3 into an auxiliary area transition band.

[0090] In the specific extraction process, the data processing server first converts the video frames during the period when faces are unavailable into grayscale images. Then, it performs a scale-invariant feature transformation algorithm on the non-face-visible areas in the grayscale images, outputting local image feature points. Subsequently, the data processing server inputs the local image feature points from adjacent video frames into the Pyramid Lucas-Cannard optical flow algorithm to obtain the forward tracking position and backward lookup position of the local image feature points between adjacent video frames, and calculates the optical flow lookup error accordingly.

[0091]

[0092] in, During the period when the face is unavailable The first video frame The optical flow back lookup error of a local image feature point is used to determine whether the movement correspondence of the local image feature point between adjacent video frames is valid. During the period when the face is unavailable The first video frame The location of local image feature points is derived from the scale-invariant feature transformation algorithm; For the first The local image feature point from the th The first video frame is traced forward by optical flow to the next adjacent video frame, and then traced back to the first video frame by reverse optical flow. The position obtained after one video frame; The Euclidean distance on the image plane; The dimension of is pixel distance.

[0093] The optical flow regression boundary is determined by the optical flow regression error corresponding to the non-face-visible areas that can occur consecutively during the face unavailability period in the breakpoint segment group. The data processing server sorts these optical flow regression errors in ascending order of value and takes the upper quartile as the optical flow regression boundary; when When the optical flow is not greater than the optical flow lookup boundary, the first A number of local image feature points are identified as effective motion feature points. Effective motion feature points that appear consecutively in multiple adjacent video frames are grouped into the same auxiliary visible region according to their image location proximity. The image location proximity is determined based on the distance between local image feature points and the scale of the local image feature points, calculated as follows:

[0094]

[0095] in, During the period when the face is unavailable The first video frame The local image feature point and the first Proximity determination values ​​between local image feature points; For the first The first video frame The location of a local image feature point; For the first The first video frame The location of a local image feature point; For the first The scale of local image feature points is derived from the scale-invariant feature transformation algorithm; For the first The scale of a local image feature point, its origin and Consistent. When If the optical flow lookup error of two local image feature points is not greater than the adjacent boundary of the auxiliary region, and both local image feature points satisfy the condition that the optical flow lookup error is not greater than the optical flow lookup boundary in adjacent video frames, then the two local image feature points are classified into the same auxiliary visible region; the adjacent boundary of the auxiliary region is obtained by taking the upper quartile of the adjacent judgment values ​​of consecutive local image feature points that appear during the face unavailability period; when When the optical flow is greater than the optical flow lookup boundary, the first Local image feature points are not written into the auxiliary visible region features.

[0096] The data processing server further performs a position lookup of the auxiliary visible area to the video frames before the breakpoint near the first boundary and the recovered video frames near the second boundary. The boundary area is determined by the number of boundary lookup frames, which is based on the number of conflicting adjacent frames. The area near the first boundary is the continuous area before the first boundary. The range that a stable, visible video frame can cover is defined by the area near the second boundary, which is the continuous range after the second boundary. The range that can be covered by the recoverable visible video frames; the positional continuity is established if the effective moving feature points in the auxiliary visible area can be back-checked frame by frame along the adjacent video frames to the vicinity of the first boundary or the second boundary, and each back-check satisfies that the optical flow back-check error is not greater than the optical flow back-check boundary; the auxiliary visible area that can form image positional continuity near the boundary is written into the auxiliary visible area feature, and the transient visible area that cannot form a connection with the video frame near the boundary is not written into the auxiliary visible area feature; the face is unusable when the examinee looks down, but the collar edge and the desktop edge still appear continuously in the picture. The data processing server extracts local image feature points from these non-face visible areas and confirms the movement correspondence of local image feature points between adjacent video frames through the pyramid Lucas-Cannard optical flow algorithm. The collar edge and desktop edge that can extend from the occlusion period to the vicinity of the first boundary or the second boundary are written into the auxiliary visible area feature. The hand afterimage that briefly enters the picture, the compressed noise, or the local image feature points that cannot be back-checked in the adjacent video frames are not written into the auxiliary visible area feature.

[0097] After the identity features before the breakpoint, the identity features after recovery, and the auxiliary visible area features are all formed, the data processing server performs temporal association according to the original time positions of the video frames in the breakpoint segment group. The video frame positions before the breakpoint before the first boundary are written with the identity features before the breakpoint. The video frame positions during the period when the face is unavailable between the first and second boundaries are written with the auxiliary visible area features. The video frame positions after the second boundary after recovery are written with the identity features after recovery. After the association is completed, the breakpoint reconnection feature is formed. The breakpoint reconnection feature retains the temporal correspondence between the first boundary, the second boundary, the identity features before the breakpoint, the auxiliary visible area features, and the identity features after recovery. S3 can organize the pre-breakpoint connection band, the post-recovery connection band, and the auxiliary area connection band according to the breakpoint reconnection feature.

[0098] S3. Based on the breakpoint reconnection features, obtain the face restoration misalignment amount and auxiliary region breakage amount. Search for the same conflict segment formed by these two segments along the time sequence of the breakpoint segment group. Combine this with the restored identity features and the face similarity of the pre-stored candidate image to form a result indicating discontinuous identity trajectory or continuous identity verification during the examination process. Specifically:

[0099] The calculation of face restoration misalignment first generates a face continuity change result, and then performs a re-connection comparison. The face continuity change result is jointly formed by the difference in face feature vectors between adjacent video frames, the displacement of the face region center, and the pose changes of face key points. The face continuity change result is determined according to the maximum value among the three types of normalized changes between adjacent video frames, and the calculation formula is as follows:

[0100]

[0101] in, For the first The facial transition results of each video frame relative to the previous adjacent video frame; For the first The difference in facial feature vectors between a video frame and its previous adjacent video frame is obtained by subtracting the cosine similarity of the adjacent facial feature vectors from 1. For the first The displacement of the center of the face region in each video frame relative to the previous adjacent video frame is obtained by dividing the Euclidean distance between the center points of the two face regions by the length of the diagonal of the face region in the previous adjacent video frame. For the first The facial landmark pose change of each video frame relative to the previous adjacent video frame is obtained by dividing the maximum positional offset of the facial landmarks corresponding to the eyes, nose, and mouth after pose alignment by the length of the diagonal of the standard face region. The data processing server reads consecutive video frames near the first boundary in the pre-breakpoint continuation band to form the facial continuation change result at the end of the pre-breakpoint continuation band, and reads consecutive video frames near the second boundary in the post-restoration band to form the facial continuation change result at the beginning of the post-restoration band. The facial continuation change result at the beginning of the post-restoration band is then compared with the facial continuation change result at the end of the pre-breakpoint continuation band.

[0102] To ensure the reconnection deviates from the executable computational boundary, the data processing server sorts the facial reconnection changes between adjacent video frames within the reconnection band before the breakpoint in ascending order of numerical values, and takes the upper quartile as the natural range of variation; the facial restoration misalignment is calculated according to the following formula:

[0103]

[0104] in, To restore the first of the bearing belt The face recovery misalignment amount corresponding to each initial video frame is used to represent the degree of reconnection deviation of the recovered reconnection band relative to the reconnection band before the breakpoint. To restore the first of the bearing belt The facial transitions corresponding to each initial video frame, according to... The calculation method is used to obtain it; To determine the facial transition at the end of the transition band near the first boundary before the breakpoint, the nearest adjacent video frame to the first boundary in the transition band before the breakpoint is selected. ; The range of natural variation originates from the facial transition changes between adjacent video frames within the transition band before the breakpoint; both the numerator and denominator of the formula represent the amplitude of the facial transition changes. The result is dimensionless.

[0105] when When the value is greater than 1, the initial change of the resume band at the corresponding video frame position after recovery exceeds the natural change range of the resume band itself before the breakpoint, and the data processing server writes the corresponding video frame position into the face misalignment change band; when When the value is not greater than 1, the corresponding video frame position is not written into the facial misalignment change band; if the initial stage of the restored band is affected by exposure recovery, focus change, or slight motion blur, it will cause facial misalignment. It briefly exceeds 1, but in subsequent video frames... If the value decreases sequentially to no greater than 1, the corresponding video frame range is considered as the initial stage of image restoration and is not included in the facial misalignment change band; if If the number of consecutive frames in the initial stage of the restored transition zone is greater than 1 and cannot fall back to the natural change range of the transition zone before the breakpoint in chronological order, then the corresponding continuous video frame range is determined as the facial misalignment change zone. If the candidate lowers their head to cover their face and gradually moves downwards, and the face continues to appear from the same position and posture after restoration, then no facial misalignment change zone is formed. If the face suddenly appears from a different position in the frame after restoration and the changes in facial features cannot be reconnected along the change at the end of the transition before the occlusion, then the corresponding restored video frame is written into the facial misalignment change zone.

[0106] The data processing server continues to read the auxiliary visible region features from the breakpoint reconnection features, with the processing starting point being the video frame position during the period when the face was unavailable, between the first and second boundaries. The data processing server connects the auxiliary visible region features according to the time sequence of the video frames during the period when the face was unavailable, and sequentially connects the auxiliary visible regions in adjacent video frames that can form a movement correspondence through effective movement of feature points, forming an auxiliary region continuity band. The auxiliary region continuity band represents the changes in the screen area that can still be tracked during the period when the face was unavailable, and is subsequently used to determine whether the screen continuity near the restored visible position is broken.

[0107] After the auxiliary region continuation band is formed, the data processing server reads the continuity of the auxiliary visible region features near the second boundary and checks whether the auxiliary region continuation band can extend continuously from the period when the face is unavailable to the video frame where it is restored to visibility. When the auxiliary region continuation band is divided into two or more image regions that cannot be connected together near the video frame where it is restored to visibility, a split band is formed in the corresponding video frame range. When the auxiliary region continuation band loses effective moving feature point continuity near the video frame where it is restored to visibility and cannot be connected to the corresponding image position in the restored video frame, a break segment is formed in the corresponding video frame range. Both the split band and the break segment are written into the auxiliary break change band.

[0108] The fracture amount in the auxiliary region is calculated using the continuity ratio of the auxiliary region, which is formed by the continuity relationship of the number of effective moving feature points.

[0109]

[0110] in, During the period when the face is unavailable The video frame to the The length of the auxiliary region between video frames is used to indicate the degree of continuity of the auxiliary region between adjacent video frames. For from the first The successful movement of the video frame corresponds to the [number]th video frame. The number of effective moving feature points per video frame is derived from the auxiliary visible region feature extraction process in S2. For the first The number of valid moving feature points contained in the auxiliary region band of each video frame; This is a proportional value, dimensionless.

[0111] The data processing server sorts the continuity ratios of the auxiliary regions that have been continuously covered during the face unavailability period in ascending order of numerical value, takes the quartile as the boundary of the auxiliary region coverage, and then calculates the auxiliary region breakage amount according to the following formula:

[0112]

[0113] in, For the first The amount of auxiliary region breakage corresponding to each video frame is used to represent the auxiliary region bearing band in the [number]th video frame. The degree of continuity breakage at each video frame position; The auxiliary region boundary is derived from the lower quartile of the auxiliary region's continuity ratio during periods when the face is unavailable. For the first The video frame to the The continuity ratio of the auxiliary area between video frames; when Below hour, If the value is positive, an auxiliary break change band is written at the corresponding video frame position; when... achieve At that time, the corresponding video frame position is not written into the auxiliary break change band; both the numerator and denominator in the formula are proportional values, therefore The result is dimensionless.

[0114] While the candidate's face is unavailable, the edges of the collar and the desktop continue to move continuously in the image. The proportion of the auxiliary area remains above the boundary of the auxiliary area, and no auxiliary break change zone is formed. If the collar edge suddenly disappears near the visible video frame, the edge of the desktop cannot be reconnected to the image position after the second boundary, or the originally single collar edge splits into two image areas that cannot be reconnected at the visible position, the data processing server writes the corresponding video frame range into the auxiliary break change zone and forms the auxiliary area break amount according to the proportion of the auxiliary area and the boundary of the auxiliary area.

[0115] The data processing server maps the facial misalignment change zone and the auxiliary break change zone back to the original video frame positions in the breakpoint segment group, respectively. The mapping process is based solely on the time order of the breakpoint segment group. The facial misalignment change zone corresponds to the video frame range in the restored continuation zone, and the auxiliary break change zone corresponds to the video frame range in the auxiliary region continuation zone. Once both fall into the same breakpoint segment group's time frame, the data processing server begins to search for the same conflict segment.

[0116] like Figure 2 As shown, the same conflict segment is determined based on the positional relationship of video frames between the facial misalignment change zone and the auxiliary fracture change zone, and corresponds to the processing branch near the second boundary. The same conflict segment is determined based on the positional relationship of video frames between the facial misalignment change zone and the auxiliary fracture change zone; when the facial misalignment change zone and the auxiliary fracture change zone have overlapping video frames, the overlapping video frame range is determined as the same conflict segment; when there is no video frame overlap, but the number of video frames between them does not exceed the number of adjacent conflict frames, and they are both located within the recovery process range near the second boundary, the facial misalignment change zone, the auxiliary fracture change zone, and the video frames between them are collectively determined as the same conflict segment; the number of adjacent conflict frames is determined according to the following formula:

[0117]

[0118] in, The number of conflicting adjacent frames is used to determine whether the facial misalignment change zone and the auxiliary fracture change zone belong to the same recovery process; The frame rate of the exam process video stream is derived from the encoding information of the exam process video stream; The adjacent time of the recovery process is pre-configured by the examination administration identity verification process and is used to represent the maximum time interval during which facial deviation and auxiliary fractures are allowed to exist within the same recovery process. The number of video frames corresponding to the adjacent time in the recovery process is rounded down; Used to ensure that the number of conflicting adjacent frames is at least 1.

[0119] After a conflict segment is formed, the data processing server will perform a face similarity verification between the restored identity features and the pre-stored candidate image. The face feature vector in the restored identity features and the face feature vector corresponding to the pre-stored candidate image are obtained using the same convolutional neural network face feature extraction algorithm. The face similarity calculation formula is as follows:

[0120]

[0121] in, The facial similarity between the restored identity features and the pre-stored candidate image is used to determine whether the restored identity features pass the static identity verification. The facial feature vector corresponding to the restored identity features is determined by the median feature vector of the continuous facial feature vectors in the restored continuum. The facial feature vector corresponding to the pre-stored candidate image is derived from the result of the pre-stored candidate image being processed by the same convolutional neural network facial feature extraction algorithm; It is the inner product of two facial feature vectors; and These are the magnitudes of the two facial feature vectors, respectively; For dimensionless results; when When the facial similarity reaches the boundary, the facial similarity between the restored identity features and the pre-stored candidate image passes the test. The facial similarity passing boundary adopts the existing passing boundary used in the examination identity verification of the pre-stored candidate image.

[0122] The second boundary is near the number of conflicting adjacent frames. It is determined that when the number of video frames between any video frame in the same conflict segment and the corresponding video frame at the second boundary does not exceed [a certain threshold], [the following conditions apply]. When the number of video frames between all video frames in the same conflict segment and the corresponding video frame at the second boundary is greater than 1, it is determined that the same conflict segment is located near the second boundary; when the number of video frames between all video frames in the same conflict segment and the corresponding video frame at the second boundary is greater than 1, it is determined that the same conflict segment is located near the second boundary. If the same conflict segment does not fall near the second boundary, the data processing server will mark the corresponding face breakpoint as an identity trajectory discontinuity anomaly and write it into the continuous identity verification result of the examination process. If the recovered identity feature passes the face similarity test with the pre-stored candidate image, but the same conflict segment does not fall near the second boundary, the data processing server will shift the second boundary to the side where the conflict segment first appears. If the facial misalignment change band appears before the auxiliary break change band, the second boundary will be shifted to the video frame position where the facial misalignment change band is located. If the auxiliary break change band appears before the facial misalignment change band, the second boundary will be shifted to the video frame position where the auxiliary break change band is located. The first boundary remains unchanged. After the boundary shift is completed, S2 will be re-executed. After each second boundary shift, the data processing server will record the new second boundary position. When the new second boundary position is the same as the previous second boundary position, or when the new second boundary position results in no recovered video frame after the second boundary that can form the recovered identity features, the data processing server stops boundary translation and forms a continuous identity verification result based on the facial similarity between the current recovered identity features and the pre-stored candidate image. After re-forming the identity features before the breakpoint, the recovered identity features, the auxiliary visible area features, and the breakpoint reconnection features, S3 is executed again. When the same conflict segment no longer exists in the re-formed breakpoint reconnection features, a continuous identity verification result is formed based on the facial similarity between the recovered identity features and the pre-stored candidate image. When the same conflict segment is still located near the second boundary, the facial breakpoint is marked as an identity trajectory discontinuity anomaly. When the facial similarity between the recovered identity features and the pre-stored candidate image fails, a continuous identity verification result is formed directly based on the facial similarity verification result.

[0123] Please see Figure 3 This invention provides a visual recognition-based examination administration identity verification system, comprising:

[0124] The status recognition module is used to acquire the video stream of the examination process, perform face visibility status recognition on the video frames in the video stream of the examination process, obtain the stable visible, unavailable, and restored visible face visibility status corresponding to each video frame, and pass the face visibility status to the segment extraction module.

[0125] The segment extraction module is used to extract the continuous changes in the face visibility state in time sequence from stable visibility to unusable to restored visibility, based on the face visibility state transmitted by the state recognition module. These changes are then extracted into breakpoint segment groups, with the video frames from stable visibility to unusable and from unusable to restored visibility serving as the boundaries of the breakpoint segment groups. The breakpoint segment groups and their boundaries are then transmitted to the feature extraction module.

[0126] The feature extraction module is used to divide the breakpoint segment group and its boundaries into video frames before the breakpoint, video frames during the period when the face is unavailable, and video frames after recovery, based on the breakpoint segment group and its boundaries passed by the segment extraction module. It extracts the identity features before the breakpoint and the identity features after recovery from the video frames before the breakpoint and the video frames after recovery, extracts the auxiliary visible area features from the video frames during the period when the face is unavailable, and passes the identity features before the breakpoint, the identity features after recovery, and the auxiliary visible area features to the reconnection module.

[0127] The reconnection module is used to associate the identity features before the breakpoint, the identity features after the recovery, and the auxiliary visible area features transmitted by the feature extraction module in chronological order to obtain the breakpoint reconnection features, and then transmit the breakpoint reconnection features to the conflict handling module.

[0128] The conflict resolution module is used to obtain the face restoration misalignment amount and auxiliary region breakage amount based on the breakpoint reconnection features transmitted by the reconnection association module. It searches for the same conflict segment formed by the two along the time sequence of the breakpoint segment group, and combines the restored identity features with the face similarity of the pre-stored candidate image to form the identity trajectory discontinuity anomaly or the continuous identity verification result of the examination process.

[0129] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for verifying the identity of examination staff based on visual recognition, characterized in that, include: S1. Obtain the video stream of the examination process, perform face visibility status recognition on the video frames in the video stream of the examination process, and extract the continuous changes of face visibility status in time sequence from stable visibility to unusable to restored visibility as breakpoint segment groups, and use the video frames that change from stable visibility to unusable and the video frames that change from unusable to restored visibility as the boundaries of the breakpoint segment groups. S2. Based on the boundary of the breakpoint segment group, the breakpoint segment group is divided into video frames before the breakpoint, video frames during the period when the face is unavailable, and video frames after recovery. The identity features before the breakpoint and the identity features after recovery are extracted from the video frames before the breakpoint and the video frames after recovery. The auxiliary visible area features are extracted from the video frames during the period when the face is unavailable. The breakpoint reconnection features are obtained by associating them in chronological order. S3. Based on the breakpoint reconnection features, obtain the face restoration misalignment amount and auxiliary region breakage amount. Search for the same conflict segment formed by the two along the time sequence of the breakpoint segment group. Combine the restored identity features with the face similarity of the pre-stored candidate image to form the identity trajectory discontinuity anomaly or the continuous identity verification result of the examination process.

2. The examination administration identity verification method based on visual recognition according to claim 1, characterized in that, In S1, the data processing server extracts the face region location and face region confidence results frame by frame; Before calculating the boundary of the face region, candidate continuous face regions are formed by consecutive video frames in which face regions can be extracted in adjacent video frames and the confidence result of the face region reaches the boundary of face detection. Then, the face region boundary is determined based on the face region boundary value corresponding to the candidate continuous face region. When the face region coverage value of the current video frame reaches the face region coverage boundary, the face region of the current video frame is included in the same face region change sequence. If the face region coverage value of the current video frame does not reach the face region coverage boundary, the face region of the current video frame will not be included in the same face region change sequence.

3. The examination administration identity verification method based on visual recognition according to claim 2, characterized in that, In S1, the data processing server marks the face visibility status of each video frame based on the face region confidence result, face region acceptance value, and face key point localization result; When all three conditions are met, the current video frame is marked as stable and visible. If any of the three results fails to meet the corresponding boundary, the current video frame will be marked as unusable. When subsequent video frames that are unavailable meet the corresponding boundaries again and are included in the same face region change sequence, the subsequent video frames are marked as restored to visibility. When the number of video frames contained in the unavailable state segment does not exceed the boundary of the number of isolated unavailable frames and the face regions on both sides can be accessed by the same face region change sequence, the unavailable state segment will be merged into the stable visible state segment. When the above merging conditions are not met, unavailable state segments are retained to participate in the formation of breakpoint segment groups.

4. The examination administration identity verification method based on visual recognition according to claim 1, characterized in that, In S2, the data processing server reads the face regions from the video frames before the breakpoint and the video frames after the recovery, respectively. Perform facial landmark localization on the read face region and output the positions of the facial landmarks corresponding to the eyes, nose and mouth; When the facial key points are fully output, the facial region is aligned in pose and the facial feature vector is extracted. When the facial key points are not fully output, the corresponding video frame is not written into the identity features before the breakpoint or the identity features after the recovery. The facial feature vectors of adjacent video frames are judged for continuity. Facial feature vectors that reach the identity feature continuity boundary and satisfy the facial region continuity relationship are written into the identity feature before the breakpoint or the identity feature after recovery. Facial feature vectors that do not reach the identity feature continuity boundary are not written into the corresponding identity feature.

5. The examination administration identity verification method based on visual recognition according to claim 1, characterized in that, In S2, the data processing server extracts non-face-visible regions from video frames during the period when the face is unavailable. The non-face-visible regions include at least one of the collar, shoulder, desktop edge, and background edge. Non-face visible areas are converted into grayscale images, and local image feature points are extracted using a scale-invariant feature transform algorithm. The forward tracking position and backward lookup position of local image feature points between adjacent video frames are obtained by using the Pyramid Lucas-Cornard optical flow algorithm. When the optical flow lookup error is not greater than the optical flow lookup boundary, the corresponding local image feature points are determined as valid moving feature points; When the optical flow lookup error is greater than the optical flow lookup boundary, the corresponding local image feature points are not written into the auxiliary visible area features.

6. The examination administration identity verification method based on visual recognition according to claim 5, characterized in that, In S2, the data processing server determines the proximity relationships of image locations based on the position and scale of local image feature points; When two local image feature points satisfy the auxiliary region proximity boundary, and both local image feature points satisfy the optical flow lookup error not greater than the optical flow lookup boundary in adjacent video frames, the two local image feature points are assigned to the same auxiliary visible region. When two local image feature points do not satisfy the auxiliary region proximity boundary, or when the optical flow lookup error of any local image feature point is greater than the optical flow lookup boundary, the two local image feature points will not be classified into the same auxiliary visible region. Auxiliary visible region features can be written to form an auxiliary visible region that continues the image position near the first boundary or the second boundary.

7. The examination administration identity verification method based on visual recognition according to claim 1, characterized in that, In S3, when the facial similarity between the restored identity features and the pre-stored candidate image passes, and the same conflict segment is not located near the second boundary, the data processing server will shift the second boundary to the side where the conflict segment first appeared. When the facial misalignment change zone appears before the auxiliary break change zone, the second boundary is shifted to the video frame position where the facial misalignment change zone is located. When the auxiliary fracture change zone appears before the facial misalignment change zone, the second boundary is shifted to the video frame position where the auxiliary fracture change zone is located. S2 is re-executed after each translation of the second boundary. When the new second boundary position is the same as the previous second boundary position, or when the new second boundary position results in the absence of a recovered video frame that can form the recovered identity features after the second boundary, the boundary translation stops and a continuous identity verification result is formed.

8. The examination administration identity verification method based on visual recognition according to claim 1, characterized in that, In S1, the data processing server outputs the breakpoint segment group, the face visibility status of each video frame within the breakpoint segment group, and the first and second boundaries. In S2, the data processing server retains the temporal correspondence between the first boundary, the second boundary, the identity features before the breakpoint, the auxiliary visible area features, and the identity features after recovery in the breakpoint reconnection feature; in S3, when an identity trajectory discontinuity anomaly is formed, the identity trajectory discontinuity anomaly is written into the continuous identity verification result of the examination process. When no discontinuity or anomaly in the identity trajectory is formed, the continuous identity verification result is written based on the facial similarity between the recovered identity features and the pre-stored candidate image.

9. The examination administration identity verification method based on visual recognition according to claim 7, characterized in that, In S3, the data processing server records the new position of the second boundary after each translation of the second boundary. When the same conflict segment no longer exists in the reconnected breakpoint features, the verification result corresponding to the face similarity between the current restored identity features and the pre-stored candidate image is written into the continuous identity verification result. When the same conflict segment that re-forms is still located near the second boundary, the face breakpoint is marked as an anomaly of discontinuity in the identity trajectory; If the facial similarity between the restored identity features and the pre-stored candidate image fails, the facial similarity verification result will be written into the continuous identity verification result.

10. A visual recognition-based examination administration identity verification system, used to implement the visual recognition-based examination administration identity verification method according to any one of claims 1-9, characterized in that, include: The status recognition module is used to acquire the video stream of the examination process, perform face visibility status recognition on the video frames in the video stream of the examination process, obtain the stable visible, unavailable, and restored visible face visibility status corresponding to each video frame, and pass the face visibility status to the segment extraction module. The segment extraction module is used to extract the continuous changes in the face visibility state in time sequence from stable visibility to unusable to restored visibility, based on the face visibility state transmitted by the state recognition module. These changes are then extracted into breakpoint segment groups, with the video frames from stable visibility to unusable and from unusable to restored visibility serving as the boundaries of the breakpoint segment groups. The breakpoint segment groups and their boundaries are then transmitted to the feature extraction module. The feature extraction module is used to divide the breakpoint segment group and its boundaries into video frames before the breakpoint, video frames during the period when the face is unavailable, and video frames after recovery, based on the breakpoint segment group and its boundaries passed by the segment extraction module. It extracts the identity features before the breakpoint and the identity features after recovery from the video frames before the breakpoint and the video frames after recovery, extracts the auxiliary visible area features from the video frames during the period when the face is unavailable, and passes the identity features before the breakpoint, the identity features after recovery, and the auxiliary visible area features to the reconnection module. The reconnection module is used to associate the identity features before the breakpoint, the identity features after the recovery, and the auxiliary visible area features transmitted by the feature extraction module in chronological order to obtain the breakpoint reconnection features, and then transmit the breakpoint reconnection features to the conflict handling module. The conflict resolution module is used to obtain the face restoration misalignment amount and auxiliary region breakage amount based on the breakpoint reconnection features transmitted by the reconnection association module. It searches for the same conflict segment formed by the two along the time sequence of the breakpoint segment group, and combines the restored identity features with the face similarity of the pre-stored candidate image to form the identity trajectory discontinuity anomaly or the continuous identity verification result of the examination process.

Images