Online interview large model cheating detection system and method based on eye movement tracking

Abstract

The invention relates to the technical field of large model cheating detection, in particular to an online interview large model cheating detection system and method based on eye movement tracking, and the detection method comprises the steps: S1, system initialization and environment adaptation; s2, eye movement tracking calibration and baseline establishment; s3, collecting and preprocessing real-time eye movement data; s4, executing an anomaly detection algorithm; and S5, carrying out risk assessment and outputting a result. According to the scheme, based on the WebGazer.js technology, the mapping relation between the fixation point and the screen coordinates is established through nine-point initialization calibration, and the personalized coefficient k is generated in combination with the pupil diameter and the facial features of the candidate, so that the eye movement tracking error is greatly reduced, and the large model cheating detection precision requirement can be met without special hardware. The design not only reduces the deployment cost, but also breaks through the hardware limitation, so that the technology can adapt to mainstream notebook equipment, is convenient for large-scale popularization and application, and solves the contradiction that high precision and low cost cannot be achieved at the same time in the traditional scheme.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large model cheating detection, in particular to an online interview large model cheating detection system and method based on eye tracking. BACKGROUND

[0002] With the popularization of remote office mode, online interview has become one of the main forms of enterprise recruitment, and the rapid development of large language models (LLM) such as ChatGPT and Claude makes the interview cheating behavior show new concealment characteristics - candidates can query AI assistants for answers in real time, and traditional cheating detection methods that rely on pre-prepared materials are difficult to cope with. Under this background, computer vision and eye tracking technology has been gradually applied to the online examination field, and by analyzing the eye movement trajectory to identify abnormal behavior, it has become an important direction of anti-cheating research. At present, related technologies have achieved preliminary application in examination scenarios, such as monitoring head movement, eye deviation through cameras, or identifying AI-generated content through text detection tools, but special solutions for online interview scenarios are still in the exploratory stage.

[0003] However, the existing technology has significant limitations: first, the scene adaptability is insufficient, the existing eye movement monitoring system is mostly designed for standardized tests, and it is difficult to meet the high interactivity and real-time requirements of online interviews; second, the detection dimension is single, traditional systems only focus on explicit cheating such as external material review, and cannot identify the unique fast scanning and abnormal reading patterns of LLM cheating; third, the hardware threshold is high, professional eye movement equipment is expensive, and ordinary cameras have insufficient eye tracking accuracy based on WebGazer.js and other tools; fourth, there is a lack of real-time feedback mechanism, and it mostly relies on post-analysis, so the interviewer cannot intervene in a timely manner during the interview. In addition, the existing system has weak ability to distinguish between normal thinking and abnormal behavior, with a high false positive rate, which seriously affects the interview experience. Therefore, there is an urgent need for an eye movement analysis solution based on ordinary hardware, adapted to the interview scene, and capable of real-time identification of large model cheating. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides an online interview large model cheating detection system and method based on eye tracking, thereby solving the technical problems mentioned in the background art.

[0005] To achieve the above purpose, the present application realizes the following technical solutions:

[0006] The online interview large model cheating detection method based on eye tracking comprises:

[0007] S1: System initialization and environment adaptation, establish stable collection conditions through camera permission application, device environment detection;

[0008] S2: Eye tracking calibration and baseline establishment, 9-point initialization calibration based on WebGazer.js, personalized coefficients generated by combining pupil diameter and facial features, and candidate normal eye movement baseline established through standardized test questions;

[0009] S3: Real-time eye movement data collection and preprocessing, collecting gaze points, saccade trajectories, and pupil change data, generating structured data frames after cleaning and feature extraction;

[0010] S4: Execute anomaly detection algorithm, identify large model cheating related eye movement anomalies through three-layer algorithm of off-screen gaze detection, abnormal saccade pattern recognition, and LSTM model reading behavior discrimination;

[0011] S5: Risk assessment and output results, calculate comprehensive risk value based on abnormal feature set, real-time synchronization of risk level and candidate eye movement highlights, saccade trajectories to interviewer end, and generate visual analysis report containing artificial intervention information.

[0012] In one possible implementation, step S1 system initialization and environment adaptation includes:

[0013] Request camera usage rights and screen interaction permissions, load WebGazer.js eye tracking library, establish encrypted connection with cloud analysis service, and initialize dual-end interface synchronization engine;

[0014] Capture images through the camera, calculate light intensity parameters, facial feature point coverage, head pitch angle, and yaw angle;

[0015] Light intensity parameters are calculated using Weber contrast algorithm, facial feature point coverage is determined based on 68 pre-set feature points, and head pitch angle and yaw angle are limited to the qualified range;

[0016] If the environmental parameters meet the standards, proceed to step S2, otherwise generate adjustment prompts and re-detect, if the standards are not met for 5 consecutive times, terminate the interview.

[0017] In one possible implementation, step S2 eye tracking calibration and baseline establishment includes:

[0018] Initialization calibration phase, present 9 red dots in a 3x3 grid layout on the screen, guide the candidate to gaze in turn, record gaze point coordinates (g x ,g y ) and map the actual coordinates (s x ,s y ) through least squares fitting with the screen, the formula is: Where e1 is the initial average error, reflecting the eye tracking accuracy before optimization, i∈[1,9], representing the 9 red dots;

[0019] Dynamic calibration stage, according to the candidate pupil diameter and the preset average pupil diameter, the personalized coefficient k is generated, and the initial error is corrected to within 2°;

[0020] Through 3 standardized test questions, the fixation features and saccade features are collected, the fixation features include average fixation time, average fixation times, problem area fixation proportion, the saccade features include average saccade speed, average saccade amplitude, average saccade direction distribution;

[0021] When the core parameter standard deviation is less than or equal to 10%, it is determined that the baseline is stable and step S3 is entered, otherwise, the test time is extended or recalibration is performed.

[0022] In a possible implementation, the real-time eye movement data collection and preprocessing in step S3 includes:

[0023] The fixation point coordinates, fixation start and end time, pupil change rate, saccade starting point and ending point are collected at a frequency of 30fps;

[0024] The data is transmitted through WebSocket long connection encryption, and incremental compression is used to reduce delay;

[0025] The data is cleaned, and the blinking state and coordinate jump data exceeding the screen range for 3 consecutive frames without saccade connection are removed, and the effective rate needs to be greater than or equal to 95%;

[0026] The fixation features, saccade features, time sequence features and spatial features are extracted, and a timestamped structured data frame is generated.

[0027] In a possible implementation, the off-screen fixation detection in step S4 includes:

[0028] Whether it is off-screen fixation is judged by comparing the fixation point coordinates with the screen width and height, the start time is recorded and the duration is calculated;

[0029] The threshold value is dynamically adjusted according to the interview stage, the thinking stage threshold value is set to 5 seconds, and the answering stage threshold value is set to 2 seconds;

[0030] For the candidate whose problem area fixation proportion p0 in the baseline is less than 0.7, the threshold is relaxed by 10%;

[0031] If the off-screen fixation is timed out, an exception record containing the exception type, duration and timestamp is generated.

[0032] In a possible implementation, the abnormal saccade mode recognition in step S4 includes:

[0033] Based on the baseline parameters in step S2, the abnormal saccade speed threshold and amplitude threshold are set, both of which are 1.5 times the baseline value;

[0034] When both the saccade speed and amplitude are above the threshold, it is determined as fast and large amplitude saccade;

[0035] Detecting blank area saccade and periodic saccade, the blank area saccade is that both the start and end points of the saccade are located in the non-problem / input area and is continuous for more than or equal to 2 times, and the periodic saccade is that the saccade mode in the same question is repeated for more than or equal to 3 times;

[0036] Generate an abnormal record containing the type of abnormality, characteristic value and confidence.

[0037] In a possible implementation, the reading behavior discrimination of the LSTM model in step S4 includes:

[0038] Input the 5-second sliding window time sequence features extracted in step S3 into the LSTM model, and output the reading state probability;

[0039] According to the question display time determination stage, 0-10 seconds after the question display is the expected reading stage, and 10 seconds later is the expected thinking stage;

[0040] The reading state probability p in the expected thinking stage r > 0.8 is determined as abnormal reading behavior; the reading state probability p in the expected reading stage r < 0.2 is determined as expected reading absence;

[0041] Generate an abnormal record containing the reading state probability value, scene stage and determination result.

[0042] In a possible implementation, the risk assessment and output of the result in step S5 include:

[0043] Calculate the comprehensive risk value r s , and the calculation formula is r s =α×o s +β×s s +γ×r read , o s is the out-of-screen abnormality score, s s is the saccade abnormality score, and r read is the reading abnormality score, α, β and γ are the weights of each dimension, and α+β+γ=1;

[0044] The out-of-screen abnormality score , wherein Δt is the actual overtime, and T is the threshold value corresponding to the stage; s s =abnormal saccade number / total saccade number; r read =p r -0.5×2, wherein p r is the reading probability output by the LSTM model;

[0045] Divide the risk level according to the comprehensive risk value r sC is low risk, C≤r s D is medium risk, r s ≥D is high risk, wherein C, D are customizable risk thresholds;

[0046] The risk level and the set of abnormal features are synchronized in real time to the interviewer side, and the abnormal area is highlighted.

[0047] Low risk only displays the risk value, medium risk continuously updates monitoring, and high risk pushes intervention prompts.

[0048] After the interview, an analysis report containing abnormal statistics, eye movement visualization, confidence score and model verification is generated, encrypted and stored and pushed to the interviewer side.

[0049] In one possible implementation, the online interview big model cheating detection system based on eye tracking includes:

[0050] The system access and permission management module is used to request camera and screen interaction permissions and monitor the status;

[0051] The environment detection module is used for multi-dimensional detection of light, facial integrity and camera angle;

[0052] The eye tracking calibration module is used to optimize camera accuracy and establish a personalized baseline;

[0053] The real-time data acquisition and preprocessing module is used to acquire eye movement data and generate structured features;

[0054] The multi-dimensional anomaly detection module is used to execute three-layer algorithm to identify cheating-related eye movement anomalies;

[0055] The risk assessment and early warning module is used to calculate the risk value and push real-time visual prompts to the interviewer side;

[0056] The double-end synchronization module is used for pixel-level synchronization of the candidate side and the interviewer side interface and eye movement trajectory, including real-time encryption synchronization of the candidate side's fixation point highlight, saccade trajectory and fixation heat map to the interviewer side, supporting the interviewer to view, backtrack, manually mark abnormal areas and record intervention information;

[0057] The data storage and report generation module is used to encrypt the whole process data and automatically generate an analysis report.

[0058] Compared with the prior art, the beneficial effects are:

[0059] 1. In this scheme, based on the WebGazer.js technology, the mapping relationship between the gaze point and the screen coordinates is established through 9-point initialization calibration, and the personalized coefficient k is generated by combining the candidate pupil diameter and facial features, which can greatly reduce the tracking error of eye movement and meet the precision requirement of large model cheating detection without special hardware. This design not only reduces the deployment cost, but also breaks through the hardware limitation, making the technology adaptable to mainstream notebook devices, facilitating large-scale application, and solving the contradiction between high precision and low cost in traditional schemes;

[0060] 2. In this scheme, a three-layer detection algorithm system is constructed, the out-of-screen gaze detection identifies the line-of-sight deviation behavior through dynamic threshold, the abnormal saccade recognition captures rapid and irregular eye movement based on baseline parameters, and the LSTM model distinguishes normal thinking from abnormal reading through 5-second sliding window time series features. The three-layer algorithm works together to accurately identify the eye movement pattern specific to large model cheating, making up for the defect that traditional systems can only detect explicit cheating. At the same time, the system realizes low-latency data transmission through the WebSocket protocol to ensure that the interviewer can obtain real-time risk prompts;

[0061] 3. In this scheme, the context perception mechanism is introduced, the detection threshold is dynamically adjusted according to the interview stage and the candidate's baseline features, the abnormality is accurately marked, and the false positive rate is significantly reduced. In addition, the interviewer can intuitively display the eye movement trajectory visualization, heat map, and abnormal area highlight on the interviewer side, which not only provides objective basis for decision-making, but also avoids excessive intervention in the normal interview process, making the system adaptable to various scenarios such as technical interviews and comprehensive interviews, and has strong practicality and industrialization prospects;

[0062] 4. In this scheme, through the visualization of highlight points, trajectories, and heat maps, the system converts the abstract abnormalities recognized by the algorithm into concrete information that can be directly observed by the interviewer, forming a double insurance of efficient screening by the algorithm and complex scene judgment by humans. This mechanism retains the algorithm's processing capacity of thirty frames per second while introducing manual review, greatly reducing the false positive rate and covering edge cheating behaviors caused by atypical eye movements such as vision differences, achieving a balance between anti-cheating effect and interview experience. BRIEF DESCRIPTION OF DRAWINGS

[0063] The above description is only a summary of the technical scheme of the present application. In order to more clearly understand the technical means of the present application and can be implemented according to the content of the description, the following will be described in detail with the preferred embodiments of the present application and with the help of the drawings.

[0064] Figure 1 The flowchart of the online interview large model cheating detection method based on eye movement tracking;

[0065] Figure 2 The system framework diagram of the online interview large model cheating detection system based on eye movement tracking. DETAILED DESCRIPTION

[0066] The preferred embodiments of the present application will be described in detail by referring to the attached drawings, however the present application can be realized in various different forms, therefore the present application is not limited to the embodiments described below, in addition, in order to more clearly describe the present application, components not connected with the invention will be omitted from the drawings;

[0067] The technical solutions in the embodiments of the present application are to solve the problems in the above background art, and are generally as follows:

[0068] Embodiments:

[0069] As Figure 1 , the present embodiment introduces an online interview large model cheating detection method based on eye tracking, the specific steps are as follows:

[0070] S1: System initialization and environment adaptation

[0071] This step is the preparation stage of the interview, through system access, permission configuration and environment detection, to ensure the stability and accuracy of the subsequent eye tracking and cheating detection process. The core goal of this step is to establish a hardware environment and software connection that meets the detection requirements.

[0072] After the candidate enters the online interview system through the specified link, the system automatically loads the core module and pops up the permission application interface. At this time, the system will explicitly request two key permissions: one is the camera usage right, which is used to collect face images and eye movement data in the interview process; the second is the screen interaction permission, which is only used to synchronize the candidate interface layout to the interviewer end, and does not obtain any screen content information. After the candidate authorizes, the system starts the initialization process, which includes loading the WebGazer.js eye tracking library, establishing an encrypted connection with the cloud analysis service (using the TLS1.3 protocol), and initializing the double-end interface synchronization engine (based on DOM structure mirroring technology).

[0073] Environment detection is the core link of this step, which aims to judge whether the current environment meets the eye tracking requirements through multi-dimensional parameters. The system collects the candidate's face and surrounding environment images in real time through the camera, and calculates the following parameters for 3 seconds:

[0074] The light intensity parameter is calculated using the Weber contrast algorithm, the formula is: Where C represents the brightness contrast between the face and the background, L1 is the average brightness of the candidate's face area (unit: cd / m 2 ), and L2 is the average brightness of the background area (unit: cd / m 2The standard for sufficient light is C≥0.3, at which time the facial feature edges are clear, and the eye movement feature is easy to extract; if C<0.3, the system will prompt "the current light is insufficient, please adjust the light source to the front of the face". Thus, during the entire interview process, the system ensures that the ambient light is appropriate when collecting the candidate's face and the surrounding environment image in real time, so that the collected data is real and usable.

[0075] The facial integrity is judged by the coverage rate of 68 preset facial feature points, and the calculation formula is: Wherein, cov is the feature point coverage rate, N detected is the actual number of feature points detected by the system, and N total is the total number of preset feature points, which is fixed at 68, including eyebrows, eyes, nose, mouth and other key areas. The standard for facial integrity is cov≥0.85, which ensures that the eye movement can be stably tracked; if cov<0.85, the system will prompt "the face is not completely in the mirror, please adjust the camera angle or sitting posture". Ensure that the candidate's face is always relatively complete in the camera capture area during the entire interview process, and ensure that the collected image data can accurately reflect the candidate's behavior change.

[0076] The camera angle is judged by the head posture parameters, including the pitch angle θ and the yaw angle φ. θ represents the angle of the head tilting up and down, and the qualified range is [-15°, 15°]; φ represents the angle of the head turning left and right, and the qualified range is [-10°, 10°]. The standard for angle qualification is that θ and φ are within the above range, which ensures that the eye is in the best capture angle of the camera; if the angle exceeds the range, the system will prompt "the camera angle is skewed, please keep your head facing the screen".

[0077] The system performs real-time judgment on the environmental detection parameters: when C≥0.3, cov≥0.85, and θ, φ are within the qualified range, it is determined that the environment is suitable, the current environmental parameters are automatically saved, and S2 is entered; if any parameter does not meet the standard, the system will generate a targeted prompt according to the specific unqualified item, and remain in this step until the candidate adjusts and re-detects through. If the detection still does not meet the standard for 5 consecutive times, the system will determine that the device adaptation fails, terminate the interview process and prompt "the current device or environment cannot meet the interview requirements, please replace the device and try again".

[0078] S2: Eye movement tracking calibration and baseline establishment

[0079] This step optimizes the eye movement tracking accuracy of the ordinary camera and establishes the baseline of the candidate's normal eye movement behavior, so as to determine the accurate reference standard for subsequent abnormal detection. Eye movement tracking calibration solves the problem of insufficient accuracy of ordinary cameras, and baseline establishment clearly defines the boundary of "normal behavior", which together constitute the core reference for detection.

[0080] The eye movement tracking calibration is divided into two stages: initialization calibration and dynamic calibration.

[0081] In the initialization calibration stage, the system presents 9 red dots on the candidate's screen, which are evenly distributed at the four corners, the midpoints of the four sides, and the center of the screen, forming a 3x3 grid layout. The system guides the candidate to fix his gaze on each dot in turn, with each dot displayed for 2 seconds. During this time, the eye movement data acquisition module records the eye movement trajectory at a frequency of 30 fps, obtaining the gaze point coordinates (g x ,g y ) corresponding to each dot. The cloud analysis service processes these coordinates and fits the mapping relationship between the gaze point coordinates and the actual screen coordinates (s x ,s y ) using the least squares method. The calculation formula is: where e1 is the initial average error (in degrees), reflecting the eye movement tracking accuracy before optimization, i∈[1,9] represents the 9 red dots.

[0082] The dynamic calibration stage is used to optimize the initialization calibration results. The system generates personalized calibration coefficients k based on the candidate's facial features, with the calculation formula being where d is the actual pupil diameter of the candidate (obtained through image recognition, in pixels), d avg is the system's preset average pupil diameter (based on 5000 samples, with a value of 35 pixels), and 1.2 is the base adjustment coefficient. The initial error e1 is corrected by the personalized coefficient k to obtain the corrected error e2=e1xk. The goal of dynamic calibration is to control e2 within 2°, at which point the eye movement tracking accuracy of ordinary cameras can meet the anti-cheating detection requirements.

[0083] After calibration, the system enters the baseline establishment stage. The purpose of baseline establishment is to record the candidate's eye movement characteristics when thinking and reading normally, serving as a reference for distinguishing abnormal behavior. The system presents 3 standardized test questions to the candidate, including 2 text questions and 1 code question, guiding the candidate to think and answer normally for 2 minutes. During this period, the system collects and stores the following eye movement baseline parameters:

[0084] Gaze features: average gaze duration t0 (in seconds), average gaze frequency n0 (in times per question), and problem area gaze proportion p0 (problem area gaze time / total gaze time);

[0085] Saccade features: average saccade speed v0 (in ° / s), average saccade amplitude s0 (in degrees), and average saccade direction distribution d0 (horizontal / vertical saccade proportion).

[0086] The criterion for baseline establishment is parameter stability: when the standard deviation of core parameters such as t0, v0, p0 is less than or equal to 10%, it is determined that the baseline is stable, the baseline parameters are saved, and S3 is entered; if the parameter fluctuation exceeds 10%, the system will extend the test answering time by 1 minute, re-collect data until the baseline is stable. If it is still unstable after extension, the system will return to the initialization calibration phase of S2, and re-calibrate before establishing the baseline.

[0087] S3: Real-time eye movement data collection and preprocessing

[0088] This step continuously collects eye movement raw data during the interview process, and through cleaning, feature extraction and other processing, the raw data is converted into structured features that can be used for anomaly detection, providing usable data for the next step of data detection. Real-time data collection and data quality can accurately reflect the eye movement behavior of the candidate.

[0089] Real-time data collection is achieved through the eye movement data collection module on the candidate side. Based on the WebGazer.js library, the module captures eye movement at a frequency of 30 fps (i.e., every 33 milliseconds), and the raw data collected includes: gaze point coordinates (g x ,g y ), gaze start and end times (t s ,t e ), pupil change rate r, r = current pupil area / initial pupil area, saccade start point and end point These data are transmitted to the cloud analysis service through WebSocket long connection. To ensure real-time performance, the data transmission uses an incremental compression algorithm that only transmits the difference from the previous frame, i.e. and is encrypted by AES-128 algorithm to reduce transmission delay.

[0090] Raw data needs to be preprocessed before it can be used for detection. Preprocessing includes data cleaning and feature extraction. The purpose of data cleaning is to eliminate invalid data and avoid interfering with the detection results. The system identifies invalid data by the following rules: when the pupil change rate r is less than 0.5, it is determined to be a blinking state, and the eye movement data in the corresponding time period is marked as invalid; when the gaze point coordinates (g x ,g y ) exceed the screen range for 3 consecutive frames and there is no saccade trajectory connection, it is determined to be data jump and marked as invalid. The effective rate of cleaned data should be ≥95%, effective rate = effective data amount / total data amount, if it is lower than this value, the system will check the network status (through ping value detection, packet loss rate > 3% is determined as network anomaly), when the network is abnormal, the local cache mechanism (cache the last 30 seconds of data) is started, and the data is retransmitted when the network is restored; if it is a device anomaly, the candidate is prompted to adjust the device and re-collect.

[0091] Feature extraction is the process of converting cleaned data into structured features. The extracted features should be able to reflect the key attributes of eye movement behavior, including:

[0092] Gaze features: single-frame gaze duration t=t e -t s (unit: s), single-question gaze count n (counting the number of gazes under the current question), gaze area label a, where a=1 represents the question display area, a=2 represents the answer input area, and a=3 represents other areas, determined by coordinate matching;

[0093] Saccade features: saccade amplitude (unit: °), saccade velocity v=s / (t e -t s (unit: ° / s), saccade direction (unit: °, 0° represents horizontal right, 90° represents vertical up);

[0094] Timing features: t, v sequence in 5-second sliding window (arranged in time order to form a timing array);

[0095] Spatial features: distribution density of gaze points on the screen (gaze count per 10x10 pixel area).

[0096] After preprocessing, the system generates structured data frames containing the above features, each frame with a timestamp, and transmits them to the anomaly detection module of S4 in real time. If the data frame format is complete (contains all feature items) and the timestamp is continuous (adjacent frame interval ≤33ms), it is determined that the preprocessing is qualified and enters S4; otherwise, the system will re-extract the features of the corresponding time period until a qualified data frame is generated.

[0097] S4: Execute anomaly detection algorithm

[0098] This step is based on the structured feature data after step S3 preprocessing, and identifies abnormal eye movement patterns related to large language model cheating through three-layer detection algorithm. From different dimensions, it captures the characteristics of cheating behavior, ensuring the comprehensiveness and accuracy of detection. The three-layer algorithm targets off-screen gaze, abnormal saccade pattern, and abnormal reading behavior, forming a complementary detection system.

[0099] The first layer is off-screen gaze detection, which is used to identify the candidate's gaze deviating from the interview interface and the behavior of possibly consulting external AI tools. The system compares the gaze point coordinates (g x ,g y ) with the screen width and height (w, h) to determine whether it is off-screen, with the formula isOffScreen=(g x <0 ∨ g x >w ∨ gy <0∨g y >h), when isOffScreen is true, the system records the start time t start of the off-screen gaze; when isOffScreen is false, the system calculates the duration of the off-screen gaze Δt = current time - t start .

[0100] To avoid false positives, the threshold of off-screen gaze is dynamically adjusted according to the interview context: when the system detects that the candidate has not entered content and the duration is ≥ 3 seconds, it is determined to be a “thinking phase”, and the threshold is set to 5 seconds, i.e. Δt > 5 seconds is marked as an exception; when it is detected that the candidate is entering content (the input area has character changes), it is determined to be an “answering phase”, and the threshold is tightened to 2 seconds, i.e. Δt > 2 seconds is marked as an exception. At the same time, the system will combine the baseline parameter p0 (problem area gaze proportion) established in step S2 for candidates with p0 < 0.7 (baseline vision is easy to deviate) to relax the threshold by 10%. If the off-screen gaze is detected to be overdue, an exception record is generated, including the exception type (“off-screen gaze”), the duration Δt and the occurrence timestamp.

[0101] The second layer is abnormal saccade pattern recognition, which is used to capture rapid and irregular saccadic behavior related to real-time query AI answers. The system sets dynamic thresholds based on the baseline parameters v0 and s0 of step S2: v threshold = 1.5 × v0, s threshold = 1.5 × s0, where v threshold is the abnormal saccadic velocity threshold, s threshold is the abnormal saccadic amplitude threshold. When the saccadic velocity v > v threshold and the amplitude s > s threshold , it is determined to be “fast and large amplitude saccade”, which is a high-risk exception.

[0102] In addition, the system also detects two typical abnormal saccade patterns: one is “blank area saccade”, i.e. the starting point and the ending point of the saccade are located in the a = 3 region (non-problem / input area) and appear continuously ≥ 2 times; the other is “periodic saccade”, i.e. the saccade pattern repeats ≥ 3 times in the same question (calculated by cosine similarity, similarity > 0.8 is determined to be repeated). If any of the above patterns is detected, an exception record is generated, including the exception type (“fast saccade”, “blank area saccade”, etc.), the exception characteristic value (such as v = 350° / s) and the confidence (the number of abnormal characteristics / total number of characteristics).

[0103] The third layer is LSTM-based reading behavior discrimination, which is used to distinguish between normal thinking and abnormal reading (such as reading AI-generated answers). The system inputs the 5-second sliding window time series features (t, v, s sequence) extracted in step S3 into the LSTM model, and the model outputs the reading state probability pr (p r ∈[0,1], p r The closer it is to 1, the more likely it is that the person is reading.

[0104] The determination of reading behavior is combined with the interview scenario context: the system determines the stage based on the question presentation time—0-10 seconds after the question is presented is the "expected reading stage" (reading the question is normal behavior during this time), and after 10 seconds is the "expected thinking stage" (at this time, one should focus on thinking rather than continuous reading). If p is in the expected thinking stage... r If the value is greater than 0.8, it is considered "abnormal reading behavior"; if it is within the expected reading stage... r A value <0.2 is considered "missing expected reading" (the problem may not have been understood; a comprehensive judgment based on other characteristics is needed). Both cases generate an anomaly record containing p. r Value, scenario stage, and judgment result.

[0105] The results of multi-dimensional anomaly detection are summarized into an anomaly feature set, which includes the anomaly type, occurrence time, feature value, and confidence level of each layer of detection. If the anomaly feature set is empty (no anomalies), the system returns to step S3 to continue collecting data; if the anomaly feature set is not empty, it proceeds to step S5 for risk assessment.

[0106] S5: Risk assessment and output results

[0107] This step, based on the abnormal feature set in step S4, transforms the technical detection results into decision-making basis that the interviewer can understand by quantifying risk values, providing real-time warnings, and generating analysis reports, thus deriving conclusions from data transformation.

[0108] The core of risk assessment is calculating the comprehensive risk value r. s The calculation formula is r s =α×o s +β×s s +γ×r read , where o s The score for off-screen anomalies (range 0-1) is calculated as follows: Where Δt is the actual timeout time, and T is the threshold for the corresponding stage. For example, in the thinking stage, T = 5 seconds, and if the timeout is 3 seconds, then o s =3 / 5=0.6; s s The score for saccade anomalies (range 0-1) is calculated as s. s =Number of abnormal scans / Total number of scans (statistics within a single question); r read The reading anomaly score (range 0-1) is calculated as r. read =p r -0.5×2(p rThe reading probability output by the LSTM model, the farther the deviation from 0.5, the higher the score); a, b, g are the weights of each dimension, and a+b+g=1, which can be dynamically allocated according to the type of interview, and set according to the direction of different abnormalities to the cheating behavior, which are respectively set to 0.3, 0.3 and 0.4 in this embodiment.

[0109] According to r s Value risk level: r s <C is low risk (green mark), C≤r s <D is medium risk (yellow mark), r s ≥D is high risk (red mark), wherein C, D are customizable risk thresholds, and in this embodiment, C, D are 0.3 and 0.7 respectively. The risk level and the abnormal feature set are synchronized to the interviewer interface in real time—the interviewer interface and the candidate interface are pixel-level consistent (synchronized through DOM mirroring), and the abnormal area is highlighted (such as the out-of-screen gaze direction is marked with a red arrow, and the abnormal saccade trajectory is marked with a red line), and the r s value and the abnormal occurrence time are displayed.

[0110] The eye movement information of the candidate interface is presented in a visual form: the gaze point is dynamically displayed as a yellow highlight point, the size of the highlight point increases with the gaze duration, and the highlight point is marked with “gaze duration X seconds”; the out-of-screen gaze direction is highlighted with a red arrow, and the length of the arrow increases with the duration, specifically, it is extended to 20 pixels after more than 2 seconds, and marked with “out-of-screen duration X seconds”; the abnormal saccade trajectory is marked with a 2px thick red line, and the line color deepens with the speed, and the speed is more than 1.5 times the baseline, and the color is deep red, and marked with “speed X° / s, amplitude X°”; the high-frequency gaze area is superimposed with a heat map, and red represents a gaze frequency of >5 times / minute, yellow represents 3-5 times / minute, and blue represents <3 times / minute.

[0111] The interviewer interface supports “real-time observation” and “trajectory backtracking” modes. In real-time observation, the highlight point and the trajectory are dynamically updated with eye movement; in trajectory backtracking, the time axis slider can be dragged to view data in any time period (such as the trajectory of the 2-6 minute interview). If the interviewer finds an abnormal highlight point distribution (such as continuous gaze on the corner of the screen that is not marked by the algorithm), the area can be clicked to mark it with an orange box (line width 3px), the system automatically associates the eye movement data (pupil change rate, head posture) in the corresponding time period, and increases the “manual suspiciousness” weight in the risk value (default 15%, can be adjusted to 10%-20% through the sliding bar). The interviewer can click “pause interview” to ask the candidate (such as “What is the reason for your gaze on the left side of the screen at XX time?”), and the system automatically records the inquiry time, response content and corresponding eye movement segment as the “manual intervention record” of the final report.

[0112] The real-time warning mechanism triggers according to the risk level: low risk, only display the risk value, do not disturb the interview; medium risk, update the r s value every 5 seconds, continuous monitoring; high risk, the system automatically sends a prompt to the interviewer "the candidate currently has a high risk of cheating, it is recommended to intervene in real time". The interviewer can choose to pause the interview for inquiry, at this time the system will pause data collection, record the pause time point, and continue to collect from the pause time point after the interview resumes, to ensure data continuity.

[0113] After the interview, the system enters the result output stage, and generates a complete analysis report. The report contains the following core content:

[0114] Abnormal statistics: the number of occurrences of each type of anomaly (such as 3 times of off-screen anomaly, 5 times of scanning anomaly), total duration (such as total anomaly duration of 12 seconds), highest risk value r s-max and occurrence time;

[0115] Eye movement visualization: whole process eye movement trajectory graph (drawn in time sequence), fixation heat map (use color depth to represent fixation time distribution, red for high frequency area), anomaly point marking graph (mark all anomaly occurrence positions);

[0116] Confidence score: Among them is the average value of r s (range 0-1), c s closer to 1 indicates that the candidate's behavior is more in line with the normal mode;

[0117] Model verification: backtracking accuracy acc = correct judgment times / total judgment times based on known cheating samples (used to explain the reliability of the detection model and the accuracy of model judgment)

[0118] After the report is generated, the system will perform integrity check to check whether it contains all core content (abnormal statistics, visualization charts, c s value, acc value). If complete, the report is stored encrypted (SHA-256 algorithm is used to generate check code) and pushed to the interviewer end; if missing, the corresponding data is supplemented from the database to regenerate until the report is complete. Finally, the report serves as an auxiliary basis for interview decision, supporting the interviewer to make the final conclusion combined with artificial judgment.

[0119] For example Figure 2 , on the basis of embodiment one, an online interview large model cheating detection system based on eye movement tracking is also introduced, which includes:

[0120] System access and permission management module is the entrance for candidates to enter the online interview system. It guides the candidates to complete system access through a standardized web interface, supports candidates to access and automatically load core resources such as eye tracking library through specified links. At the same time, this module will request the use right of the camera and the screen interaction permission (only for interface synchronization) through the browser API, and monitor the permission state in real time;

[0121] Environment detection module detects whether the interview environment meets the eye tracking requirements through computer vision technology, including light intensity, face integrity and camera angle detection in three dimensions. Light intensity is calculated using Weber contrast algorithm, face integrity is judged by 68 feature point coverage, and camera angle is evaluated by head pitch angle and yaw angle;

[0122] Eye tracking calibration module is used to optimize the accuracy of eye tracking of ordinary cameras. After calibration, it will also establish a normal eye movement baseline, record parameters such as average fixation duration and saccade speed, and use them as reference standards for anomaly detection;

[0123] Real-time data acquisition and preprocessing module collects eye movement raw data, transmits them through WebSocket protocol encryption and uses incremental compression to reduce delay. In the data preprocessing stage, invalid data such as blinking and coordinate jumping are removed, and fixation features, saccade features, time sequence features and spatial features are extracted to generate structured data frames and transmit them to the multi-dimensional anomaly detection module;

[0124] Multi-dimensional anomaly detection module identifies abnormal eye movement patterns related to large model cheating through three layers of algorithms: the first layer detects off-screen fixation, combined with dynamic adjustment of timeout threshold during the interview stage; the second layer identifies abnormal saccade patterns such as rapid and large amplitude saccades and blank area saccades, and sets threshold values based on baseline parameters; the third layer uses LSTM model to judge reading behavior, combined with scene stage to determine whether there is an anomaly. After detecting an anomaly, it will generate an anomaly record containing type, timestamp and other information;

[0125] Risk assessment and early warning module calculates the comprehensive risk value based on the anomaly record, integrates the scores of off-screen anomaly, saccade anomaly and reading anomaly through weight allocation, divides the risk into low, medium and high levels and corresponds to green, yellow and red markers. Risk information will be synchronized to the interviewer's end in real time, and high-risk automatic intervention prompts will be pushed, supporting the interviewer to pause the interview and ask questions. Low-risk will continue to monitor and update the risk value;

[0126] The double-end synchronization module realizes interface layout pixel-level synchronization through DOM structure mirroring technology. Meanwhile, the module synchronizes visual data such as eye movement track and gaze heat map of the candidate to the interviewer end in real time, supports display switching of the track and the heat map, and facilitates the interviewer to intuitively observe the eye movement state of the candidate. Specifically, the candidate end: the fixation point coordinates are converted into dynamic highlight points (70% transparency to avoid blocking the content) through WebGL technology; the saccade track is drawn in a time sequence with a 2px red dotted line, and the nodes are updated every 50ms; a local heat map is generated every 5 seconds, that is, the current question area is taken as the center, and a 200x200 pixel range is taken, and the heat map moves with the screen scrolling and the question content synchronously.

[0127] The transmission layer adopts WebSocket long connection encryption transmission (AES-256 algorithm), synchronizes one frame of data every 33ms (consistent with the eye movement collection frame rate), compresses the single-frame data to within 512 bytes (only the changed coordinates are transmitted), and ensures that the delay is less than or equal to 100ms.

[0128] The interviewer end supports three kinds of view switching: "highlight point view", "track view" and "fusion view", and can display the highlight point, track and heat map at the same time; clicking the highlight point can pop up a detail panel to display the timestamp, area label (such as "question area 3rd paragraph" and "right side of the screen") and pupil diameter curve; screenshot export (PNG format) and 30-second eye movement video recording (MP4 format) are supported.

[0129] The data storage and report generation module stores all the data, including original eye movement data, abnormal records, risk values and the like, in a MySQL cluster, and ensures data integrity through encryption and check code. After the interview ends, an analysis report is automatically generated, which includes abnormal statistics, eye movement visual charts, confidence score and model verification results, and is pushed to the interviewer end after integrity verification, thereby providing a basis for interview decision-making.

[0130] In the application, a number of formulas involved are calculated by taking their numerical values after being dimensionless, and the establishment of the formulas is obtained by software simulation of a large amount of data to obtain a formula closest to the real situation. Some coefficients or weights in the formula are set by the person skilled in the art according to the actual situation, so no more will be described here.

[0131] The above embodiments can be realized wholly or partially by software, hardware, firmware or any other combination. When realized by software, the above embodiments can be realized in the form of a computer program product wholly or partially. Those skilled in the art can realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solutions.

[0132] Finally, it should be noted that the above-mentioned embodiments are merely intended for the purpose of illustration and are not intended to limit the embodiments. Based on the above description, those skilled in the art can make other variations or changes in the form of the embodiments. It is not necessary or possible to exhaust all the embodiments. The obvious changes or variations derived therefrom are still within the scope of the present application.

Claims

1. A cheating detection method for large-scale online interviews based on eye-tracking, characterized in that, include: S1: System initialization and environment adaptation, establishing stable data acquisition conditions through camera permission application and device environment detection; S2: Eye tracking calibration and baseline establishment. Nine-point initialization calibration is performed based on WebGazer.js. Personalized coefficients are generated by combining pupil diameter and facial features. A normal eye tracking baseline for candidates is established through standardized test questions. S3: Real-time eye-tracking data acquisition and preprocessing, collecting fixation point, saccade trajectory and pupil change data, and generating structured data frames after cleaning and feature extraction; S4: Execute the anomaly detection algorithm, which uses a three-layer algorithm—off-screen gaze detection, abnormal saccade pattern recognition, and LSTM model reading behavior discrimination—to identify eye movement anomalies related to cheating in large models. S5: Risk assessment and output results, calculate comprehensive risk value based on abnormal feature set, synchronize risk level and candidate's eye movement high spot and saccade trajectory to interviewer in real time, and generate a visual analysis report containing human intervention information.

2. The online interview large-scale cheating detection method based on eye tracking as described in claim 1, characterized in that, Step S1, system initialization and environment adaptation, includes: Request camera access and screen interaction permissions, load the WebGazer.js eye-tracking library, establish an encrypted connection with the cloud analytics service, and initialize the dual-end interface synchronization engine; Images are captured by a camera, and light intensity parameters, facial feature point coverage, head pitch angle, and yaw angle are calculated. The light intensity parameter was calculated using the Weber contrast algorithm, the facial feature point coverage was determined based on 68 preset feature points, and the head pitch angle and yaw angle were limited to the qualified range. If the environmental parameters meet the requirements, proceed to step S2; otherwise, generate an adjustment prompt and retest. If the requirements are not met after 5 consecutive tests, the interview will be terminated.

3. The online interview large-scale cheating detection method based on eye tracking as described in claim 1, characterized in that, Step S2, eye-tracking calibration and baseline establishment, includes: During the initial calibration phase, nine red dots in a 3×3 grid layout are displayed on the screen to guide the candidate to gaze at them sequentially, and the coordinates of the gaze points are recorded (g). x ,g y ) and fit the actual coordinates (s) using the screen least squares method. x ,s y The mapping relationship is calculated using the following formula: Where e1 is the initial average error, reflecting the eye-tracking accuracy before optimization, and i∈[1,9] represents 9 red dots; During the dynamic calibration phase, a personalized coefficient k is generated based on the candidate's pupil diameter and the preset average pupil diameter to correct the initial error to within 2°. Fixation and saccade characteristics were collected using three standardized test questions. Fixation characteristics included average fixation duration, average number of fixations, and the percentage of fixations in the problem area. Saccade characteristics included average saccade speed, average saccade amplitude, and average saccade direction distribution. If the standard deviation of the core parameter is ≤10%, the baseline is considered stable and proceeds to step S3; otherwise, the test time is extended or recalibration is performed.

4. The online interview large-scale cheating detection method based on eye tracking as described in claim 1, characterized in that, Step S3, real-time eye-tracking data acquisition and preprocessing, includes: At a frequency of 30fps, the coordinates of the fixation point, the start and end times of fixation, the pupillary rate of change, and the start and end points of saccades are collected. Data is transmitted encrypted via a WebSocket long-lived connection, and incremental compression is used to reduce latency. Clean the data, removing data from blinking states and coordinate jumps that exceed the screen range for three consecutive frames without a glancing transition, with an effectiveness rate of ≥95%. Extract gaze features, saccade features, temporal features, and spatial features to generate structured data frames with timestamps.

5. The online interview large-scale cheating detection method based on eye tracking as described in claim 1, characterized in that, Step S4, off-screen gaze detection, includes: Determine whether the gaze is off-screen by comparing the coordinates of the gaze point with the screen width and height, record the start time and calculate the duration; The thresholds are dynamically adjusted according to the interview stage: the threshold for the thinking stage is set at 5 seconds, and the threshold for the answering stage is set at 2 seconds. For candidates whose baseline fixation percentage for problem areas is p0 < 0.7, the threshold is relaxed by 10%. If off-screen gaze timeout occurs, an exception record is generated that includes the exception type, duration, and timestamp.

6. The online interview large-scale cheating detection method based on eye tracking as described in claim 1, characterized in that, Step S4, abnormal scanning pattern recognition, includes: Based on the baseline parameters in step S2, set the abnormal saccade speed threshold and amplitude threshold, both of which are 1.5 times the baseline value; When both the scanning speed and amplitude exceed the threshold, it is judged as rapid and large-amplitude scanning; Detect blank area scanning and periodic scanning. Blank area scanning is defined as scanning where the start and end points are both located in the non-question / input area and occur consecutively ≥2 times. Periodic scanning is defined as scanning patterns repeating ≥3 times within the same question. Generate anomaly records that include anomaly type, feature value, and confidence level.

7. The online interview large-scale cheating detection method based on eye tracking as described in claim 1, characterized in that, Step S4, the LSTM model's reading behavior discrimination includes: Input the 5-second sliding window temporal features extracted in step S3 into the LSTM model and output the reading state probability; The stage is determined by the timing of the question presentation: 0-10 seconds after the question is presented is the expected reading stage, and 10 seconds after that is the expected thinking stage. The probability p of the reading state during the expected thinking stage r When the value is greater than 0.8, it is judged as abnormal reading behavior; the probability p of the reading state during the expected reading stage. r When the value is less than 0.2, it is considered as missing expected reading. Generate anomaly records that include reading state probability values, scene stage, and judgment results.

8. The online interview large-scale cheating detection method based on eye tracking as described in claim 1, characterized in that, Step S5, which involves risk assessment and outputting results, includes: Calculate the overall risk value r s The calculation formula is r s =α×o s +β×s s +γ×r read o s For off-screen anomaly scores, s s For the score of saccade anomalies and r read The sum of reading abnormality scores is a weighted sum, where α, β, and γ are the weights of each dimension, and α + β + γ = 1; Off-screen anomaly scores Where Δt is the actual timeout time, and T is the threshold value for the corresponding stage; s s =Number of abnormal saccades / Total number of saccades; r read =|p r -0.5|×2, where p r The reading probability output by the LSTM model; Risk levels are classified according to the comprehensive risk value, r s <C represents low risk, C≤r s <D represents medium risk, r s ≥D indicates high risk, where C and D are customizable risk thresholds; Risk levels and abnormal feature sets are synchronized to the interviewer's end in real time, and abnormal areas are highlighted. Low risk only displays the risk value, medium risk is continuously updated and monitored, and high risk pushes intervention prompts; After the interview, an analysis report containing anomaly statistics, eye-tracking visualization, confidence scores, and model validation is generated, encrypted, stored, and pushed to the interviewer.

9. A large-scale online interview cheating detection system based on eye tracking, implementing the cheating detection method according to any one of claims 1 to 8, characterized in that, include: The system access and permission management module is used to request camera and screen interaction permissions and monitor their status. The environmental detection module is used for multi-dimensional detection of light, facial integrity, and camera angle; An eye-tracking calibration module is used to optimize camera accuracy and establish a personalized baseline; The real-time data acquisition and preprocessing module is used to acquire eye-tracking data and generate structured features; The multi-dimensional anomaly detection module is used to execute a three-layer algorithm to identify cheating-related eye movement anomalies; The risk assessment and early warning module is used to calculate risk values ​​and push visual prompts to the interviewer's end in real time; The dual-end synchronization module is used for pixel-level synchronization of the candidate's interface and eye movement trajectory with the interviewer's interface. This includes real-time encrypted synchronization of the candidate's fixation highlight, saccade trajectory, and fixation heatmap to the interviewer's interface, allowing the interviewer to view, review, manually mark abnormal areas, and record intervention information. The data storage and report generation module is used to encrypt and store all data and automatically generate analysis reports.

Images