Anti-cheating check-in method and system based on cross-modal feature cooperation consistency inference

Abstract

The application discloses a consistency inference anti-cheating check-in method and system based on cross-modal feature cooperation, and belongs to the technical field of identity verification. The multi-modal terminal is used for collecting face, voiceprint, positioning and action original features in real time, and a feature encoder is used for mapping into a feature embedding vector with unified dimensions. The system firstly calculates the identity consistency, and constructs a cross-modal time sequence feature matrix after the verification is passed. The feature change amount of adjacent node positioning and action is extracted to screen suspicious nodes. A sliding monitoring mechanism and a cross-modal conflict index evaluation model are introduced. The local cheating risk score is dynamically calculated according to the suspicious frequency, and compared with a blocking threshold, so as to identify and intercept cheating behaviors such as virtual positioning and video signing. The application breaks the single mode limitation, accurately identifies the behavior logic anomaly, effectively reduces the false alarm rate in complex scenes, and realizes a high-robustness check-in management closed loop.

Description

Technical Field

[0001] This invention relates to the field of identity verification technology, specifically to a method and system for preventing cheating during sign-in based on consistency inference using cross-modal feature collaboration. Background Technology

[0002] With the rapid development of information technology and the widespread adoption of mobile internet, scenarios such as remote work, online education, large-scale conferences and events, and corporate field management have created extremely broad demands for remote identity verification and attendance management. In these complex scenarios that are no longer subject to traditional physical supervision, ensuring the authenticity of the identity of those signing in and the objectivity and validity of their attendance, while preventing various forms of cheating, has become a core pain point that urgently needs to be addressed in the digital management of various industries.

[0003] Most mainstream attendance systems on the market currently rely on single-modal biometrics (such as simple facial recognition or voiceprint comparison) or single-geographic location (such as GPS check-in). However, single-dimensional verification mechanisms have significant security vulnerabilities and are easily compromised by malicious cheating tools. To improve security, some more advanced systems have begun to introduce multi-modal data collection (such as "face + GPS" dual verification), but these existing technologies still face the following major technical challenges in practical applications: Existing multimodal attendance systems often only perform independent "result comparisons" for each single modality. For example, the system checks whether the face matches and whether the GPS is within the attendance range; if both pass, attendance is considered successful. This architecture fails to detect deeper logical paradoxes: cheaters can tamper with GPS coordinates using virtual location software, but the gyroscope and accelerometer (motion modality) of the attendance terminal show the device as absolutely stationary. Because existing technology lacks collaborative analysis of the change processes across different modalities, it cannot accurately identify this "pseudo-stationary teleportation" logical paradox caused by tampering with single-dimensional data, resulting in a still relatively low barrier to cheating.

[0004] Existing anti-cheating monitoring typically employs a static threshold trigger mechanism, meaning that any significant abrupt change in location data or features during a single data collection is immediately flagged as cheating. However, in complex real-world geographical environments (such as GPS signal multipath effects caused by densely populated urban areas with high-rise buildings, or location drift after entering underground spaces) or in specific dynamic commuting scenarios, normal signal drift and other data abrupt changes are easily misjudged by the system. The system lacks the ability to comprehensively examine single-point anomalies within a continuous time context and cannot eliminate environmental noise, resulting in a persistently high false alarm rate.

[0005] The few existing systems that attempt to introduce time-series monitoring typically have monitoring windows that are passively triggered and independent of each other. Once cheaters figure out the system's monitoring patterns, they can easily circumvent the system's punishment and blocking mechanisms by precisely controlling the duration of their cheating software's on and off states and using the window's sliding boundary to directly "reset" cheating records. Summary of the Invention

[0006] The purpose of this invention is to provide a consistency inference anti-cheating sign-in method and system based on cross-modal feature collaboration to solve the problems mentioned in the background art.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A consistency inference-based anti-cheating sign-in system based on cross-modal feature collaboration. This system includes: The multimodal feature acquisition and encoding module configures multimodal verification strategies based on the check-in scenario. It performs dimensionality reduction mapping on the acquired original multimodal features through a unified dimensional feature encoder to obtain a unified real-time feature embedding vector and a pre-registered baseline feature vector. The cross-modal temporal feature aggregation module is used to aggregate the real-time feature embedding vectors output by various multimodal feature acquisition units of the same user under a set equal-interval temporal sampling node, and construct a single-user cross-modal temporal feature matrix; The consistency analysis and suspect marking module, based on the pre-registration baseline feature vector and the single-user cross-modal temporal feature matrix, sequentially performs identity feature group consistency analysis and behavior feature group consistency analysis, and marks the temporal sampling node suspected of cheating when the identity verification is passed and the behavior feature group undergoes an abnormal change; The sliding window monitoring and risk response module is used to construct a sliding monitoring mechanism for the time-series sampling node sequence when a suspected cheating determination is triggered, extract the local suspected subset within the time-series sliding window to calculate the cross-modal conflict index, evaluate the local cheating risk score, and output a check-in response command based on the comparison result of the local cheating risk score and the blocking threshold.

[0008] Furthermore, the multimodal feature acquisition and encoding module includes: The multimodal check-in terminal has a built-in face capture unit, voiceprint capture unit, positioning capture unit, and motion capture unit. The unified dimension feature mapping unit contains a lightweight fully connected layer network that independently corresponds to each modality feature. It is used to map the original feature vectors of different dimensions and the original features of the pre-registered identity to a fixed dimension, and outputs the real-time feature embedding vector and the pre-registered benchmark feature vector with completely consistent dimensions and non-negative elements.

[0009] Furthermore, the cross-modal temporal feature aggregation module includes: Independent coding units are used to independently identify and encode the current check-in user, multimodal feature acquisition unit, and time-series sampling node; The matrix construction unit is used to initialize the equally spaced temporal sampling node sequence, obtain the real-time feature embedding vectors output by each feature acquisition unit of the same user at each temporal sampling node, and construct the single-user cross-modal temporal feature matrix with modality classification and temporal node as the cross dimension.

[0010] Furthermore, the consistency analysis and suspect labeling module includes: The identity consistency calculation unit calculates the identity consistency index based on the cosine similarity between the real-time feature embedding vectors of face and voiceprint and the corresponding pre-registered baseline feature vectors, and introduces a preset zero-removal constant. If the identity consistency index is less than the identity consistency threshold, the check-in process is blocked; otherwise, the behavior consistency calculation unit is executed. The behavior consistency calculation unit is used to extract the real-time feature embedding vectors of geographical location and limb movement under adjacent time-series sampling nodes from the time-series feature matrix, so as to calculate the corresponding feature change amount; if the feature change amount is greater than the preset location change threshold or action change threshold, it is determined as suspected cheating check-in and the current time-series sampling node is marked.

[0011] Furthermore, the sliding window monitoring and risk response module includes: The sliding window initialization unit is used to construct a timing sliding window with a set timing length and sliding step size when cheating suspicion is triggered. The suspected frequency statistics unit is used to extract the time-series sampling nodes marked as suspected cheating within the time-series sliding window to form a local suspected subset, and to calculate the frequency of suspected occurrences within the sliding window. The cross-modal conflict calculation unit is used to extract the corresponding changes in geographical location features and changes in limb movement features for temporal sampling nodes within a local suspected subset, and calculate the cross-modal conflict index by combining the preset modal alignment weight coefficient and abnormal mutation penalty coefficient. The risk decision and iteration unit calculates the local cheating risk score based on the frequency of suspected occurrences, the cross-modal conflict index, and the preset decision weights. If the local cheating risk score reaches or exceeds the preset blocking threshold, the unit outputs a confirmation interception result. If the score is less than the blocking threshold, the suspected marker on the node is removed, and the time-series sliding window is iterated forward until the check-in process ends.

[0012] A consistency inference-based anti-cheating sign-in method based on cross-modal feature collaboration includes the following steps: Step S1: Configure a multimodal verification strategy based on the check-in scenario, and perform dimensionality reduction mapping on the collected original multimodal features through a unified dimensional feature encoder to obtain a multidimensional unified real-time feature embedding vector and a pre-registration benchmark feature vector. Step S2: Under the set equal-interval time-series sampling nodes, the real-time feature embedding vectors output by each multimodal feature acquisition unit of the same user are aggregated to construct a single-user cross-modal time-series feature matrix; Step S3: Based on the pre-registration baseline feature vector and the single-user cross-modal temporal feature matrix, perform identity feature group consistency analysis and behavior feature group consistency analysis in sequence. If the identity verification passes and the behavior feature group undergoes an abnormal change, mark the temporal sampling node suspected of cheating. Step S4: If a suspected cheating determination is triggered, a sliding monitoring mechanism is constructed for the time-series sampling node sequence, a local suspected subset is extracted within the time-series sliding window to calculate the cross-modal conflict index, and the local cheating risk score is evaluated. Based on the local cheating risk score, a check-in response instruction is output.

[0013] Furthermore, the specific implementation process of step S1 includes: A multimodal verification strategy is configured based on the check-in scenario. The verification strategy is used to set the collection accuracy of face, voiceprint, geographical location, and body movement. The multimodal check-in terminal is a check-in hardware designed for single users, and it has a multimodal feature acquisition unit that includes a face acquisition unit, a voiceprint acquisition unit, a positioning acquisition unit, and a motion acquisition unit. The unified dimension feature encoder incorporates a modal lightweight fully connected layer network, comprising a first fully connected layer, a second fully connected layer, a third fully connected layer, and a fourth fully connected layer, each independently corresponding to face, voiceprint, geographic location, and body movement. The original feature vectors output by each multimodal feature acquisition unit and the original features of the user's pre-registered identity are respectively input into the corresponding first fully connected layer, second fully connected layer, third fully connected layer, and fourth fully connected layer. Using ReLU as the activation function, the original feature vectors of different dimensions are uniformly mapped to a fixed dimension D, outputting real-time feature embedding vectors with completely consistent dimensions and non-negative elements, along with the pre-registration baseline feature vector, which are jointly stored in the terminal's local feature temporary cache area.

[0014] Furthermore, the specific implementation process of step S2 includes: The current check-in user, multimodal feature acquisition unit, and time-series sampling node are independently encoded; Initialize an equally spaced temporal sampling node sequence, and at each temporal sampling node, obtain the D-dimensional real-time feature embedding vector output by each multimodal feature acquisition unit of the same user; Using modality type as the row and temporal sampling node as the column, a single-user cross-modal temporal feature matrix M is constructed. The matrix element M(a, i) is the D-dimensional real-time feature embedding vector of the a-th type of multimodal feature acquisition unit at the i-th temporal sampling node.

[0015] Furthermore, the specific implementation process of step S3 includes: Let the multimodal feature acquisition unit of type a be denoted as... Where a=1 corresponds to the face acquisition unit, a=2 corresponds to the voiceprint acquisition unit, a=3 corresponds to the positioning acquisition unit, and a=4 corresponds to the motion acquisition unit. The i-th time-series sampling node is denoted as Extract matrix elements M(a, i) from the single-user cross-modal temporal feature matrix M, as... exist Output real-time feature embedding vector Real-time feature embedding vector The same-dimensional pre-registration baseline feature vector is denoted as ; Perform identity feature group consistency analysis: Based on the cosine similarity between the real-time feature embedding vectors of face and voiceprint and the pre-registered baseline feature vectors, the identity consistency index of the current check-in user U is calculated. max() is the function to maximize the value. This is a preset feature to prevent the division of positive numbers into zero; A preset identity consistency threshold is set. If the identity consistency index is less than the identity consistency threshold, it is determined that the sign-in is confirmed as cheating and the sign-in process is blocked. If the identity consistency index is greater than or equal to the identity consistency threshold, the identity verification is determined to be successful and the behavior feature group consistency analysis is entered. Consistency analysis of behavioral characteristic groups: Extract real-time feature embedding vectors of geographical location and limb movements of adjacent temporal sampling nodes from the single-user cross-modal temporal feature matrix M, and calculate the feature change of geographical location. Calculate the characteristic changes in limb movements In the formula, and These are, in order, the timing sampling nodes. and Real-time feature embedding vectors of geographic location and These are, in order, the timing sampling nodes. and Real-time feature embedding vectors of lower limb movements; Preset thresholds for location change and action change. If the change in geographical location features exceeds the location change threshold or the change in limb movement features exceeds the action change threshold, the node is identified as suspected cheating during check-in and the time-series sampling node is marked. If the change in geographical location does not exceed the location mutation threshold or the change in limb movement does not exceed the movement mutation threshold, it is considered a normal check-in.

[0016] Furthermore, the specific implementation process of step S4 includes: When performing consistency analysis of behavioral feature groups, if a suspected cheating check-in is triggered, a sliding monitoring mechanism is constructed for the time-series sampling node sequence, initializing a time-series sliding window with a time-series length of L and a sliding step size of f. Where m is the iteration number of the sliding window, then the time-series sliding window... It contains L consecutive time-series sampling nodes; Extracting time-series sliding window The time-series sampling nodes that are internally marked as suspected of cheating during check-in are used to form a local suspected subset. The number of time-series sampling nodes forming the local suspected subset is then counted. And calculate the timing sliding window Frequency of suspected incidents within ; If the local suspected subset is not empty, for any k-th time-series sampling node in the local suspected subset... Extracting the feature changes of geographical location and characteristic changes in limb movements Calculate the cross-modal conflict index In the formula, and These are the preset modal alignment weight coefficients for geographical location and body movement, respectively. The default abnormal mutation penalty coefficient is set, and max() is the maximum value function. Based on time-series sliding window Frequency of suspected incidents within and cross-modal conflict index Evaluation of time-series sliding window Local cheating risk score In the formula, and These are the preset decision weights; If the local suspected subset is empty, then the number of time-series sampling nodes Then let ; The local cheating risk score is compared with a preset blocking threshold: If the local cheating risk score is greater than or equal to the blocking threshold, a blocking command is immediately triggered, terminating subsequent time-series sampling nodes and outputting a confirmed cheating interception result. If the local cheating risk score is less than the blocking threshold, then a time-series sliding window is determined. The anomaly within is environmental noise. Remove the suspected flag from the time-series sampling node and slide the time-series sliding window forward by step size f. Repeat the comparison between the local cheating risk score and the preset blocking threshold until the entire time sequence sampling of the check-in process is completed, and output the check-in success result.

[0017] The beneficial effects achieved by this invention are: Unlike existing technologies that only focus on "result comparison" of a single modality, this invention delves into the "causal binding" of different modal change processes. By calculating the cross-modal conflict index between changes in geographical location and changes in limb movements, it can accurately identify the logical paradox of "pseudo-stationary teleportation" caused by tampering with single-dimensional data (such as tampering with GPS data but not gyroscope data), thus raising the threshold for cheating. Unlike the static threshold triggering mechanism in existing technologies, this invention places single-point anomalies (suspect markers) into a sliding time window for comprehensive review. By weighting the frequency of suspect occurrences with the conflict index, it can eliminate data mutations caused by signal drift as environmental noise, which helps to reduce the false alarm rate in complex environments. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0019] Figure 1 This is a schematic diagram illustrating the steps of the consistency inference anti-cheating sign-in method based on cross-modal feature collaboration of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] In this first embodiment: a consistency inference anti-cheating check-in system based on cross-modal feature collaboration is provided, the system comprising: The multimodal feature acquisition and encoding module configures multimodal verification strategies based on the check-in scenario. It performs dimensionality reduction mapping on the acquired original multimodal features through a unified dimensional feature encoder to obtain a unified real-time feature embedding vector and a pre-registered baseline feature vector. Specifically, the multimodal feature acquisition and encoding module includes: The multimodal check-in terminal has a built-in face capture unit, voiceprint capture unit, positioning capture unit, and motion capture unit. The unified dimension feature mapping unit contains a lightweight fully connected layer network that independently corresponds to each modality feature. It is used to map the original feature vectors of different dimensions and the original features of pre-registered identity to a fixed dimension. It outputs real-time feature embedding vectors with completely consistent dimensions and non-negative elements and pre-registered baseline feature vectors.

[0022] The cross-modal temporal feature aggregation module is used to aggregate the real-time feature embedding vectors output by various multimodal feature acquisition units of the same user under the set equal-interval temporal sampling nodes, and construct a single-user cross-modal temporal feature matrix. Specifically, the cross-modal temporal feature aggregation module includes: Independent coding units are used to independently identify and encode the current check-in user, multimodal feature acquisition unit, and time-series sampling node; The matrix construction unit is used to initialize the equally spaced temporal sampling node sequence. At each temporal sampling node, it obtains the real-time feature embedding vectors output by each feature acquisition unit of the same user, and constructs a single-user cross-modal temporal feature matrix with modality classification and temporal node as the cross dimension.

[0023] The consistency analysis and suspect marking module, based on the pre-registration baseline feature vector and the single-user cross-modal temporal feature matrix, sequentially performs identity feature group consistency analysis and behavior feature group consistency analysis, and marks the temporal sampling node suspected of cheating when the identity verification is passed and the behavior feature group undergoes an abnormal change; Specifically, the consistency analysis and suspect labeling module includes: The identity consistency calculation unit calculates the identity consistency index based on the cosine similarity between the real-time feature embedding vectors of face and voiceprint and the corresponding pre-registered baseline feature vectors, and introduces a preset zero-removal constant. If the identity consistency index is less than the identity consistency threshold, the check-in process is blocked; otherwise, the behavior consistency calculation unit is executed. The behavior consistency calculation unit is used to extract the real-time feature embedding vectors of geographical location and limb movement under adjacent temporal sampling nodes from the temporal feature matrix, so as to calculate the corresponding feature change amount. If the feature change amount is greater than the preset position change threshold or action change threshold, it is determined as suspected cheating check-in and the current temporal sampling node is marked.

[0024] The sliding window monitoring and risk response module is used to construct a sliding monitoring mechanism for the time-series sampling node sequence when a suspected cheating judgment is triggered. It extracts the local suspected subset within the time-series sliding window to calculate the cross-modal conflict index and evaluate the local cheating risk score. Based on the comparison result between the local cheating risk score and the blocking threshold, it outputs the check-in response instruction. Specifically, the sliding window monitoring and risk response module includes: The sliding window initialization unit is used to construct a timing sliding window with a set timing length and sliding step size when cheating suspicion is triggered. The suspect frequency statistics unit is used to extract the time-series sampling nodes marked as suspected cheating within the time-series sliding window to form a local suspect subset, and to calculate the frequency of suspected occurrences within the sliding window. The cross-modal conflict calculation unit is used to extract the corresponding changes in geographical location features and changes in limb movement features for temporal sampling nodes within a local suspected subset, and calculate the cross-modal conflict index by combining the preset modal alignment weight coefficient and abnormal mutation penalty coefficient. The risk decision and iteration unit calculates the local cheating risk score based on the frequency of suspected occurrences, cross-modal conflict index, and preset decision weights. If the local cheating risk score reaches or exceeds the preset blocking threshold, the unit outputs a confirmation interception result. If the score is less than the blocking threshold, the suspected marker on the node is removed, and the time-series sliding window is iterated forward until the check-in process ends.

[0025] Please see Figure 1 In this second embodiment: a consistency inference anti-cheating sign-in method based on cross-modal feature collaboration is provided, which includes the following steps: Step S1: Configure a multimodal verification strategy based on the check-in scenario, and perform dimensionality reduction mapping on the collected original multimodal features through a unified dimensional feature encoder to obtain a multidimensional unified real-time feature embedding vector and a pre-registration benchmark feature vector. For example, a multimodal verification strategy is configured based on the check-in scenario. The verification strategy is used to set the collection accuracy of face, voiceprint, geolocation, and body movements. The multimodal check-in terminal is a check-in hardware designed for single users, with a built-in multimodal feature acquisition unit that includes a face acquisition unit, a voiceprint acquisition unit, a positioning acquisition unit, and a motion acquisition unit. The unified dimensional feature encoder incorporates a lightweight fully connected layer network, which includes a first, second, third, and fourth fully connected layer corresponding to face, voiceprint, geolocation, and body movement, respectively. The original feature vectors output by each multimodal feature acquisition unit and the original features of the user's pre-registered identity are input into the corresponding first, second, third, and fourth fully connected layers, respectively. Using ReLU as the activation function, the original feature vectors of different dimensions are uniformly mapped to a fixed dimension D, and the output real-time feature embedding vectors with completely consistent dimensions and non-negative elements and the pre-registration baseline feature vector are stored together in the terminal's local feature temporary cache area. Each multimodal feature acquisition unit of the unified dimension feature encoder corresponds to the first fully connected layer, the second fully connected layer, the third fully connected layer, and the fourth fully connected layer. During the training phase, the contrastive loss function or the triplet loss function is used for optimization, so that the Euclidean distance of the behavioral feature group vector is positively correlated with the actual geographical displacement and limb movement deviation.

[0026] It is worth mentioning that, for the training phase of the unified dimension feature encoder, the supervision signal acquisition method for "the Euclidean distance of the behavioral feature group vector is positively correlated with the actual geographical displacement and limb movement deviation" is as follows: real-time acceleration and angular velocity data are collected by calling the inertial measurement unit (IMU) built into the check-in terminal as physical deviation labels for limb movements, and the absolute physical distance is obtained by calling the GPS / BeiDou dual-mode positioning module as geographical displacement labels. Considering that the ReLU activation function may generate resting neurons, leading to extremely sparse feature vectors and causing a sharp drop in cosine similarity under extreme environmental noise, in actual deployment, the activation functions of the first to fourth fully connected layers can be optimized to Leaky ReLU or combined with L2 normalization. Through L2 normalization, the magnitude of the output real-time feature embedding vector is constrained to 1, so that the values ​​in subsequent exponential operations have a strict upper bound, thus eliminating the risk of floating-point overflow.

[0027] Step S2: Under the set equal-interval time-series sampling nodes, aggregate the real-time feature embedding vectors output by each multimodal feature acquisition unit of the same user to construct a single-user cross-modal time-series feature matrix; For example, the current check-in user, the multimodal feature acquisition unit, and the time-series sampling node are independently encoded; Initialize an equally spaced temporal sampling node sequence, and at each temporal sampling node, obtain the D-dimensional real-time feature embedding vector output by each multimodal feature acquisition unit of the same user; Using modality type as the row and temporal sampling node as the column, a single-user cross-modal temporal feature matrix M is constructed. The matrix element M(a, i) is the D-dimensional real-time feature embedding vector of the a-th type of multimodal feature acquisition unit at the i-th temporal sampling node.

[0028] Step S3: Based on the pre-registration baseline feature vector and the single-user cross-modal temporal feature matrix, perform identity feature group consistency analysis and behavior feature group consistency analysis in sequence. If the identity verification passes and the behavior feature group undergoes an abnormal change, mark the temporal sampling node suspected of cheating. For example, the a-th type of multimodal feature acquisition unit is denoted as Where a=1 corresponds to the face acquisition unit, a=2 corresponds to the voiceprint acquisition unit, a=3 corresponds to the positioning acquisition unit, and a=4 corresponds to the motion acquisition unit. The i-th time-series sampling node is denoted as Extract matrix elements M(a, i) from the single-user cross-modal temporal feature matrix M, as... exist Output real-time feature embedding vector Real-time feature embedding vector The same-dimensional pre-registration baseline feature vector is denoted as ; Perform identity feature group consistency analysis: Based on the cosine similarity between the real-time feature embedding vectors of face and voiceprint and the pre-registered baseline feature vectors, the identity consistency index of the current check-in user U is calculated. max() is the function to maximize the value. For the preset prevention of dividing by zero positive numbers ( The value is This is used to avoid the division-by-zero anomaly caused by the ReLU activation function resulting in all feature vectors being zero. A preset identity consistency threshold is set. If the identity consistency index is less than the identity consistency threshold, it is determined that the sign-in is confirmed as cheating and the sign-in process is blocked. If the identity consistency index is greater than or equal to the identity consistency threshold, the identity verification is determined to be successful and the behavior feature group consistency analysis is entered. Consistency analysis of execution behavior feature groups: Extract real-time feature embedding vectors of geographical location and limb movements of adjacent temporal sampling nodes from the single-user cross-modal temporal feature matrix M, and calculate the feature change of geographical location. Calculate the characteristic changes in limb movements In the formula, and These are, in order, the timing sampling nodes. and Real-time feature embedding vectors of geographic location and These are, in order, the timing sampling nodes. and Real-time feature embedding vectors of lower limb movements; Preset thresholds for location change and action change. If the change in geographical location features exceeds the location change threshold or the change in limb movement features exceeds the action change threshold, the node is identified as suspected cheating during check-in and the time-series sampling node is marked. If the change in geographical location does not exceed the location mutation threshold or the change in limb movement does not exceed the movement mutation threshold, it is considered a normal check-in.

[0029] It is worth mentioning that when performing consistency analysis of behavioral feature groups to calculate feature changes, the initial boundary conditions for the time-series sampling node sequence are as follows: when i=1 (at this time...) (Not present), the system configuration defaults to inheriting the baseline state, making and This is to prevent array out-of-bounds errors or null pointer exceptions in the program.

[0030] Step S4: If a suspected cheating determination is triggered, a sliding monitoring mechanism is constructed for the time-series sampling node sequence, the local suspected subset within the time-series sliding window is extracted to calculate the cross-modal conflict index, and the local cheating risk score is evaluated. Based on the local cheating risk score, a check-in response instruction is output. For example, when performing consistency analysis of behavioral feature groups, if a suspected cheating check-in is triggered, a sliding monitoring mechanism is constructed for the time-series sampling node sequence, and a time-series sliding window with a time-series length of L and a sliding step size of f is initialized. Where m is the iteration number of the sliding window, then the time-series sliding window... It contains L consecutive time-series sampling nodes; Extracting time-series sliding window The time-series sampling nodes that are internally marked as suspected of cheating during check-in are used to form a local suspected subset. The number of time-series sampling nodes that constitute the local suspected subset is counted. And calculate the timing sliding window Frequency of suspected incidents within ; If the local suspect subset is not empty, for any temporal sampling node in the local suspect subset... Extracting the feature changes of geographical location and characteristic changes in limb movements Calculate the cross-modal conflict index In the formula, and These are the preset modal alignment weight coefficients for geographical location and body movement, respectively. The default abnormal mutation penalty coefficient is set, and max() is the maximum value function. Based on time-series sliding window Frequency of suspected incidents within and cross-modal conflict index Evaluation of time-series sliding window Local cheating risk score In the formula, and These are the preset decision weights; If the local suspect subset is empty, then the number of time-series sampling nodes Then let ; The local cheating risk score is compared with a preset blocking threshold: If the local cheating risk score is greater than or equal to the blocking threshold, a blocking command is immediately triggered, terminating subsequent time-series sampling nodes and outputting a confirmed cheating interception result. If the local cheating risk score is less than the blocking threshold, then a time-series sliding window is determined. The anomaly within is environmental noise. Remove the suspected flag from the time-series sampling node and slide the time-series sliding window forward by step size f. Repeat the comparison between the local cheating risk score and the preset blocking threshold until the entire time sequence sampling of the check-in process is completed, and output the check-in success result.

[0031] It is worth mentioning that, considering the possibility of a rapid increase in geographical displacement while limb movements remain still, leading to an exponential explosion in the cross-modal conflict index and resulting in false positives by the system, a "dynamic scene limiting mechanism" is introduced: the system monitors the velocity scalar of the positioning acquisition unit in real time, and if the velocity scalar continues to exceed a preset commuting threshold, the system dynamically decays the preset abnormal mutation penalty coefficient. Alternatively, a non-linear smoothing function (such as Sigmoid or Tanh) can be used to smooth the formula. The input items are subjected to amplitude limiting mapping to reduce the false alarm rate; Furthermore, considering that the sliding monitoring mechanism for the temporal sampling node sequence is not passively triggered by a single suspected node, but rather adopts a "globally continuous concurrent, locally independent evaluation" architecture, specifically, the temporal sliding window starts immediately when the first temporal sampling node is generated and continues to slide forward for calculation, ensuring that any low-frequency, highly deceptive cheating behavior can be smoothly considered within the historical context; the "removal of suspicion flag" operation when the local cheating risk score is less than the blocking threshold, that is, when the temporal sliding window is determined to be... When the anomaly within the system is environmental noise, the system only masks the suspected weight of that node in the risk score calculation of the current window, but retains the original feature deviation record of that node in the global single-user cross-modal time series feature matrix. When the window slides to a certain position by step size f, the system will remove the suspected weight of that node. Furthermore, when the overlapping node is covered again, the system will independently re-evaluate it based on the context of the new adjacent nodes to effectively prevent cheaters from "clearing" cheating records by precisely controlling the cheating duration and using the window sliding boundary point.

[0032] It should be noted that this embodiment has several preset parameters, which can be obtained through a combination of offline data calibration or scenario experience statistics. To make this embodiment easier to understand, the preset parameters are explained as follows: Identity consistency threshold, location mutation threshold, and action mutation threshold: These are obtained by relying on the historical operational database of the check-in system. During system initialization, a historical sample set containing both normal check-ins and known fraudulent check-ins is extracted, and the feature deviation distribution of the samples in the identity feature group and behavior feature group is calculated separately. By plotting the receiver operation characteristic curve (ROC curve), a balance point of equal error rate (EER) is sought, or empirical values ​​are determined based on the tolerance of false positive rate (FAR) and false negative rate (FRR) according to the specific business scenario. For example, in the financial field check-in scenario with extremely high security requirements, the location mutation threshold will be conservatively determined by referring to the side of the ROC curve with high false negative rate; while in ordinary office check-in, the EER balance point is taken.

[0033] Blocking threshold: Based on the historical statistical distribution of local cheating risk scores, the system extracts the risk scores of known cheating sample sets and takes 95% of the lower limit of its confidence interval as the benchmark blocking threshold to ensure that the vast majority of high-risk anomalies can trigger the interception command.

[0034] Modal alignment weight coefficient and Because the changes in geographical location features and the changes in limb movement features have inherent differences in physical magnitude and feature dimension, and This is used to align both to the same numerical order of magnitude. The method for obtaining this information can be as follows: Construct a controlled verification set containing "stationary and tampered GPS" and "normal movement" data; use a grid search algorithm to optimize multidimensional parameters within the interval [0, 1]; and take maximizing the F1-Score of the system's recognition of cross-modal logical anomalies (i.e., large displacement but stationary movement) as the optimization objective to determine the optimal [data / method / mechanics]. and Coefficient pair.

[0035] Abnormal mutation penalty coefficient This coefficient controls the amplification factor of the exponential function on abrupt changes. Its calibration method is based on inverse derivation of the expected environmental noise variance, ensuring that when... or When the fluctuation is within the range of normal environmental noise (such as normal drift caused by GPS multipath effect), the exponential decay term It approaches 1 (i.e., does not amplify the error); the exponential penalty is only triggered when the mutation value significantly exceeds the noise variance.

[0036] Decision weights and In the calculation of local cheating risk scores, the preset decision weights are... and (and satisfy) Initial values ​​can be obtained through expert scoring (AHP). To further improve accuracy, linear classifiers such as Logistic Regression or Support Vector Machine (SVM) can be used for supervised training on a labeled validation set, and the weight coefficients of the corresponding features of the model after training convergence can be extracted as preset values.

[0037] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0038] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A consistency inference-based anti-cheating sign-in method based on cross-modal feature collaboration, characterized in that, The method includes the following steps: Step S1: Configure a multimodal verification strategy based on the check-in scenario, and perform dimensionality reduction mapping on the collected original multimodal features through a unified dimensional feature encoder to obtain a multidimensional unified real-time feature embedding vector and a pre-registration benchmark feature vector. Step S2: Under the set equal-interval time-series sampling nodes, the real-time feature embedding vectors output by each multimodal feature acquisition unit of the same user are aggregated to construct a single-user cross-modal time-series feature matrix; Step S3: Based on the pre-registration baseline feature vector and the single-user cross-modal temporal feature matrix, perform identity feature group consistency analysis and behavior feature group consistency analysis in sequence. If the identity verification passes and the behavior feature group undergoes an abnormal change, mark the temporal sampling node suspected of cheating. Step S4: If a suspected cheating determination is triggered, a sliding monitoring mechanism is constructed for the time-series sampling node sequence, the local suspected subset within the time-series sliding window is extracted to calculate the cross-modal conflict index, and the local cheating risk score is evaluated. Based on the local cheating risk score, a check-in response instruction is output. The specific implementation process of step S1 includes: A multimodal verification strategy is configured based on the check-in scenario. The verification strategy is used to set the collection accuracy of face, voiceprint, geographical location, and body movements. The multimodal check-in terminal is a check-in hardware designed for single users, and it has a multimodal feature acquisition unit that includes a face acquisition unit, a voiceprint acquisition unit, a positioning acquisition unit, and a motion acquisition unit. The unified dimension feature encoder incorporates a modal lightweight fully connected layer network, comprising a first fully connected layer, a second fully connected layer, a third fully connected layer, and a fourth fully connected layer, each independently corresponding to face, voiceprint, geographic location, and body movement. The original feature vectors output by each multimodal feature acquisition unit and the original features of the user's pre-registered identity are respectively input into the corresponding first fully connected layer, second fully connected layer, third fully connected layer, and fourth fully connected layer. Using ReLU as the activation function, the original feature vectors of different dimensions are uniformly mapped to a fixed dimension D, outputting real-time feature embedding vectors with completely consistent dimensions and non-negative elements, along with the pre-registration baseline feature vector, which are jointly stored in the terminal's local feature temporary cache area.

2. The anti-cheating sign-in method based on cross-modal feature collaboration for consistency inference according to claim 1, characterized in that, The specific implementation process of step S2 includes: The current check-in user, multimodal feature acquisition unit, and time-series sampling node are independently encoded; Initialize an equally spaced temporal sampling node sequence, and at each temporal sampling node, obtain the D-dimensional real-time feature embedding vector output by each multimodal feature acquisition unit of the same user; Using modality type as the row and temporal sampling node as the column, a single-user cross-modal temporal feature matrix M is constructed. The matrix element M(a, i) is the D-dimensional real-time feature embedding vector of the a-th type of multimodal feature acquisition unit at the i-th temporal sampling node.

3. The anti-cheating sign-in method based on cross-modal feature collaboration according to claim 1, characterized in that, The specific implementation process of step S3 includes: Let the multimodal feature acquisition unit of type a be denoted as . Where a=1 corresponds to the face acquisition unit, a=2 corresponds to the voiceprint acquisition unit, a=3 corresponds to the positioning acquisition unit, and a=4 corresponds to the motion acquisition unit. The i-th time-series sampling node is denoted as Extract matrix elements M(a, i) from the single-user cross-modal temporal feature matrix M, as... exist Output real-time feature embedding vector Real-time feature embedding vector The same-dimensional pre-registration baseline feature vector is denoted as ; Perform identity feature group consistency analysis: Based on the cosine similarity between the real-time feature embedding vectors of face and voiceprint and the pre-registered baseline feature vectors, the identity consistency index of the current check-in user U is calculated. max() is the function to maximize the value. This is a preset feature to prevent the division of positive numbers into zero; A preset identity consistency threshold is set. If the identity consistency index is less than the identity consistency threshold, it is determined that the sign-in is confirmed as cheating and the sign-in process is blocked. If the identity consistency index is greater than or equal to the identity consistency threshold, the identity verification is determined to be successful and the behavior feature group consistency analysis is entered. Consistency analysis of execution behavior feature groups: Extract real-time feature embedding vectors of geographical location and limb movements of adjacent temporal sampling nodes from the single-user cross-modal temporal feature matrix M, and calculate the feature change of geographical location. Calculate the characteristic changes in limb movements In the formula, and These are, in order, the timing sampling nodes. and Real-time feature embedding vectors of geographic location and These are, in order, the timing sampling nodes. and Real-time feature embedding vectors of lower limb movements; Preset thresholds for location change and action change. If the change in geographical location features exceeds the location change threshold or the change in limb movement features exceeds the action change threshold, the system is identified as suspected cheating during check-in and the timing sampling node is marked. If the change in geographical location does not exceed the location mutation threshold or the change in limb movement does not exceed the movement mutation threshold, it is considered a normal check-in.

4. The anti-cheating sign-in method based on cross-modal feature collaboration for consistency inference according to claim 1, characterized in that, The specific implementation process of step S4 includes: When performing consistency analysis of behavioral feature groups, if a suspected cheating check-in is triggered, a sliding monitoring mechanism is constructed for the time-series sampling node sequence, initializing a time-series sliding window with a time-series length of L and a sliding step size of f. Where m is the iteration number of the sliding window, then the time-series sliding window... It contains L consecutive time-series sampling nodes; Extracting time-series sliding window The time-series sampling nodes that are internally marked as suspected of cheating during check-in are used to form a local suspected subset. The number of time-series sampling nodes forming the local suspected subset is then counted. And calculate the timing sliding window Frequency of suspected incidents within ; If the local suspected subset is not empty, for any k-th time-series sampling node in the local suspected subset... Extracting the feature changes of geographical location and characteristic changes in limb movements Calculate the cross-modal conflict index In the formula, and These are the preset modal alignment weight coefficients for geographical location and body movement, respectively. The default abnormal mutation penalty coefficient is set, and max() is the maximum value function. Based on time-series sliding window Frequency of suspected incidents within and cross-modal conflict index Evaluation of time-series sliding window Local cheating risk score In the formula, and These are the preset decision weights; If the local suspected subset is empty, then the number of time-series sampling nodes Then let ; The local cheating risk score is compared with a preset blocking threshold: If the local cheating risk score is greater than or equal to the blocking threshold, a blocking command is immediately triggered, terminating subsequent time-series sampling nodes and outputting a confirmed cheating interception result. If the local cheating risk score is less than the blocking threshold, then a time-series sliding window is determined. The anomaly within is environmental noise. Remove the suspected flag from the time-series sampling node and slide the time-series sliding window forward by step size f. Repeat the comparison between the local cheating risk score and the preset blocking threshold until the entire time sequence sampling of the check-in process is completed, and output the check-in success result.

5. A consistency inference-based anti-cheating sign-in system based on cross-modal feature collaboration, executing the consistency inference-based anti-cheating sign-in method based on cross-modal feature collaboration as described in any one of claims 1-4, characterized in that, The system includes: The multimodal feature acquisition and encoding module configures multimodal verification strategies based on the check-in scenario. It performs dimensionality reduction mapping on the acquired original multimodal features through a unified dimensional feature encoder to obtain a unified real-time feature embedding vector and a pre-registered baseline feature vector. The cross-modal temporal feature aggregation module is used to aggregate the real-time feature embedding vectors output by various multimodal feature acquisition units of the same user under a set equal-interval temporal sampling node, and construct a single-user cross-modal temporal feature matrix; The consistency analysis and suspect marking module, based on the pre-registration baseline feature vector and the single-user cross-modal temporal feature matrix, sequentially performs identity feature group consistency analysis and behavior feature group consistency analysis, and marks the temporal sampling node suspected of cheating when the identity verification is passed and the behavior feature group undergoes an abnormal change; The sliding window monitoring and risk response module is used to construct a sliding monitoring mechanism for the time-series sampling node sequence when a suspected cheating determination is triggered, extract the local suspected subset within the time-series sliding window to calculate the cross-modal conflict index, evaluate the local cheating risk score, and output a check-in response command based on the comparison result of the local cheating risk score and the blocking threshold.

6. The anti-cheating sign-in system based on cross-modal feature collaboration and consistency inference as described in claim 5, characterized in that, The multimodal feature acquisition and encoding module includes: The multimodal check-in terminal has a built-in face capture unit, voiceprint capture unit, positioning capture unit, and motion capture unit. The unified dimension feature mapping unit contains a lightweight fully connected layer network that independently corresponds to each modality feature. It is used to map the original feature vectors of different dimensions and the original features of the pre-registered identity to a fixed dimension, and outputs the real-time feature embedding vector and the pre-registered benchmark feature vector with completely consistent dimensions and non-negative elements.

7. The consistency inference anti-cheating sign-in system based on cross-modal feature collaboration according to claim 5, characterized in that, The cross-modal temporal feature aggregation module includes: Independent coding units are used to independently identify and encode the current check-in user, multimodal feature acquisition unit, and time-series sampling node; The matrix construction unit is used to initialize the equally spaced temporal sampling node sequence, obtain the real-time feature embedding vectors output by each feature acquisition unit of the same user at each temporal sampling node, and construct the single-user cross-modal temporal feature matrix with modality classification and temporal node as the cross dimension.

8. The anti-cheating sign-in system based on cross-modal feature collaboration and consistency inference as described in claim 5, characterized in that, The consistency analysis and suspect labeling module includes: The identity consistency calculation unit calculates the identity consistency index based on the cosine similarity between the real-time feature embedding vectors of face and voiceprint and the corresponding pre-registered baseline feature vectors, and introduces a preset zero-removal constant. If the identity consistency index is less than the identity consistency threshold, the check-in process is blocked; otherwise, the behavior consistency calculation unit is executed. The behavior consistency calculation unit is used to extract the real-time feature embedding vectors of geographical location and limb movement under adjacent time-series sampling nodes from the time-series feature matrix, so as to calculate the corresponding feature change amount; if the feature change amount is greater than the preset location change threshold or action change threshold, it is determined as suspected cheating check-in and the current time-series sampling node is marked.

9. The anti-cheating sign-in system based on cross-modal feature collaboration and consistency inference as described in claim 5, characterized in that, The sliding window monitoring and risk response module includes: The sliding window initialization unit is used to construct a timing sliding window with a set timing length and sliding step size when cheating suspicion is triggered. The suspected frequency statistics unit is used to extract the time-series sampling nodes marked as suspected cheating within the time-series sliding window to form a local suspected subset, and to calculate the frequency of suspected occurrences within the sliding window. The cross-modal conflict calculation unit is used to extract the corresponding changes in geographical location features and changes in limb movement features for temporal sampling nodes within a local suspected subset, and calculate the cross-modal conflict index by combining the preset modal alignment weight coefficient and abnormal mutation penalty coefficient. The risk decision and iteration unit calculates the local cheating risk score based on the frequency of suspected occurrences, the cross-modal conflict index, and the preset decision weights. If the local cheating risk score reaches or exceeds the preset blocking threshold, the unit outputs a confirmation interception result. If the score is less than the blocking threshold, the suspected marker on the node is removed, and the time-series sliding window is iterated forward until the check-in process ends.

Images