A sports cheating identification system and method based on multi-modal perception and cloud three-level auditing

Abstract

The application discloses a kind of based on multi-modal perception and cloud three-level auditing's sports cheating identification system and method, it is related to sports monitoring and anti-cheating technical field.System includes local acquisition module, cloud processing module and teacher final audit module.Local acquisition module is through the GPS of mobile terminal, camera and microphone synchronous acquisition trajectory, video and audio data of whole process of movement;Cloud processing module carries out hash comparison, speed analysis and voiceprint identification to data, generates suspect mark and evidence package;Teacher final audit module is manually reviewed by teacher and makes final determination, AI does not have the final arbitrating right.The application only relies on smart phone standard sensor, without additional hardware, can identify physical cheating and Root injection cheating simultaneously, and through teacher final audit mechanism, AI misjudgment risk is eliminated.

Description

Technical Field

[0001] This invention relates to the field of sports monitoring and anti-cheating technology, specifically to a method and system for identifying physical cheating and software-injected cheating in physical education assessments by combining the built-in sensors (GPS, camera, microphone) of a smartphone with cloud-based AI-assisted detection and a teacher's manual final review mechanism. Background Technology

[0002] Currently, fitness tracking apps, such as "LeRunTrack," primarily rely on built-in phone sensors (such as accelerometers and GPS) to verify the authenticity of exercise data. However, two main types of cheating methods exist in practice: **The first category is physical cheating**, including but not limited to placing a mobile phone on a means of transportation such as light rail, bus, or private car, or strapping it to an animal (such as a dog), or attaching it to a drone, to deceive GPS and accelerometers through external physical movement in order to falsify the trajectory and speed.

[0003] The second type is system-level cheating (i.e., root injection cheating or mapping cheating). Cheaters gain root access to obtain the highest level of control over the phone, install root hiding tools (such as Magisk Hide) to bypass the app's root detection, and then load a kernel module to inject fake acceleration, gyroscope, and step count data into the sensor driver layer, making all motion parameters read by the app from the system API fictitious values. Furthermore, cheaters can also deceive the camera by looping pre-recorded real running videos.

[0004] Currently, no existing technology can effectively identify both types of cheating methods simultaneously. Especially regarding root injection cheating, because it forges data at the system's lowest level, application-layer algorithms cannot distinguish between real sensor signals and forged signals, rendering existing anti-cheating measures completely ineffective. Furthermore, existing solutions rely excessively on AI for automatic judgment and lack a human review mechanism, posing a risk of misjudgment and potentially resulting in unfair punishment for students. Summary of the Invention

[0005] (a) Purpose of the invention The present invention aims to provide a method and system for identifying sports cheating that does not rely on high-precision sensors, cannot be bypassed by root injection, and requires all final judgments to be manually confirmed by teachers, in order to overcome the shortcomings of existing technologies that cannot simultaneously identify physical cheating and software injection cheating, and lack a manual review mechanism.

[0006] (II) Technical Solution To achieve the above-mentioned objectives, the present invention provides the following technical solution: **1. System Overall Architecture** This invention adopts a three-layer architecture of "local data collection + cloud processing + final review by teachers": - Local acquisition layer: GPS, front camera (480p / 15fps), and microphone (16kHz sampling rate) are enabled throughout the exercise. The raw data is only stored in local temporary files and no real-time analysis is performed.

[0007] - Cloud processing layer: After the exercise is completed, the data packet is uploaded to the cloud server. The AI ​​performs multiple tests in sequence, such as hash comparison, pace analysis, voiceprint recognition, and scene recognition, to generate suspect markers and evidence packages.

[0008] - Teacher final review layer: All records marked as suspicious by AI are pushed to teachers for manual review, and teachers make the final decision. AI does not have any final judgment authority.

[0009] **2. Hash value comparison (for video playback cheating)** After the exercise, the entire video was downgraded to 480p resolution, and keyframes were extracted at a rate of 1 frame per second. A perceptual hash value (pHash) was calculated for each frame to generate a hash fingerprint sequence for that exercise. This fingerprint sequence was then compared one by one with the user's historical fingerprint database on the server. - If a hash fingerprint is found to be completely identical (Hamming distance = 0) or highly similar (Hamming distance < 5) to any historical record, a "video playback suspect" tag is generated; - Extract the matching fragment as evidence and send it along with the suspected flag to the teacher for review.

[0010] **3. Pace and Trajectory Analysis (Targeting Physics Cheating)** Calculate the total speed based on the GPS trajectory, plot the speed-time (vt) curve, and classify it according to the waveform shape: - If the speed curve shows a trapezoidal wave shape (rapid rise → long period of constant speed → rapid drop), a "suspected motor vehicle" label will be generated. - If the velocity curve shows a high-frequency sawtooth wave (peak interval 0.3~0.5 seconds), a "canine animal suspect" marker is generated; - If the speed curve is a horizontal straight line (with fluctuations of <0.3km / h throughout), a "suspected drone" label will be generated; At the same time, the GPS trajectory is matched with the local pre-stored open source map database (including bus routes). If the overlap between the trajectory and the bus route is greater than 80% and there are periodic stops (speed < 1 km / h, lasting 45~120 seconds), a "public transportation suspected" mark is generated.

[0011] **4. Voiceprint Recognition (Targeting Ambient Sound Cheating)** The microphone captures ambient audio throughout the process, extracts MFCC features, and compares them with a pre-stored cheating voiceprint template library: - Match bus / light rail announcement voice templates → "Public transport suspected" flag; - Match dog barking template (1~2kHz pulse sequence) → "Suspected of dog tying" flag; - Match the drone motor sound template (200~400Hz continuous sine wave) → "Drone Suspect" tag; - Match car engine sound template (50~150Hz low frequency) → "Suspected private car" mark; - Match the bicycle chain sound template (periodic metallic clanging sound) → "Bicycle Suspect" tag; When the matching degree is greater than 85%, a corresponding suspect tag is generated, and an audio clip is extracted as evidence.

[0012] **5. Camera Scene Recognition** Within the time periods corresponding to abnormal speed or abnormal voiceprint, extract the camera frame with the corresponding timestamp and identify the scene using a lightweight image classification model: - Identify features inside the bus (seats, handrails, pull rings) → Mark as "In-vehicle Scene"; - Identify bus stop features (bus stop sign, waiting shelter) → Mark "Bus Stop Scene"; - Non-motorized vehicle lane markings detected → “Non-motorized vehicle lane” mark; - Identified overpass / motor vehicle lane → "Motor vehicle lane" marker.

[0013] **6. Teacher Final Review Mechanism** The most crucial innovation of this invention lies in the teacher final review mechanism: - AI is only used as a tool for evidence collection and auxiliary judgment, and does not have any final decision-making power; - Regardless of the type of suspicion marker or confidence score generated by AI, all marked records must be sent to the teacher; Teachers can view the complete evidence package (GPS track, full video, full audio, AI analysis report) through the management backend and make independent judgments; - After confirming cheating, the teacher must enter the complete confirmation statement in English and press Enter before the score can be cleared. If a teacher believes that the AI ​​has made a mistake, they can choose to "reject the suspicion" and the score will remain valid.

[0014] **7. Home-School Collaboration Mechanism** Once the cheating is confirmed by the teacher, the system will automatically generate a cheating notification, which will be sent to the homeroom teacher and parents via WeChat, DingTalk, or email. The cheating record will be entered into the student's integrity file and used as a reference for comprehensive quality evaluation.

[0015] (III) Beneficial Effects Compared with the prior art, the present invention has the following advantages: 1. Low hardware dependency and low cost: It only relies on three standard smartphone sensors: GPS, camera, and microphone. No additional hardware is required, and it is suitable for all commercially available smartphones.

[0016] 2. Comprehensive coverage, impossible to bypass: Through multimodal cross-verification such as hash comparison, VT waveform analysis, voiceprint recognition, and scene recognition, it covers both physical cheating and root injection cheating. Cheats cannot bypass all detections by a single means.

[0017] 3. Excellent user experience: It adopts a post-processing batch processing architecture of "local data collection and cloud processing", without performing any real-time analysis during exercise, without consuming CPU computing power, and without affecting the running experience.

[0018] 4. Prevent AI Misjudgments: AI is only used as an auxiliary tool and does not have the final say. All suspected cases must be manually reviewed by teachers, thus eliminating the possibility of unfair punishment for students due to AI misjudgments.

[0019] 5. Grade reset is irreversible: Teachers are required to enter a complete English confirmation statement and press Enter to execute the reset operation, preventing accidental activation and ensuring the seriousness and traceability of the grade reset process.

[0020] 6. Dual constraints of technology and education: Home-school collaboration and integrity record keeping form a complete closed loop of technological discovery, evidence solidification, and educational intervention, effectively curbing cheating behavior. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall system architecture of the present invention.

[0022] Figure 2 This is a flowchart of the three-level processing of the present invention.

[0023] Figure 3 This is a schematic diagram of the VT waveform classification of the present invention.

[0024] Figure 4 This is a schematic diagram of the teacher review backend interface of the present invention. Detailed Implementation

[0025] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0026] I. Local Data Acquisition Phase When a user initiates a fitness recording request via a mobile device (such as a smartphone), the system automatically performs the following operations: 1. Request and obtain GPS location permission, camera permission, and microphone permission; 2. Enable the GPS module at a fixed sampling rate (e.g., 1Hz) and record latitude, longitude, and timestamp; 3. Use the front-facing camera at 480p resolution and 15fps to record ambient video; 4. Turn on the microphone at a 16kHz sampling rate and record ambient audio; 5. The above data is written to a local temporary file in real time, without any real-time analysis or uploading.

[0027] When the user actively ends the exercise or the system detects that the exercise has stopped for more than a preset time (such as 5 minutes), the system automatically ends data collection, packages the video, audio, GPS track, and device information (model, system version) into a compressed file, and stores it locally.

[0028] II. Cloud Data Upload and Preprocessing Stage After the exercise is completed, the system automatically uploads the compressed file to the cloud server in a Wi-Fi environment (or via cellular network after user confirmation).

[0029] After receiving the data packet, the server performs the following preprocessing: 1. Unzip the data package and verify file integrity; 2. Extract GPS track sequences, video files, audio files, and device information; 3. Convert all video files to 480p resolution (if the original resolution is higher than 480p) to standardize subsequent processing; 4. Align GPS, video, and audio data by timestamp to create a unified timeline index for easy cross-validation later.

[0030] III. AI-Assisted Detection Stage This stage is executed by a cloud-based AI server. The AI ​​is only responsible for detection and labeling, and does not make any final judgment. All detection results are output in the form of "suspect marker + evidence package".

[0031] **(1) Hash value comparison (detecting video playback)** 1. Extract keyframes from the preprocessed video at a rate of 1 frame per second; 2. Calculate the perceptual hash value (pHash) for each frame to generate a hash fingerprint sequence for that motion; 3. Compare the fingerprint sequence with all historical fingerprints of the user in the server database one by one; 4. If a hash fingerprint is found to be completely identical (Hamming distance = 0) or highly similar (Hamming distance < 5) to any historical record, a "video playback suspect" flag is generated, and the matching segment is extracted as evidence.

[0032] **(2) Pace and trajectory analysis (detecting physical cheating)** 1. Based on GPS latitude and longitude sequences and timestamps, calculate the average velocity v = Δs / Δt every 30 seconds, and generate a velocity-time series; 2. Perform waveform analysis on the sequence: - If the speed curve shows a trapezoidal wave shape (rapid rise → long period of constant speed → rapid drop), a "suspected motor vehicle" label will be generated. - If the velocity curve shows a high-frequency sawtooth wave (peak interval 0.3~0.5 seconds), a "canine animal suspect" marker is generated; - If the speed curve is a horizontal straight line (with fluctuations of <0.3km / h throughout), a "suspected drone" label will be generated; 3. Match the GPS trajectory with the local pre-stored open-source map database (including bus routes). If the overlap between the trajectory and the bus route is greater than 80% and there are periodic stops (speed < 1 km / h, lasting 45~120 seconds), generate a "public transportation suspected" marker.

[0033] **(3) Voiceprint recognition (detecting ambient sound cheating)** 1. Perform frame segmentation on the audio stream (frame length 25ms, frame shift 10ms) and extract MFCC features; 2. Compare the features with the pre-stored cheating voiceprint template library: - Match bus / light rail announcement voice templates → "Public transport suspected" flag; - Match dog barking template (1~2kHz pulse sequence) → "Suspected of dog tying" flag; - Match the drone motor sound template (200~400Hz continuous sine wave) → "Drone Suspect" tag; - Match car engine sound template (50~150Hz low frequency) → "Suspected private car" mark; - Match the bicycle chain sound template (periodic metallic clanging sound) → "Bicycle Suspect" tag; 3. When the matching degree is greater than 85%, a corresponding suspect tag is generated, and an audio segment is extracted as evidence.

[0034] **(4) Camera Scene Recognition** 1. Extract camera frames with the corresponding timestamps during the time periods corresponding to abnormal speed or abnormal voiceprint. 2. Identify scene elements using lightweight image classification models (such as MobileNet): - Identify features inside the bus (seats, handrails, pull rings) → Mark as "In-vehicle Scene"; - Identify bus stop features (bus stop sign, waiting shelter) → Mark "Bus Stop Scene"; - Non-motorized vehicle lane markings detected → “Non-motorized vehicle lane” mark; - Identified overpass / motor vehicle lane → "Motor vehicle lane" marker.

[0035] IV. Teacher Final Review Stage **(1) Review Task Push** For all records flagged as "suspicious" by AI, the system automatically generates a review task in the teacher management backend, which includes the following information: - Student's name, class, and exercise time; - List of suspected AI markers (multiple items allowed); - Complete evidence package: GPS track map, complete video, audio clips, AI analysis report; - AI confidence score (for reference only).

[0036] **(2) Teacher Review Process** Teachers can view the complete evidence through the management backend and make independent judgments: - If the teacher confirms cheating: proceed with the grade reset confirmation process; - If the teacher believes that the AI ​​has made a mistake: select "Reject Suspected", the grade will remain valid, and the system will record the reason for rejection.

[0037] **(3) Grade Reset Confirmation Mechanism** After the teacher confirms the cheating, the system pops up a confirmation window displaying the following English confirmation statement: *"I have reviewed the evidence and confirm that this student cheated. I understand that this will invalidate the run."* Teachers must manually enter the above English statement in its entirety (copy and paste must be disabled in the input box), and press Enter to submit after completion. Once the system verifies that the input matches perfectly, it will reset the grade to zero and record the teacher's account, submission time, and IP address as part of the audit log.

[0038] V. Follow-up Processing Stage **(1) Home-school collaboration** After the grades are cleared and confirmed, the system will automatically generate a cheating notification and push it to the homeroom teacher and parents via WeChat, DingTalk or email. The notification will include the time of cheating, the method of cheating, and the result of the handling.

[0039] **(2) Integrity Records** Cheating records will be entered into students' integrity files and used as a reference for comprehensive quality evaluation.

[0040] **(3) Data retention and cleanup** All raw data (including video, audio, and GPS tracks) will be stored in the cloud for 30 days. The data will be automatically deleted after 30 days if there is no appeal or verification request.

Claims

1. A motion cheating detection system based on multimodal perception, characterized in that, include: The local acquisition module is used to synchronously acquire GPS trajectory data, video data, and audio data through the GPS module, camera module, and microphone module of the mobile terminal during exercise; The cloud processing module is used to receive data packets uploaded by the local acquisition module, and perform hash comparison, speed analysis and voiceprint recognition on the data packets to generate suspect tags and corresponding evidence packages; The teacher final review module is used to receive the suspect markers and evidence packages pushed by the cloud processing module, which are then manually reviewed by the teacher to make a final judgment. The cloud processing module does not have the final decision-making authority; all marked records must be pushed to the teacher final review module.

2. The system according to claim 1, characterized in that, The camera module records ambient video at 480p resolution and 15fps, and the microphone module records ambient audio at a 16kHz sampling rate.

3. The system according to claim 1, characterized in that, The hash comparison includes: extracting keyframes from the video, calculating the perceptual hash value (pHash) of each frame, generating a hash fingerprint sequence, and comparing the hash fingerprint sequence with the user's historical fingerprint database. If the Hamming distance is less than 5, a "video playback suspected" tag is generated.

4. The system according to claim 1, characterized in that, The speed analysis includes: calculating the speed-time curve based on the GPS trajectory data, classifying it according to the waveform shape; if the speed curve presents a trapezoidal wave shape, a "suspected motor vehicle" mark is generated; if it presents a high-frequency sawtooth wave, a "suspected canine" mark is generated; and if it presents a horizontal straight line, a "suspected drone" mark is generated.

5. The system according to claim 1, characterized in that, The voiceprint recognition includes: extracting the MFCC features of the audio data, comparing them with a pre-stored cheating voiceprint template library, and generating a corresponding suspect label when the matching degree is greater than 85%. The cheating voiceprint template library includes at least bus stop announcement sound templates, dog barking sound templates, drone motor sound templates, car engine sound templates, and bicycle chain sound templates.

6. The system according to claim 1, characterized in that, The cloud processing module also includes a scene recognition submodule, which is used to extract camera frame images with corresponding timestamps during periods of abnormal speed or sound patterns, and to identify scene images through an image classification model. The scene images include at least the interior of a bus, a bus stop, a non-motorized vehicle lane, and a motorized vehicle lane.

7. The system according to claim 1, characterized in that, The teacher final review module includes a mandatory confirmation submodule. When a teacher confirms cheating, the mandatory confirmation submodule pops up a confirmation window, displays a preset English statement, and requires the teacher to manually enter the English statement completely and press the Enter key before the score can be cleared.

8. The system according to claim 1, characterized in that, It also includes a home-school collaboration module, which automatically sends a notification to the homeroom teacher and parents after the cheating is confirmed by the teacher, and records the cheating in the student's integrity file.

9. A motion cheating detection method based on multimodal perception, characterized in that, Includes the following steps: Local data acquisition steps: During the exercise, GPS trajectory data, video data, and audio data are simultaneously acquired through the GPS module, camera module, and microphone module of the mobile terminal; Cloud processing steps: Receive the uploaded data packet, perform hash comparison, pace analysis and voiceprint recognition, and generate a suspect marker and corresponding evidence package; Teacher's final review steps: Upon receiving the suspected markers and evidence package, the teacher will conduct a manual review and make a final judgment; The cloud processing steps do not generate a final judgment result; all marked records must proceed to the teacher's final review step.

10. The method according to claim 9, characterized in that, In the teacher's final review process, when a teacher confirms cheating, they must manually enter a pre-set English sentence and press Enter before the score can be cleared.

Images