A Time-Constrained Intelligent Detection Method and System for Firearm Inspection Red Line Actions

By combining a lightweight dual YOLO model and FFmpeg streaming decoding with a finite state machine approach, intelligent detection of red-line actions during gun inspection was achieved. This solves the problems of high cost and false alarms/missed alarms associated with manual supervision in existing technologies, forming an efficient and reliable electronic supervision system and reducing security risks in the gun inspection process.

CN123071891APending Publication Date: 2026-11-10JIASHAN LITONG INFORMATION & TECH CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611215691.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-11
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing technologies rely on manual supervision during the gun inspection process, which results in high computational overhead, high false alarm and false alarm rates, lack of time constraints and digital supervision capabilities, inability to effectively identify red-line actions in gun inspection, and lack of anti-shake and time smoothing mechanisms, leading to unreal-time and inaccurate supervision.

Method used

A lightweight dual YOLO model is used to detect key firearm components and hand targets. Combined with FFmpeg streaming decoding and a multi-threaded asynchronous decoupled architecture, the timing stages of the firearm inspection process are bound by a finite state machine. Combined with sliding window timing smoothing and spatial geometric constraints, the red line action is intelligently determined, and standardized events are pushed in real time through SSE long connections.

Benefits of technology

It achieves low-latency and high-efficiency detection of red-line actions in firearm inspection, reduces hardware deployment costs, improves the accuracy and reliability of detection, and creates electronic training records, ensuring the safety and digital management of the firearm inspection process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123071891A_ABST
    Figure CN123071891A_ABST
Patent Text Reader

Abstract

This invention relates to the field of intelligent video surveillance and discloses an intelligent detection method and system for gun inspection red line actions based on time-series constraints. The method includes the following steps: S1, acquiring the video stream of the officer inspecting the gun, and obtaining BGR format images by decoding frame by frame using FFmpeg. In this invention, a lightweight dual YOLO model is used to perform local detection only on key parts of the gun and hand targets. Combined with FFmpeg streaming decoding and a multi-threaded asynchronous decoupled architecture, the processing time for a single frame can be controlled within 43ms. A single ordinary server can stably support the concurrent real-time analysis of 8 high-definition video streams, with a warning latency as low as 118ms, significantly reducing the hardware deployment threshold and computing power cost. Simultaneously, by binding the gun inspection process time-series stages with a finite state machine, and superimposing a five-layer verification mechanism—mutually exclusive target selection, gun physical rule correction, sliding window time-series smoothing, continuous frame anti-shake counting, and spatial geometric constraints—the compliance differences of the same action at different stages are effectively distinguished.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent video surveillance, and in particular to an intelligent detection method and system for gun inspection red line actions based on time-series constraints. Background Technology

[0002] In integrated policing and counter-terrorism training bases, standardized gun inspection before Type 92 pistol shooting training is a core pre-processing step to prevent accidental discharges and eliminate firearms safety incidents. Currently, supervision of violations during the gun inspection process mainly relies on a full-time on-site supervision model by human instructors. Under this model, the training base needs to assign dedicated instructors to supervise participating officers one-on-one or in groups, visually observing the entire gun inspection process, identifying violations such as fingers entering the trigger guard or the muzzle pointing at the hand, and verbally correcting any deviations on the spot.

[0003] With the development of artificial intelligence technology, some existing technologies have proposed visual detection schemes based on full-frame comparison of standard action databases. However, these schemes require feature matching with massive standard samples frame by frame, resulting in extremely high computational overhead and poor real-time performance for multi-channel concurrency. They only compare visual features of the image and do not establish stage constraints in the gun inspection process, making it impossible to distinguish the compliance differences of the same action at different stages, leading to extremely high false alarm and false negative rates. At the same time, they lack anti-shake and time-series smoothing mechanisms, and alarms are triggered by single-frame detection deviations, resulting in weak anti-interference capabilities. In addition, the scheme does not link the detection of subdivided parts of the gun, nor can it automatically capture violation frames, generate standardized event logs, or push warning messages in real time, lacking complete digital supervision and evidence collection capabilities. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an intelligent detection method and system for red line action in firearm inspection based on time-series constraints, thus solving the above problems.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an intelligent detection method for the red line action of firearm inspection based on time-series constraints, comprising the following steps: S1. Obtain the video stream of the police officer inspecting the gun, and decode it frame by frame using FFmpeg to obtain a BGR format image; S2. Each frame of image is input in parallel into the general YOLO target detection model and the OBB rotating frame detection model for joint inference. The general YOLO target detection model is used to identify 11 types of targets, including hands, magazines, grips, safety open / closed states, slide extension / retraction states, whether fingers are inside the guard ring, and hammer ready / release states. The OBB rotating frame detection model is used to detect the slide and output the coordinates of the four vertices of the tilt detection frame. S3. Perform three-layer filtering optimization on the inference results: retain the detection box with the highest confidence for the same type of target, filter each group of mutually exclusive categories to retain only the optimal state, and correct the detection results based on the physical rules of firearms; S4. Based on the sliding window, perform multi-frame voting timing smoothing on the states of safety, sleeve, finger, and hammer, and combine spatial geometric constraints to determine the red line action. The spatial geometric constraints include: constructing a muzzle ray based on the sleeve OBB tilt detection frame and detecting the intersection with the hand detection frame to determine that the muzzle is pointing at the hand; and constructing a dangerous rectangular area at the tail of the sleeve based on the sleeve OBB tilt detection frame and calculating the intersection-union ratio with the hand detection frame to determine whether the hand is located directly behind the sleeve. S5. Based on the finite state machine, the gun inspection process is constructed and the standardized gun inspection process is divided into eight stages: disassembling the magazine, disengaging the safety, first slide pull, second slide pull, aiming and firing at a safe area, turning the hammer from the released state to the ready-to-fire state, disengaging the safety, and magazine reset. Each stage is independently configured with a set of prohibited red line actions. Only when the red line action corresponding to the current stage is detected and the number of frames of the action exceeds the preset anti-shake threshold is it judged as a valid violation. S6. Perform phased deduplication on red line actions that are determined to be valid violations. In the same phase, the same red line action is only triggered for the first time to automatically capture the violation frame, encode and upload it to the MinIO object storage, and generate an accessible image URL as a violation certificate. S7. Push standardized events to the front end in real time via SSE long connection. The standardized events include stage start, stage completion, red line action warning, heartbeat keep-alive, full process completion and video stream end events.

[0006] Preferably, the eight prohibited actions in the red line include: finger entering the trigger guard, unauthorized disengagement of the safety, unauthorized disengagement of the safety, unauthorized pulling of the slide, moving the hammer from the released state to the ready-to-fire state, moving the hammer from the ready-to-fire state to the released state, hand being directly behind the slide, and muzzle pointing at the hand.

[0007] Preferably, the correction of the detection results based on the physical rules of the firearm includes: when the slide is detected to be in a stretched state, the safety state is forcibly corrected to the open state; the mutual exclusion category screening includes: for four mutually exclusive detection categories, safety open / closed, slide extension / retraction, finger in / out of the guard ring, and hammer ready / release, only the single state with the highest confidence is retained for each group.

[0008] Preferably, the construction of the dangerous rectangular region at the tail of the sleeve based on the backward extension of the sleeve OBB tilt detection frame specifically involves: taking the side length of the sleeve OBB tilt detection frame connecting the muzzle edge and the tail edge as a reference, extending one-third of the side length backward from the tail edge in a direction away from the muzzle to generate the dangerous rectangular region at the tail; when the intersection-exclusion ratio of the dangerous rectangular region at the tail and the hand detection frame is not less than 0.5, it is determined that the hand is located directly behind the sleeve.

[0009] Preferably, in the timing control of the gun inspection process based on the finite state machine, the completion judgment logic for each stage includes: the magazine disassembly stage is determined by the state change sequence of the intersection-union ratio of the magazine detection frame and the grip detection frame changing from zero to greater than zero and then back to zero; the safety disengagement stage is determined by the switching judgment of the safety state changing from closed to open; the slide pulling stage is determined by the state change sequence of the slide detection category going from unstretched to stretched and then back to unstretched; the aiming and firing stage is determined by the combination of the finger entering the guard and the hammer changing from the ready-to-fire state to the released state; the hammer cocking stage is determined by the switching judgment of the hammer changing from the released state to the ready-to-fire state; the safety disengagement stage is determined by the switching judgment of the safety state changing from open to closed; and the magazine reset stage is determined by the state change sequence of the intersection-union ratio of the magazine detection frame and the grip detection frame changing from zero to greater than zero and then back to zero.

[0010] A time-constrained intelligent detection system for red-line actions in firearm inspection includes: The video decoding module is used to decode the police officer's gun inspection video stream frame by frame based on FFmpeg and output a BGR format image; The AI ​​inference module includes a general YOLO target detection submodule and an OBB rotating frame detection submodule. The general YOLO target detection submodule is used to identify 11 types of targets in each frame of image, including hands, magazines, grips, safety open / closed states, slide extension / retraction states, whether fingers are inside the guard ring, and hammer ready / release states. The OBB rotating frame detection submodule is used to detect the slide and output the coordinates of the four vertices of the tilt detection frame. The result filtering module is used to perform three-layer filtering optimization on the reasoning results: selection of similar targets, screening of mutually exclusive categories, and correction of gun physical rules. The timing smoothing and geometric verification module is used to perform multi-frame voting timing smoothing of the states of the safety, sleeve, finger, and hammer based on a sliding window, and to construct the muzzle ray and dangerous rectangular area at the tail of the sleeve based on the sleeve OBB tilt detection box. Spatial geometric hazard is determined by ray intersection detection and intersection-union ratio calculation. The finite state machine management module is used to divide the standardized gun inspection process into eight stages, each stage is independently configured with a set of prohibited red line actions, and the red line actions are judged in stages by combining the anti-shake frame count. The violation evidence collection module is used to deduplicat valid violation actions in stages. In the same stage, the same red line action is only triggered for the first time to automatically capture the violation frame and upload it to the MinIO object storage to generate violation evidence. The message push module is used to push standardized events, including stage start, stage completion, red line action warning, heartbeat keep-alive, full process completion and video stream end, to the front end in real time via SSE long connection.

[0011] Preferably, the finite state machine management module has a built-in dictionary of current stage variables and action completion markers. The current stage variables flow sequentially between the eight stages as the gun inspection process progresses, and the action completion marker dictionary is used to record the completion status of each stage. Each stage independently maintains a set of violation actions to achieve deduplication alarms for red line actions of the same type in the same stage.

[0012] Preferably, in the timing smoothing and geometric verification module, the muzzle ray is constructed as follows: taking the midpoint of the sleeve OBB tilt detection frame near the tail as the starting point and the midpoint of the side near the muzzle as the direction reference point, a ray is generated that penetrates the sleeve. If the ray intersects with the hand detection frame, it is determined that the muzzle is pointing at the hand.

[0013] Preferably, it also includes a background asynchronous thread scheduling module, which creates a heartbeat keep-alive thread and a detection main thread. The heartbeat keep-alive thread pushes heartbeat packets to the message queue at a preset period to maintain the SSE long connection. The detection main thread independently executes video decoding, AI inference, state machine judgment and violation evidence collection logic, so as to realize the parallel operation of video decoding, AI inference, message push and file storage.

[0014] Preferably, the red line action warning event in the standardized event carries a task identifier, timestamp, current gun inspection stage number, red line action type code, and access URL of the violation frame image stored in MinIO; the message push module builds an SSE server based on the FastAPI framework, caches the events to be pushed through the message queue, and the event generator reads and encapsulates them into a standard SSE data stream for continuous push to the front end, and automatically disconnects the long connection after the task is completed.

[0015] Beneficial effects This invention provides a method and system for intelligent detection of red-line actions in firearm inspection based on time-series constraints. Compared with existing technologies, it has the following advantages: 1. In this invention, a lightweight dual YOLO model is used to perform local detection only on key components of firearms and hand targets. Combined with FFmpeg streaming decoding and a multi-threaded asynchronous decoupling architecture, the processing time for a single frame can be controlled within 43ms. A single ordinary server can stably support the concurrent real-time analysis of 8 high-definition video streams, with a warning latency as low as 118ms, significantly reducing the hardware deployment threshold and computing power cost. At the same time, by binding the timing stages of the firearm inspection process with a finite state machine, and superimposing a five-layer verification mechanism of mutually exclusive target selection, firearm physical rule correction, sliding window timing smoothing, continuous frame anti-shake counting, and spatial geometric constraints, the compliance differences of the same action at different stages are effectively distinguished, and invalid alarms caused by changes in light, camera shake, hand occlusion, and instantaneous model deviation are filtered out. The overall false alarm rate is only 1.07%, and false alarms are extremely rare in complex interference scenarios, significantly improving the reliability and accuracy of real-time risk warnings at police training sites. 2. In this invention, machine judgment completely replaces the supervision of human instructors, which not only significantly frees up police resources and reduces labor costs, but also completely eliminates management blind spots such as the reluctance of participating police officers to correct deviations and the failure of supervision caused by the higher rank of the participating police officers than the on-site instructors. This ensures that all violations are detected in real time without discrimination. When a valid violation is detected, the system automatically captures the violation frame and uploads it to the MinIO distributed storage, simultaneously recording the timestamp, the gun inspection stage, the red line action type code, and the accessible image URL, forming an immutable electronic training file for easy review, tracing, and assessment. Based on SSE long connections, structured and standardized events are continuously pushed to the front end, which can be directly connected to the police training management platform to realize real-time monitoring on a large screen and digital management of police training data statistics. The entire solution relies on general open-source components, is lightweight and easy to deploy, and can accurately identify high-risk red line actions such as pointing the gun muzzle at the hand or placing the hand directly behind the slide from the source, stopping dangerous operations in time and effectively reducing the probability of accidental discharge and personnel injury during the inspection of the Type 92 modified pistol. Attached Figure Description

[0016] Figure 1 This is a system framework diagram of the intelligent detection method and system for red line action of firearm inspection based on time constraints proposed in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1The present invention provides the following technical solutions, specifically including the following embodiments: Example 1: System Hardware and Software Deployment; This embodiment provides a complete deployment scheme for a time-constrained intelligent detection system for red-line actions in firearm inspection.

[0019] 1. Hardware Environment Configuration The following hardware is deployed at the standard gun inspection station in the comprehensive police combat training base: Video capture terminal: A high-definition network camera is installed 2.2m above each gun inspection station, with a frame rate of 25fps and a resolution of 1920×1080, providing complete field of view to cover the officer's gun handling and the entire gun inspection process; A single inference business server connects to 8 cameras for concurrent processing; Inference business server: Configured with an Intel Xeon processor, 128GB of memory, and 2×RTX 4090 (24GB of video memory); Storage component: Deployed MinIO object storage service, with a 1TB hard drive per node, and a dedicated storage bucket named gun-images-bucket is created for persistent storage of screenshots of illegal actions; Client equipment: A 27-inch monitoring screen and a Windows instructor management terminal, with the browser accessing the FastAPI inference service via an SSE long connection.

[0020] 2. Software Environment Deployment Operating System: Ubuntu 20.04 LTS; Video Decoding Tool: FFmpeg 6.1.6, replacing OpenCV's native reading method, parsing the video stream frame by frame; AI Inference Framework: PyTorch Version 2.6.0 deploys two lightweight YOLO models: a general YOLO target detection model that identifies 11 target categories, including hand, magazine, grip, safety on, safety off, slide extension, slide reset, finger in / trigger guard, finger out / trigger guard, hammer on, and hammer off, with an inference confidence threshold of 0.25; and an OBB rotating bounding box YOLO model specifically for slide tilt detection, outputting the coordinates of the four vertices of the slide detection bounding box. Backend services: FastAPI 0.135.3, using an asynchronous multi-threaded architecture; storage services: MinIO RELEASE.2024-10-29T16-01-48Z; frontend display: a Vue3 visual monitoring page that receives SSE standardized event pushes and renders them in real time.

[0021] 3. On-site implementation process (1) The camera is installed at a height of 2.2m above the gun inspection station to ensure full coverage of the police officer holding the gun and the entire 8-stage gun inspection process; (2) Deploy a complete set of services for decoding, AI inference, storage, and message push on the server, and start the heartbeat thread (45-second cycle keep-alive) and the detection main thread; (3) Initialize the StateManager finite state machine, load the red line action judgment rules corresponding to the 8 gun inspection stages, set the sliding window timing smoothing frame number SMOOTH_FRAMES=3, and set the anti-shake counting threshold to 3 consecutive frames. (4) Police officers conduct standardized 92 modified pistol inspection operations, with the camera streaming the process in real time and the system automatically completing the entire inspection process. (5) When a violation occurs, automatically capture a PNG image for evidence collection and store it in MinIO, and generate a standard HTTP image access URL; (6) The monitoring screen displays the current gun inspection stage, violation alarms, and evidence screenshots in real time. Electronic training files are automatically generated when training ends.

[0022] Example 2: Operation of a complete standardized gun inspection process; This embodiment uses a single police officer completing an 8-stage standardized gun inspection operation as a test sample to explain the complete operation process of the system in detail.

[0023] Step 1: Video Stream Decoding FFmpeg parses the 1080P video stream frame by frame at 25fps, and outputs BGR format images which are then fed into the dual YOLO models for parallel inference.

[0024] Step 2: Dual-model inference and three-layer detection filtering Perform the following processing on each frame of the image: (1) General YOLO inference: Output the detection boxes and confidence scores for 11 types of targets, including hand, magazine, grip, safety, slide, finger, and hammer; (2) OBB model inference: Output the coordinates of the four vertices of the sleeve tilt detection box for subsequent spatial geometry calculations; (3) Three-layer filtering optimization: Optimal selection for similar targets: Only the single detection box with the highest confidence is retained within the same category to eliminate duplicate detections; Mutually exclusive category filtering: For the four mutually exclusive targets, namely safety_on / safety_off, sleeve extension / retraction, finger in / out of the guard ring, and hammer on / off, only the optimal state with the highest confidence is retained for each group; Gun physics rule correction: If the slide is detected to be in a stretched state (muzzle_on), the safety state is forcibly corrected to open (safety_on) to eliminate model logic conflicts.

[0025] Step 3: Temporal smoothing and spatial geometric verification (1) Temporal smoothing: Construct a sliding window buffer with a length of 3 frames, and perform multi-frame voting smoothing on the states of safety, sleeve, finger, and hammer to eliminate instantaneous misidentification in a single frame; (2) Spatial geometric calculations: The IOU (Intersection over Union) of the magazine and grip detection frame is calculated to determine the magazine's installation / removal status. Based on the four vertices of the sleeve OBB tilt detection frame, construct the dangerous rectangular region at the tail of the sleeve: taking the side length connecting the muzzle edge and the tail edge as the reference, extend one-third of the side length backward from the tail edge in the direction away from the muzzle to generate the dangerous rectangle at the tail; calculate the IOU between this rectangle and the hand detection frame. If the IOU ≥ 0.5, it is determined that the hand is located directly behind the sleeve. Constructing the muzzle ray: Starting from the midpoint of the side of the sleeve OBB frame closest to the tail, a ray is generated with the midpoint of the side closest to the muzzle as the direction reference point. If the ray intersects with the hand detection frame, it is determined that the muzzle is pointing at the hand. (3) Anti-shake counting: Only when the red line action corresponding to the current stage is detected and the action lasts for more than 3 frames, it is judged as a valid violation.

[0026] Step 4: Finite state machine phased control StateManager has a built-in `current_step` variable to record the current verification stage and a built-in `action_done` dictionary to record the completion markers for each stage. Each stage is independently configured to prohibit red-line actions, and for the same violation in the same stage, only the first alert is triggered for evidence collection. Stage transitions and red-line action constraints are as follows:

[0027] Step 5: Evidence Collection and Message Push For example, in Phase 1 (disassembling the magazine), a police officer's finger enters the trigger guard: (1) The system determines that the action is a new violation in the current stage, captures the current video frame, encodes it as a PNG image, and uploads it to the MinIO storage bucket with the name taskId_frameId.png; (2) Encapsulate the SSE violation event, carrying the timestamp, stage number 1, red line action type code, and MinIO image access URL; (3) Push the alert to the monitoring screen in real time via SSE long connection and retain electronic credentials.

[0028] Step 6: Process Closure After all 8 gun verification stages are completed, an all_completed event is pushed; after the video stream is finished reading, a video_finish event is pushed, releasing the FFmpeg decoding process, memory, and thread resources, and the task ends.

[0029] Example 3: Detailed breakdown of typical violation scenarios; This embodiment provides six typical test scenarios to verify the system's detection performance under different violation and interference scenarios.

[0030] Scenario 1: Finger enters the trigger guard during magazine disassembly. The officer's finger entered the trigger guard prematurely before the magazine was completely removed. The system smoothly confirmed the finger's entry into the guard state through a 3-frame sliding window timing. After three consecutive frames of detection, it was determined to be a valid violation. The system automatically captured the current frame, encoded it, and uploaded it to MinIO. The front-end monitoring screen displayed a real-time pop-up alarm, recording the violation timestamp, stage number, and image URL.

[0031] Scenario 2: During the magazine unloading phase, the muzzle is pointed at the user's hand. The OBB tilt detection frame generates a muzzle ray. When the ray intersects with the hand detection frame, an alarm is triggered after three consecutive frames. The system automatically saves the evidence image of the dangerous posture, and the front end displays a real-time pop-up message "Stage 1 - Muzzle Pointing at Hand," while simultaneously pushing an SSE violation event.

[0032] Scenario 3: The safety switch is engaged prematurely during the second pull of the sleeve. The FSM is currently in Phase 4, during which disabling safety is prohibited. When the system detects a switch from safety_on to safety_off, it immediately determines it as a valid violation, captures the violation frame, uploads and stores it, and issues a real-time pop-up alert on the front end, without requiring manual confirmation.

[0033] Scenario 4: During the aiming and firing phase, the hand is positioned directly behind the slide. The calculated IOU value between the dangerous rectangular area extending from the sleeve's tail and the hand detection frame is 0.62, which is greater than the 0.5 threshold. The system determines that the hand is located directly behind the sleeve, touching the red line. An evidence image is generated, and an alarm is pushed, recording stage number 5 and the violation type code.

[0034] Scenario 5: Dim lighting and slight camera shake cause single-frame misidentification. The training environment was dimly lit and the camera was slightly shaky. In one frame, a finger was detected entering the protective circle, but the stabilization threshold of 3 consecutive frames was not reached. The system filtered out this momentary false detection, without triggering an alarm, capturing an image, or pushing an SSE event, thus verifying the effectiveness of the timing smoothing and stabilization mechanism.

[0035] Scenario 6: The same violation occurs repeatedly during the same period. During the magazine disassembly phase, the officer twice inserted his fingers into the protective ring. The system only triggered a violation alert for the first instance of image capture; for the second instance of the same violation, the system only logged the information and did not generate duplicate evidence images or issue repeated alerts, effectively reducing redundant storage and front-end interference.

[0036] Example 4: System performance and recognition accuracy verification; 1. Test Environment The hardware and software testing environment remained consistent with Example 1. The test dataset relied on standardized gun inspection video streams of the Type 92 modified pistol collected on-site at the comprehensive police combat training base, all of which were actual footage of police officers operating the equipment. The total sample size was 120 complete gun inspection process videos, each covering 8 standard stages, with a total duration of 180 minutes. The samples were categorized into 60 sets of standard compliant operations and 60 sets of red-line violation actions. Violation action annotations included 1120 valid red-line actions manually reviewed and annotated, covering all 8 categories of red-line actions. Interference samples included 60 sets of interference scene videos, covering complex environments such as low light and backlight, hand obstruction of the lens, slight camera shake, and partial obstruction by personnel limbs.

[0037] 2. Real-time performance and computing resource testing A continuous 3-hour concurrent test of 8 video streams was conducted, and the processing time of the complete link (FFmpeg decoding → dual YOLO inference → result filtering → temporal geometry calculation → FSM determination) was statistically analyzed: Average processing time per frame: 43ms; Maximum processing time per frame: 50ms. Average GPU memory usage: 5.5GB, peak 7.5GB; Average CPU utilization: 50%, peak 68%; Average warning latency (from the entry of inappropriate content into the camera to the front-end pop-up): 118ms; Maximum warning latency: 190ms.

[0038] 3. Action recognition accuracy test (1) Overall false alarm rate: 1,120 valid violations were manually verified, and the system successfully identified and triggered 1,108 evidence collection alarms, resulting in 12 false alarms and an overall false alarm rate of 1.07%. The false alarm scenarios were all extremely severe occlusions (the police officer's hands completely covered all key components of the sleeve, safety, and hammer for more than 5 frames), which belong to the limit of visual recognition scenarios.

[0039] (2) False alarms in complex interference scenarios: 60 sets of interference samples totaled 135,000 frames, and the system triggered 11 invalid false alarms. The system’s built-in triple verification mechanism of 3-frame sliding window timing smoothing, continuous 3-frame anti-shake counting, and gun physical rules effectively filtered out invalid alarms caused by light and shaking.

[0040] (3) Phased deduplication of repeated violations: 30 test videos were selected. The same violation was triggered 2 to 5 times in the same phase within each group. The system only captured the image and pushed an alarm on the first time, reducing a total of 450 redundant evidence images.

[0041] 4. Scenario-specific verification Individual sampling tests were conducted on the eight core red-line actions, with 140 real action samples drawn for each category. The identification results are as follows:

[0042] 5. System stability test A 3-hour uninterrupted stability test was conducted with 8 video streams continuously: system crashes and service interruptions occurred 0 times; FFmpeg decoding process exited abnormally 0 times; SSE long connection disconnection and reconnection occurred 0 times (only relying on 45-second periodic heartbeats for survival); memory leaks: no continuous memory increase or memory overflow issues occurred after 3 hours of operation.

[0043] Example 5: Background asynchronous thread decoupling architecture; The server creates two independent daemon threads when it starts up: 1. Heartbeat keep-alive thread (heartbeat_worker): Pushes heartbeat packets independently to the message queue every 45 seconds, encapsulates them as SSE heartbeat events and continuously pushes them to the front end to prevent SSE long connections from being disconnected due to idle timeout.

[0044] 2. Main detection thread (gun_check_detection): This thread fully executes video decoding, AI inference, FSM judgment, and violation evidence collection logic. It runs in parallel with the heartbeat thread and message push thread, without blocking each other.

[0045] 3. Message Generator (event_generator): Loops through the message queue, reads events to be pushed, encapsulates them in a standard SSE data stream format (data: {...}), and continuously pushes them to the front end via FastAPI's StreamingResponse. Upon completion of the task, it automatically sends a video_finish event and disconnects the persistent connection.

[0046] 4. Parallel processing flow: Video decoding (FFmpeg) → AI inference (dual YOLO parallel) → Result filtering → Temporal smoothing and geometric verification → FSM stage judgment → Violation evidence collection (MinIO upload) → Message enqueueing → SSE push. Each step is decoupled through asynchronous queues to ensure real-time processing of multiple video streams concurrently.

[0047] In summary, through system deployment, full-process operation, typical scenario testing, performance and accuracy verification, and backend architecture verification in Examples 1 to 5, it is demonstrated that this invention can stably support real-time concurrent processing of multiple video streams in a normal server hardware environment, with a single frame processing time as low as 43ms and a warning latency controlled within 118ms. By leveraging finite state machine-based phased timing control, multi-level timing smoothing, and collaborative verification of spatial geometric constraints, the overall false alarm rate is only 1.07%, with very few false alarms in complex interference scenarios. The recall rate for the eight core red-line actions all exceeds 87.8%, and there were zero crashes and zero memory leaks during three hours of high-load operation. This solution achieves fully automated, standardized, and intelligent supervision of the gun inspection process with low computational overhead, completely eliminating subjective biases and hierarchical barriers inherent in manual supervision. Simultaneously, through automatic evidence collection and real-time SSE push, a complete digital training archive is formed, significantly improving the gun safety management level of police training bases, demonstrating good practicality and promotional value.

[0048] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A time-constrained intelligent detection method for red-line action in firearm inspection, characterized in that, Includes the following steps: S1. Obtain the video stream of the police officer inspecting the gun, and decode it frame by frame using FFmpeg to obtain a BGR format image; S2. Each frame of image is input in parallel into the general YOLO target detection model and the OBB rotating frame detection model for joint inference. The general YOLO target detection model is used to identify 11 types of targets, including hands, magazines, grips, safety open / closed states, slide extension / retraction states, whether fingers are inside the guard ring, and hammer ready / release states. The OBB rotating frame detection model is used to detect the slide and output the coordinates of the four vertices of the tilt detection frame. S3. Perform three-layer filtering optimization on the inference results: retain the detection box with the highest confidence for the same type of target, filter each group of mutually exclusive categories to retain only the optimal state, and correct the detection results based on the physical rules of firearms; S4. Based on the sliding window, perform multi-frame voting timing smoothing on the states of safety, sleeve, finger, and hammer, and combine spatial geometric constraints to determine the red line action. The spatial geometric constraints include: constructing a muzzle ray based on the sleeve OBB tilt detection frame and detecting the intersection with the hand detection frame to determine that the muzzle is pointing at the hand; and constructing a dangerous rectangular area at the tail of the sleeve based on the sleeve OBB tilt detection frame and calculating the intersection-union ratio with the hand detection frame to determine whether the hand is located directly behind the sleeve. S5. Based on the finite state machine, the gun inspection process is constructed and the standardized gun inspection process is divided into eight stages: disassembling the magazine, disengaging the safety, first slide pull, second slide pull, aiming and firing at a safe area, turning the hammer from the released state to the ready-to-fire state, disengaging the safety, and magazine reset. Each stage is independently configured with a set of prohibited red line actions. Only when the red line action corresponding to the current stage is detected and the number of frames of the action exceeds the preset anti-shake threshold is it judged as a valid violation. S6. Perform phased deduplication on red line actions that are determined to be valid violations. In the same phase, the same red line action is only triggered for the first time to automatically capture the violation frame, encode and upload it to the MinIO object storage, and generate an accessible image URL as a violation certificate. S7. Push standardized events to the front end in real time via SSE long connection. The standardized events include stage start, stage completion, red line action warning, heartbeat keep-alive, full process completion and video stream end events.

2. A time-constrained intelligent detection method for red-line action in firearm inspection, characterized in that, The eight prohibited actions include: finger entering the trigger guard, unauthorized disengagement of the safety, unauthorized disengagement of the safety, unauthorized pulling of the slide, shifting the hammer from the released state to the ready-to-fire state, shifting the hammer from the ready-to-fire state to the released state, having the hand directly behind the slide, and the muzzle pointing at the hand.

3. A time-constrained intelligent detection method for red-line action in firearm inspection, characterized in that, The correction of the detection results based on the physical rules of firearms includes: when the slide is detected to be in a stretched state, the safety state is forcibly corrected to the open state; the mutual exclusion category screening includes: for four groups of mutual exclusion detection categories, safety open / closed, slide extension / retraction, finger in / out of the guard ring, and hammer ready / release, only the single state with the highest confidence is retained for each group.

4. A time-constrained intelligent detection method for red-line action in firearm inspection, characterized in that, The method of constructing a dangerous rectangular region at the tail of the sleeve based on the backward extension of the sleeve OBB tilt detection frame is as follows: taking the side length of the sleeve OBB tilt detection frame connecting the muzzle edge and the tail edge as a reference, extending one-third of the side length backward from the tail edge in a direction away from the muzzle to generate a dangerous rectangular region at the tail; when the intersection-exclusion ratio of the dangerous rectangular region at the tail and the hand detection frame is not less than 0.5, it is determined that the hand is located directly behind the sleeve.

5. A time-constrained intelligent detection method for red-line action in firearm inspection, characterized in that, In the timing control of the gun inspection process based on the finite state machine, the completion judgment logic for each stage includes: Magazine disassembly stage: completion judgment based on the state change sequence of the intersection-union ratio of the magazine detection frame and the grip detection frame changing from zero to greater than zero and then back to zero; Safety disengagement stage: completion judgment based on the switch from the safety state to open; Slide pull stage: completion judgment based on the state change sequence of the slide detection category going from unstretched to stretched and then back to unstretched; Aiming and firing stage: completion judgment based on the combined condition of the finger entering the guard and the hammer changing from the ready-to-fire state to the released state; Hammer cocking to ready-to-fire stage: completion judgment based on the switch from the released state to the ready-to-fire state; Safety disengagement stage: completion judgment based on the switch from the safety state to closed; Magazine reset stage: completion judgment based on the state change sequence of the intersection-union ratio of the magazine detection frame and the grip detection frame changing from zero to greater than zero and then back to zero.

6. A time-constrained intelligent detection system for red-line actions during firearm inspection, based on any one of the time-constrained intelligent detection methods for red-line actions during firearm inspection as described in claims 1-5, characterized in that... include: The video decoding module is used to decode the police officer's gun inspection video stream frame by frame based on FFmpeg and output a BGR format image; The AI ​​inference module includes a general YOLO target detection submodule and an OBB rotating frame detection submodule. The general YOLO target detection submodule is used to identify 11 types of targets in each frame of image, including hands, magazines, grips, safety open / closed states, slide extension / retraction states, whether fingers are inside the guard ring, and hammer ready / release states. The OBB rotating frame detection submodule is used to detect the slide and output the coordinates of the four vertices of the tilt detection frame. The result filtering module is used to perform three-layer filtering optimization on the reasoning results: selection of similar targets, screening of mutually exclusive categories, and correction of gun physical rules. The timing smoothing and geometric verification module is used to perform multi-frame voting timing smoothing of the states of the safety, sleeve, finger, and hammer based on a sliding window, and to construct the muzzle ray and dangerous rectangular area at the tail of the sleeve based on the sleeve OBB tilt detection box. Spatial geometric hazard is determined by ray intersection detection and intersection-union ratio calculation. The finite state machine management module is used to divide the standardized gun inspection process into eight stages, each stage is independently configured with a set of prohibited red line actions, and the red line actions are judged in stages by combining the anti-shake frame count. The violation evidence collection module is used to deduplicat valid violation actions in stages. In the same stage, the same red line action is only triggered for the first time to automatically capture the violation frame and upload it to the MinIO object storage to generate violation evidence. The message push module is used to push standardized events, including stage start, stage completion, red line action warning, heartbeat keep-alive, full process completion and video stream end, to the front end in real time via SSE long connection.

7. The intelligent detection system for red line action of firearm inspection based on time constraints according to claim 6, characterized in that, The finite state machine management module has built-in current stage variables and action completion marker dictionaries. The current stage variables flow sequentially between eight stages as the gun inspection process progresses. The action completion marker dictionary is used to record the completion status of each stage. Each stage independently maintains a set of violation actions to achieve deduplication alarms for red line actions of the same type in the same stage.

8. The intelligent detection system for red line action of firearm inspection based on time constraints according to claim 6, characterized in that, In the timing smoothing and geometric verification module, the muzzle ray is constructed as follows: taking the midpoint of the sleeve OBB tilt detection frame near the tail as the starting point and the midpoint of the side near the muzzle as the direction reference point, a ray is generated that penetrates the sleeve. If the ray intersects with the hand detection frame, it is determined that the muzzle is pointing at the hand.

9. The intelligent detection system for red line action of firearm inspection based on time constraints according to claim 6, characterized in that, It also includes a background asynchronous thread scheduling module, which creates a heartbeat keep-alive thread and a detection main thread. The heartbeat keep-alive thread pushes heartbeat packets to the message queue at a preset period to maintain the SSE long connection. The detection main thread independently executes video decoding, AI inference, state machine judgment and violation evidence collection logic, realizing the parallel operation of video decoding, AI inference, message push and file storage.

10. The intelligent detection system for red line action of firearm inspection based on time constraints according to claim 6, characterized in that, The red line action warning event in the standardized event carries a task identifier, timestamp, current gun inspection stage number, red line action type code, and access URL of the violation frame image stored in MinIO; the message push module builds an SSE server based on the FastAPI framework, caches the events to be pushed through the message queue, and the event generator reads and encapsulates them into a standard SSE data stream for continuous push to the front end. The long connection is automatically disconnected after the task is completed.