Method and system for recognizing training state of a firefighter based on image recognition

Abstract

The application discloses a kind of based on image recognition's identification method and system of the training state of fireman, and the present application relates to the technical field of state identification, based on the identification of the training event to determine the abnormal training behavior of fireman, based on the behavior content of the abnormal training behavior of fireman, corresponding behavior occurrence area and the past training event of fireman determines the current training state of fireman, improve the precision of the current training state of fireman.Therefore, according to the current training subject of fireman and corresponding current training state determine updated training content, according to the updated training content, the current training action of fireman and corresponding face image determine the state dynamic diagram of fireman;Based on the image recognition of state dynamic diagram to determine corresponding training upgrade node, according to the training upgrade node, the current training subject of fireman and the past training event of fireman determine next training content, improve the precision of next training content.

Description

Technical Field

[0001] This invention relates to the technical field of state recognition, and in particular to a method and system for recognizing the training state of firefighters based on image recognition. Background Technology

[0002] With the development of technology, firefighters need to conduct daily training at designated training sites to cope with various fire incidents. Current technology monitors the training process in real time and collects multiple training images of firefighters. The training process is determined based on the recognition of multiple training images. However, it ignores abnormal training behaviors of firefighters and does not control such behaviors, which affects the accuracy of the firefighter's current training status and leads to lower accuracy in the next training content. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and system for identifying the training status of firefighters based on image recognition.

[0004] This invention provides a method for identifying the training status of firefighters based on image recognition, including: In the firefighters' training ground, multiple training images of firefighters are determined based on multiple cameras and the Internet of Things in the training ground. Based on multiple training images, the corresponding image shooting time and the shape of the training ground, a dynamic training map of the firefighters is determined. Multiple sub-training behaviors are determined by image recognition of firefighters' training dynamics. The training events of firefighters in each training subject are determined based on the content of each sub-training behavior, the corresponding behavior path, and the corresponding training subject. Based on the identification of this training event, the abnormal training behavior of the firefighter is determined, and the current training status of the firefighter is determined based on the content of the abnormal training behavior, the corresponding area where the behavior occurred, and the firefighter's past training events. The updated training content is determined based on the firefighter's current training subjects and corresponding current training status. The firefighter's status animation is determined based on the updated training content, the firefighter's current training actions, and the corresponding facial images. The corresponding training upgrade node is determined by image recognition based on the state dynamic graph. The next training content is determined based on the training upgrade node, the firefighter's current training subject and the firefighter's past training events. At this time, the training level of the next training content is higher than that of the updated training content.

[0005] This invention provides an image recognition-based system for identifying the training status of firefighters. This system is applied to the aforementioned image recognition-based method for identifying the training status of firefighters. The image recognition-based system for identifying the training status of firefighters includes: The training animation module is used to determine multiple training images of firefighters in the training area based on multiple cameras and the Internet of Things. Based on multiple training images, the corresponding image capture time and the shape of the training area, the training animation of the firefighters is determined. The training event module is used to determine multiple sub-training behaviors based on image recognition of firefighters' training animations, and to determine the training events of firefighters in each training subject based on the behavior content, corresponding behavior path and corresponding training subject of each sub-training behavior. The current training status module is used to determine the abnormal training behavior of firefighters based on the identification of the training event. The current training status of firefighters is determined based on the content of the abnormal training behavior, the corresponding area where the behavior occurred, and the firefighters' previous training events. The status animation module is used to determine the updated training content based on the firefighter's current training subject and corresponding current training status, and to determine the firefighter's status animation based on the updated training content, the firefighter's current training action and corresponding facial image. The training content module is used to determine the corresponding training upgrade node based on image recognition of the state dynamic graph. Based on the training upgrade node, the firefighter's current training subject and the firefighter's past training events, the next training content is determined. At this time, the training level of the next training content is higher than that of the updated training content.

[0006] Compared with the prior art, the beneficial effects of the present invention are: In this embodiment of the invention, the method described herein determines multiple training images of firefighters in a training area based on multiple cameras and the Internet of Things (IoT). A dynamic training graph of the firefighters is then determined based on these multiple training images, their corresponding capture times, and the shape of the training area. Multiple sub-training behaviors are identified based on image recognition of the dynamic training graph. Training events for each training subject are determined based on the content of each sub-training behavior, its corresponding path, and the corresponding training subject. Abnormal training behaviors of the firefighters are identified based on the recognition of these training events. The current training status of the firefighters is determined based on the content of these abnormal training behaviors, the corresponding location of these behaviors, and the firefighters' past training events. This method introduces training events to control abnormal training behaviors, taking into account the content of these abnormal training behaviors, the corresponding location of these behaviors, and the firefighters' past training events, thereby improving the accuracy of the firefighters' current training status.

[0007] Therefore, the updated training content is determined based on the firefighter's current training subject and corresponding current training status. A dynamic state graph of the firefighter is then determined based on the updated training content, the firefighter's current training actions, and the corresponding facial image. Based on image recognition of the dynamic state graph, the corresponding training upgrade node is determined. The next training content is then determined based on this upgrade node, the firefighter's current training subject, and past training events. The introduction of the dynamic state graph further controls the training upgrade node, achieving a holistic consideration of the upgrade node, the firefighter's current training subject, and past training events, thus improving the accuracy of the next training content. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating the method for identifying the training status of firefighters based on image recognition in an embodiment of the present invention. Figure 2 This is a flowchart illustrating step S11 in the image recognition-based method for identifying the training status of firefighters in an embodiment of the present invention. Figure 3 This is a flowchart illustrating step S12 in the image recognition-based method for identifying the training status of firefighters in an embodiment of the present invention. Figure 4 This is a flowchart illustrating step S13 in the image recognition-based method for identifying the training status of firefighters in an embodiment of the present invention. Figure 5 This is a flowchart illustrating step S14 of the image recognition-based method for identifying the training status of firefighters in an embodiment of the present invention. Figure 6This is a flowchart illustrating step S15 of the image recognition-based method for identifying the training status of firefighters in an embodiment of the present invention. Figure 7 This is a schematic diagram of the structural composition of a firefighter training status recognition system based on image recognition, according to an embodiment of the present invention. Detailed Implementation

[0009] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0010] Please see Figures 1 to 7 A method for identifying the training status of firefighters based on image recognition, applied to status recognition scenarios; the method for identifying the training status of firefighters based on image recognition includes: Step S11: In the firefighters' training area, multiple training images of firefighters are determined based on multiple cameras and the Internet of Things in the training area. Based on the multiple training images, the corresponding image shooting time, and the shape of the training area, a dynamic training image of the firefighters is determined. Step S12: Based on the image recognition of the firefighter's training animation, determine multiple sub-training behaviors, and determine the firefighter's training events in each training subject based on the behavior content, corresponding behavior path, and corresponding training subject of each sub-training behavior. Step S13: Based on the identification of the training event, determine the abnormal training behavior of the firefighter, and determine the current training status of the firefighter based on the content of the abnormal training behavior, the corresponding area where the behavior occurred, and the firefighter's previous training events. Step S14: Determine the updated training content based on the firefighter's current training subject and corresponding current training status; determine the firefighter's status animation based on the updated training content, the firefighter's current training actions, and the corresponding facial image. Step S15: Determine the corresponding training upgrade node based on image recognition of the state dynamic graph. Determine the next training content based on the training upgrade node, the firefighter's current training subject, and the firefighter's past training events. At this time, the training level of the next training content is higher than that of the updated training content.

[0011] refer to Figure 2 In step S11, the specific steps are as follows: S111: Mark the training area of ​​firefighters and determine multiple cameras in the training area based on the detection of the training area. Based on the location of multiple cameras, the current location of firefighters and the Internet of Things, determine multiple training images of firefighters; the multiple training images present the training actions of firefighters in different dimensions. S112: Collect the shape of the training site, determine the first training image based on multiple training images and the shape of the training site, determine the second training image based on multiple training images and the corresponding image shooting time, and determine the firefighter's training dynamic image based on the first and second training images.

[0012] In the embodiments of this application, the training sites of firefighters are marked, and multiple cameras at the training sites are determined based on the detection of the training sites. Multiple training images of firefighters are determined based on the positions of the multiple cameras, the current position of the firefighters, and the Internet of Things. The multiple training images present different dimensions of the firefighters' training actions, and take into account the overall consideration of the positions of the multiple cameras, the current position of the firefighters, and the Internet of Things, so as to ensure the accuracy of the multiple training images of the firefighters.

[0013] At this point, using LiDAR, SLAM technology, or high-precision architectural drawings, the entire training ground is 3D scanned and modeled to generate an accurate 3D point cloud model containing all fixed facilities (such as training towers, obstacle walls, and simulated fire sources). This model is then semantically labeled, assigning functional attributes to each geometric entity. For example, a vertical structure is labeled as "climbing tower-01", and a horizontal pipe is labeled as "crossing flue-02".

[0014] At the same time, each deployed camera also needs to be precisely calibrated in this 3D model to determine its 3D coordinates and orientation (i.e., extrinsic parameters) and complete the calibration of its intrinsic parameters (distortion, focal length, etc.); the system then has a global spatial database containing the positions, field of view, and semantic information of all cameras and training facilities.

[0015] Due to the limitations of single positioning technology, we adopt a multimodal fusion positioning scheme; UWB (Ultra-Wideband) tags can be integrated into the firefighters' equipment, and centimeter-level high-precision positioning can be achieved through UWB base stations deployed in the training ground; at the same time, their attitude and gait information can be obtained by using the IMU (Inertial Measurement Unit) on the training suit.

[0016] When firefighters enter certain specific areas (such as the interior of a simulated building) and cause UWB signal attenuation, the system can seamlessly switch to visual SLAM-based positioning. This involves using images captured by firefighters' body cameras or fixed cameras to perform feature matching with a pre-built digital twin model to estimate their location. Data from all positioning sources is fused using algorithms such as Kalman filtering or particle filtering to ultimately output a highly robust, high-update-rate real-time 3D coordinate system.

[0017] The system background runs a multi-objective optimization algorithm. Its inputs include the firefighter's real-time 3D coordinates and velocity vectors, the precise pose and field of view parameters of all cameras, the semantic information of the training field digital twin model, and the preset training course process. The optimization goal of the algorithm is to select an optimal subset of cameras that can capture the key features of the training behavior to the maximum extent.

[0018] The optimized metrics include: visual clarity (prioritizing cameras facing the firefighter's main action plane), visibility of key parts (ensuring key body parts are captured based on the current task), resolution and focal length (automatically scheduling optical zoom or high-resolution cameras based on distance), and load balancing (avoiding prolonged use of the same camera). After the algorithm outputs instructions, it will activate the selected cameras in real time through the Internet of Things network and require them to transmit video streams to the central server with specified parameters. These video streams are the "multiple training images of the firefighter" that we need.

[0019] Furthermore, the shape of the training site is collected, and a first training image is determined based on multiple training images and the shape of the training site. A second training image is determined based on multiple training images and the corresponding image shooting time. A dynamic training image of the firefighters is determined based on the first and second training images, which takes into account the overall consideration of the first and second training images and ensures the accuracy of the dynamic training image of the firefighters.

[0020] At this point, the system will use the pre-calibrated camera parameters in S111 to perform 3D reconstruction on images from different cameras, and generate a 3D human point cloud model of the firefighter through a multi-view geometric algorithm; this dynamically generated 3D point cloud model will then be spatially registered and fused with a pre-built digital twin model of the training site.

[0021] This process involves spatial relationship calculations, such as collision detection (determining whether a limb is in contact with a facility), spatial containment (determining whether a specific area has been entered), and distance measurement (calculating the real-time distance between key joints and target objects). The resulting "first training map" is a snapshot of a 3D scene with rich semantic labels. It is no longer a collection of pixels, but a structured dataset that describes the firefighter's body posture, position, and precise interaction with the surrounding environment at a given moment.

[0022] The system rigorously sorts multiple training images according to their timestamps, forming a continuous video stream sequence. It then applies temporal feature extraction algorithms to analyze this sequence, such as using optical flow to calculate pixel-level motion vectors between frames to capture subtle changes in movement. The system utilizes models such as graph neural networks or recurrent neural networks, treating the key poses or behavioral units identified in each frame as nodes in a graph and their temporal connections as edges, to construct a behavioral topology graph. This graph structure clearly expresses the sequence and logical relationships of actions; for example, the "bending over" node is usually followed by the "grabbing" node. Therefore, the "second training graph" is a topological structure that describes how actions evolve over time, encoding the dynamic patterns and rhythms of behavior.

[0023] The system will associate the "first training graph" (spatial semantic snapshot) and the "second training graph" (temporal topology graph); in specific implementation, each semantic event in the "first training graph" (such as "hand touching the ladder") can be used as an enhanced attribute at the corresponding time node in the "second training graph".

[0024] Conversely, each action node in the time sequence diagram (such as "leg kick") can be found in the spatial diagram with its precise three-dimensional position and posture; the final generated "training dynamic diagram" is a four-dimensional (3D space + 1D time) data volume; it can answer both "what is the firefighter's spatial posture and interaction state at time T" and "what kind of action sequence did the firefighter complete from T1 to T2"; this data volume is the ideal input for subsequent accurate behavior recognition, state assessment and anomaly detection.

[0025] refer to Figure 3 In step S12, the specific steps are as follows: S121: Determine the corresponding training action area based on the detection of the firefighter's training dynamic map, determine the corresponding sub-training behavior based on the identification of each training action area, collect multiple sub-training behaviors, and mark the behavioral content of each sub-training behavior. S122: Based on the identification of each sub-training behavior, determine the corresponding behavior path, and based on the behavior content of each sub-training behavior, the corresponding behavior path, and the corresponding training subject, determine the training events of firefighters in each training subject.

[0026] In the embodiments of this application, the corresponding training action area is determined based on the detection of the firefighter's training animation, and the corresponding sub-training behavior is determined based on the identification of each training action area. Multiple sub-training behaviors are collected, and the behavioral content of each sub-training behavior is marked. This approach takes into account the overall consideration of the identification of each training action area and ensures the accuracy of the corresponding sub-training behavior.

[0027] At this point, the system will perform spatiotemporal feature analysis on the "training animation". For example, it will use spatiotemporal point of interest (STIP) detection to automatically identify the starting point or key frame of the action, or extract the continuous motion trajectory of the firefighter's main joints in three-dimensional space. Clustering algorithms (such as DBSCAN) will be applied to group these trajectories or feature points. The algorithm can automatically discover different numbers of clusters based on the spatial density and continuity of the trajectory points. Each cluster represents a "training action area" that is relatively concentrated in space and continuous in time. This process can effectively distinguish between the two actions of "climbing in area A" and "breaking in area B", even if they are visually similar.

[0028] The system extracts a behavior descriptor from the spatiotemporal data of each action region. This high-dimensional feature vector encodes the dynamic pattern of the action. The extraction method can be based on a deep learning model, such as inputting the data block into a pre-trained spatiotemporal graph convolutional network (ST-GCN), which can simultaneously capture the spatial topological relationships between joints and the dynamic features that change over time.

[0029] This behavior descriptor is input into a classifier and matched against a predefined sample library containing behaviors such as "crawl" and "break down doors and windows". The classifier outputs the behavior category with the highest probability, which is the identified "sub-training behavior". The system collects all identified sub-training behaviors to form a preliminary behavior list.

[0030] Each identified sub-training behavior is assigned a content label, generating a structured object rich in contextual information. This object contains at least the following attributes: a unique Behavior_ID, the semantic name of the behavior, Behavior_Content, start and end timestamps Start_Time / End_Time, the spatial boundary of the occurrence Spatial_Region, the recognition Confidence_Score, and the actor_ID. This labeling process transforms a vague concept of action into a data entity that a computer can precisely process, query, and analyze.

[0031] Specifically, firefighter A needs to use hydraulic breaching tools to open a simulated door and enter the interior; S112 has generated his complete "training animation"; the system analyzes firefighter A's "training animation" and finds that his movement trajectory forms two high-density clusters in space and time: Cluster 1: From time T1 to T15 seconds, firefighter A's center of mass and limb joints are mainly concentrated near coordinates (15,10,0), and the movement amplitude is small, but the upper limb joints have high-frequency vibrations; the system marks this area as "training action area-01" using the DBSCAN algorithm; Cluster 2: From time T16 to T20 seconds, firefighter A's overall center of mass moves rapidly from (15,10,0) to (15,10,3), forming a trajectory that traverses space; the system marks this as "training action area-02".

[0032] For region-01: The system extracts spatiotemporal data blocks of firefighter A's upper limb skeletal points within T1-T15 seconds and inputs them into the pre-trained ST-GCN model; the model's output feature vector is compared with the behavior database, and matches the behavior of "operating hydraulic tools" with 95% confidence; the system collects the first sub-training behavior; For region-02: The system extracts spatiotemporal data blocks of firefighter A's whole body skeletal points within T16-T20 seconds; the model analyzes its overall displacement and posture changes, and matches the behavior of "crawl forward" with 98% confidence; the system collects the second sub-training behavior.

[0033] The system generates structured tags for the two collected behaviors: Behavior 1 tag: {Behavior_ID:B001,Behavior_Content:“Operating hydraulic tools”,Start_Time:T1,End_Time:T15,Spatial_Region:{Center:(15,10,0),Volume:2m³},Confidence_Score:0.95,Actor_ID:“A”}; Behavior 2 marker: {Behavior_ID:B002,Behavior_Content:"Creeping",Start_Time:T16,End_Time:T20,Spatial_Region:{Center:(15,10,1.5),Volume:5m³},Confidence_Score:0.98,Actor_ID:"A"}.

[0034] Furthermore, based on the identification of each sub-training behavior, the corresponding behavior path is determined. Based on the behavior content of each sub-training behavior, the corresponding behavior path, and the corresponding training subject, the training events of firefighters in each training subject are determined. This takes into account the overall consideration of the behavior content, corresponding behavior path, and corresponding training subject of each sub-training behavior, ensuring the accuracy of the training events of firefighters in each training subject.

[0035] At this point, the system will use the Start_Time of all sub-training behaviors as the primary key to perform topological sorting on them, forming a strict time series; the system will extract the Spatial_Region attribute of each behavior object and connect these spatial points in chronological order to form a polyline in three-dimensional space, i.e., a physical path.

[0036] The system constructs a behavior path, a directed graph structure where each node represents a sub-training behavior, and directed edges between nodes represent the transition relationships between behaviors. This abstract behavior path is then interpreted as a meaningful "training event" by combining it with specific business rules and scenario information. This process requires an event reasoning engine that integrates multiple information sources: precise behavior content tags from S121, the aforementioned constructed behavior logic chain, and the top-level context from the training management system—the training subject.

[0037] The inference engine can operate based on expert-defined "IF-THEN" rules, or it can use sequence models such as Hidden Markov Models (HMMs) or Transformers to learn from a large amount of standard training data to understand which combinations of behavioral sequences are legal and meaningful under a specific subject. The resulting "training event" is a high-level data structure with clear business value. It not only describes what happened, but also explains why this event is important in the context of the current task.

[0038] Specifically, firefighter A is conducting comprehensive "breach and rescue" training; S121 has identified and marked two sub-training behaviors: B001:{Behavior_Content:“Operating hydraulic tools”,…}; B002:{Behavior_Content:“Crawling forward”,…}; the system also retrieves the current training subject from the training management system as: "Subject C: Breach and entry into confined spaces".

[0039] The system checks the timestamps of B001 and B002 and finds that B001.Start_Time is earlier than B002.Start_Time. The system constructs the behavior path (directed graph) of firefighter A: [Start]>[B001: Operate hydraulic tools]>[B002: Crawl forward]>[...]; At the same time, the system connects the spatial centroids of the two behaviors to form a physical path from "outside the simulated door" to "inside the simulated door". Thus, the system obtains the complete behavioral logic chain of firefighter A in the current stage.

[0040] The event reasoning engine is activated, with the inputs being: behavior path [B001>B002], behavior content ["operating hydraulic tools", "crawl forward"] and training subject "entering and breaching a confined space". The engine matches the standard operating procedure (SOP) for the subject "entering and breaching a confined space" and finds that the first step of the standard procedure is "breaking down obstacles" and the second step is "entering in a low posture". The engine analyzes the behavior path and finds that B001 ("operating hydraulic tools") logically corresponds perfectly to the "breaking down obstacles" step of the SOP. The following B002 ("crawl forward") logically corresponds to the "entering in a low posture" step, and its physical path also verifies the transfer of spatial position (from outside to inside).

[0041] The engine's rule base contains a rule: IF(Subject = "Entering through a confined space by breaching") AND (Behavioral path includes ["Breaching behavior", "Low-posture movement behavior"]) AND (Physical path traversing obstacles) THEN(Event = "Successfully completed breaching and established an internal passage"). Based on the above reasoning, the system constructs a high-level training event: {Event_ID:E001,Subject: "Firefighter A",Training_Drill: "Subject C: Entering through a confined space by breaching"),Event_Type: "Successfully completed breaching and established an internal passage",Event_Description: "Firefighter A successfully breached the simulated door using hydraulic tools and entered in a crawling posture, establishing a safe passage for subsequent rescue operations",Component_Behaviors:[B001,B002],Timestamp:T20,Confidence:0.98}.

[0042] refer to Figure 4 In step S13, the specific steps are as follows: S131: Collect the training event, identify multiple sub-abnormal actions of the firefighter based on the identification of the training event, determine the abnormal training behavior of the firefighter based on the action content, corresponding action trajectory and corresponding action time of the multiple sub-abnormal actions, and mark the action content of the abnormal training behavior of the firefighter. S132: Determine the corresponding behavior occurrence area based on the tracing of abnormal training behaviors of firefighters, and determine the first state coefficient based on the content of the abnormal training behaviors of firefighters and the corresponding behavior occurrence area; S133: Collect past training events of firefighters, determine the second state coefficient based on the content of the firefighter's abnormal training behavior and the firefighter's past training events, and determine the firefighter's current training state based on the mapping relationship between the first state coefficient, the second state coefficient and the current training state. In the embodiments of this application, the training event is collected, and multiple sub-abnormal actions of the firefighter are determined based on the identification of the training event. The abnormal training behavior of the firefighter is determined according to the action content, corresponding action trajectory and corresponding action time of the multiple sub-abnormal actions, and the action content of the abnormal training behavior of the firefighter is marked. This approach takes into account the overall consideration of the action content, corresponding action trajectory and corresponding action time of multiple sub-abnormal actions, ensuring the accuracy of the abnormal training behavior of the firefighter.

[0043] At this point, the system will acquire the “training events” output by S12 and focus on the spatiotemporal data of each “sub-training behavior” contained therein; the system will then perform a high-precision comparison of this real-time data with a “standard action template library” that stores ideal action patterns demonstrated by experts or outstanding trainees.

[0044] This comparison process is multi-dimensional, including calculating the pose space deviation between real-time skeleton points and the template, using the Dynamic Time Warping (DTW) algorithm to compare the motion trajectory deviation of joints, checking the action timing deviation of key action nodes, and even integrating physiological signal anomalies; when the deviation value of any dimension exceeds the preset threshold, the system records a "sub-abnormal action" along with its deviation type, magnitude, and timestamp.

[0045] The system uses density-based clustering algorithms (such as DBSCAN) to analyze all sub-abnormal actions. The clustering is based on their temporal proximity, spatial correlation, and semantic correlation. For example, sub-abnormal actions such as "wrist trajectory deviation", "elbow angle too large", and "abnormal shoulder height" that occur consecutively within 1 second will be clustered into a cluster by the algorithm because they are highly correlated in time, space and semantics. This cluster represents an "abnormal training behavior".

[0046] The system analyzes the common features of all sub-abnormal actions that make up the abnormal cluster and matches them using a predefined abnormal tag library. This tag library contains descriptions of various common abnormal patterns such as "postural imbalance" and "insufficient strength". The final generated tag is a structured "abnormal training behavior" object, which not only contains a name, but also rich diagnostic information, such as a unique Abnormal_Behavior_ID, the semantic name of the abnormality, Behavior_Content, Severity_Score calculated based on the deviation, start and end times, a list of body parts involved, and a list of IDs of all sub-abnormal actions that make up it.

[0047] Specifically, firefighter A is performing the "crawl" sub-training action in this subject; S122 has identified this as part of the training event; the system compares firefighter A's real-time skeletal point data during the "crawl" with the standard template; at T3.5 seconds, the system detects that the vertical height of his pelvic point is 45cm, while the standard template requires it to be below 30cm; the deviation value is 15cm, exceeding the threshold; sub-abnormal action SA01 is recorded: {Type:“Posture_Deviation”,Part:“Pelvis”,Value:15cm,Timestamp:T3.5}; At T4.0 seconds, the system detected that the Z-coordinate (height) of its left shoulder joint was 20cm higher than the standard value; recorded sub-abnormal action SA02: {Type:“Posture_Deviation”,Part:“Left_Shoulder”,Value:20cm,Timestamp:T4.0}; Between T3.2 seconds and T4.1 seconds, the system, using the DTW algorithm, detected that the cumulative distance between the head's motion trajectory and the standard trajectory exceeded the threshold, indicating severe head shaking; recorded sub-abnormal action SA03: {Type: "Trajectory_Deviation", Part: "Head", DTW_Distance: 8.5, Duration: 0.9s}.

[0048] The system detected three sub-abnormal actions: SA01, SA02, and SA03. Clustering analysis revealed that they were highly concentrated in time (T3.2s-T4.1s), involved the core torso (pelvis, shoulders) and head in space, and all pointed to "postural instability" in semantics. Therefore, the system aggregated these three sub-abnormal actions into a cluster, identifying an "abnormal training behavior".

[0049] The system analyzed the common characteristics of this cluster and found that the core problem was excessively high and unstable body posture; the system matched the most suitable description in the anomaly label library: "unbalanced crawling posture"; the system generated a structured anomalous training behavior label: {Abnormal_Behavior_ID:AB001,Behavior_Content:“Crawling posture imbalance”,Severity_Score:0.75 (based on a comprehensive calculation of three deviations),Start_Time:T3.2,End_Time:T4.1,Involved_Body_Parts:[“Pelvis”,“Left_Shoulder”,“Head”],Component_Sub_Abnormal_Actions:[SA01,SA02,SA03]}.

[0050] Furthermore, the corresponding behavior occurrence area is determined by tracing the abnormal training behavior of firefighters. The first state coefficient is determined based on the content of the abnormal training behavior and the corresponding behavior occurrence area, which takes into account the overall consideration of the content of the abnormal training behavior and the corresponding behavior occurrence area, thus ensuring the accuracy of the first state coefficient.

[0051] At this point, the system will obtain the time window of the abnormal behavior object and use this timestamp as the query key to perform a spatiotemporal range query in the original "training dynamic graph" four-dimensional data volume generated by S112; by locking the three-dimensional spatial coordinate sequence of the firefighter's main key points within the time window, the system can calculate the spatial centroid of the abnormal behavior.

[0052] The system performs a spatial intersection query between the spatial centroid coordinates and the digital twin model of the training field. Predefined semantic regions such as "high-risk edge area" and "narrow passage area" in the model can be matched, thereby completing the determination of the "behavior occurrence area".

[0053] Calculate a "first-state coefficient" that reflects the immediate danger of abnormal behavior. This coefficient is a comprehensive score, and its calculation model is usually a weighted function. The model's input mainly includes two vectors: one is a "behavior content vector" quantified by the severity and duration of the abnormal behavior, and the other is a "regional risk vector" quantified by the environmental hazard level and task criticality of the area where the behavior occurs. An intuitive model is the risk multiplication model, which means that the severity of a basic abnormality will be amplified because it occurs in a high-risk environment. For example, a slight "body sway" will have a low coefficient if it occurs on flat ground, but if it occurs at the edge of a 30-meter-high training tower, the coefficient will skyrocket, immediately triggering a high-level alarm.

[0054] Specifically, during firefighter A's execution of this exercise, S131 identified abnormal training behavior: {Abnormal_Behavior_ID:AB001,Behavior_Content:“Crawling Posture Imbalance”,Severity_Score:0.75,…}; The system obtains the abnormal time window of AB001 as [T3.2,T4.1]; The system queries the centroid coordinate sequence of firefighter A within this time window in the “Training Dynamic Graph”, and calculates the spatial centroid as (15.1,10.2,0.5); The system inputs this coordinate into the digital twin model of the training field for querying, and the model returns the following information: The point is located within a semantic region labeled {Region_ID:R_Narrow_01,Region_Type:“Narrow Passage”,Width:0.8m,Risk_Level:“Medium”}; therefore, the system determines that the “behavior occurrence region” of AB001 is “Narrow Passage R_Narrow_01”.

[0055] The system quantifies the input vectors: Behavioral content vector (V_content): extracted from AB001 with Base_Severity=0.75; Regional risk vector (V_region): extracted from region R_Narrow_01 with Environmental_Hazard_Level=0.6 (medium risk); Meanwhile, considering that "crawl posture imbalance" can easily lead to the body getting stuck or colliding in narrow passages, the system adds a Task_Criticism_Factor=1.2.

[0056] The system uses a risk multiplication model for calculation: Comprehensive_Risk_Factor=Environmental_Hazard_Level Task_Criticality_Factor=0.6×1.2=0.72; First_State_Coefficient=Base_Severity×Comprehensive_Risk_Factor=0.75×0.72=0.54; The system ultimately calculates the "first state coefficient" of firefighter A's abnormal training behavior as 0.54. This value of 0.54 reflects the actual situation better than the simple severity score of 0.75. It tells the system that although the abnormality itself is very serious, because it occurred in a medium-risk area, its immediate danger is comprehensively assessed as medium to high.

[0057] Therefore, by collecting past training events of firefighters, determining the second state coefficient based on the content of abnormal training behaviors and past training events, and determining the firefighter's current training state based on the mapping relationship between the first state coefficient, the second state coefficient, and the current training state, this approach takes into account the overall mapping relationship between the first state coefficient, the second state coefficient, and the current training state, ensuring the accuracy of the firefighter's current training state. Simultaneously, by introducing training events to control abnormal training behaviors, this approach takes into account the content of abnormal training behaviors, the corresponding areas where these behaviors occur, and the firefighter's past training events, further improving the accuracy of the firefighter's current training state.

[0058] At this point, the system will collect all training events of the firefighter in the same training subjects in the past and present from the personal historical database, and construct a personalized behavioral baseline model that includes historical behavior frequency, anomaly spectrum and skill proficiency curve. The system will compare the current "abnormal training behavior" with this baseline model: if the current anomaly is low-frequency or occurs for the first time, the system will determine that this is a significant negative deviation, and Second_State_Coefficient will be assigned a high value; if the current anomaly is the firefighter's "old problem", the coefficient will be assigned a low value; if the baseline model shows that similar anomalies have been on the rise recently, even if the current anomaly is not the first time, the coefficient will be assigned a moderately high value.

[0059] By fusing real-time risk assessment (first state coefficient) and personalized historical deviation (second state coefficient), a state mapping model is used to output a final, highly interpretable "current training state." This mapping model can be a multi-dimensional decision tree or a rule matrix, designed to interpret the deeper meaning behind the combination of the two coefficients. For example, two low values ​​are mapped to "normal" or "habitual technical flaws"; the first high and the second low are mapped to "skill bottlenecks"; the first medium and the second high are mapped to "fatigue"; and two high values ​​are mapped to "extreme fatigue," requiring immediate intervention.

[0060] Specifically, Firefighter A experienced an anomaly during "Subject C". S132 has calculated First_State_Coefficient = 0.54; S131 marks the anomalous behavior as {Behavior_Content: "Crawling Posture Imbalance"}; the system queries Firefighter A's historical database and finds that he has consistently performed exceptionally well in the past 10 "Subject C" training sessions, with no recorded anomalies related to "crawling posture". His personal baseline model shows that this skill is one of his strengths.

[0061] The system compares the current "crawling posture imbalance" anomaly with the baseline and finds that this is the first time an anomaly type has appeared. According to the rule "first time anomaly" belongs to a significant negative deviation, the system calculates Second_State_Coefficient to be 0.9 (high value).

[0062] The system obtained two key coefficients: First_State_Coefficient = 0.54 (medium risk); Second_State_Coefficient = 0.9 (high deviation from personal baseline); the system inputs these coefficients into the state mapping model; the model's decision tree performs matching: IsFirst_Coefficient > 0.5 > Yes; IsSecond_Coefficient > 0.7 > Yes; matching rule: IF(First_Coefficient is Medium-to-High) AND (Second_Coefficient is High) THENState = "Fatigue or Loss of Focus"; final state output: the system determines that firefighter A's current training state is "fatigue or lack of concentration".

[0063] refer to Figure 5 In step S14, the specific steps are as follows: S141: Collect the firefighter's current training subjects, determine multiple sub-training items based on the identification of the firefighter's current training subjects, and determine the updated training content based on the content of the multiple sub-training items, the corresponding order of the items and the firefighter's current training status. S142: Collect the firefighter's current training actions, determine the first-level state diagram based on the firefighter's current training actions and the updated training content, and at the same time, collect the firefighter's facial images during the training process, determine the second-level state diagram based on the updated training content and the firefighter's facial images during the training process, and determine the firefighter's state dynamic diagram based on the first-level state diagram and the second-level state diagram.

[0064] In the embodiments of this application, the current training subjects of firefighters are collected, and multiple sub-training items are determined based on the identification of the current training subjects of firefighters. The updated training content is determined based on the content of the multiple sub-training items, the corresponding order of the items, and the current training status of firefighters. This approach takes into account the overall consideration of the content of the multiple sub-training items, the corresponding order of the items, and the current training status of firefighters, ensuring the accuracy of the updated training content.

[0065] At this point, the system collects the training subjects that firefighters are currently performing and queries their definitions in a training knowledge base. This knowledge base is a structured graph database, where each subject is defined as a directed acyclic graph (DAG). Nodes in the graph represent independent "sub-training items," which are the smallest executable units of training, such as "wearing a safety harness" or "rope knot manipulation." Each node contains detailed metadata such as difficulty, duration, and physical requirements. The edges in the graph represent the dependencies and execution order between sub-items. By querying this DAG, the system can determine all the sub-training items that constitute the current subject and their standard procedures, forming a complete training blueprint.

[0066] The system's input includes the standard DAG decomposed in the previous step and the evaluation results output by S13 (such as "fatigue," "skill bottleneck," etc.). The decision engine contains a set of state-intervention mapping rules: when the state is "fatigue," it finds nodes with high physical demands and inserts a recovery node before them or replaces them with low-intensity items; when the state is "skill bottleneck," it locates relevant nodes and backtracks to insert more basic reinforcement training nodes; when the state is "stress," it selects low-difficulty items for repetition to build confidence. The engine outputs a modified DAG tailored to the firefighter's current state, namely the "updated training content," which aims to maximize training effectiveness and ensure safety.

[0067] Specifically, when firefighter A is performing "Subject C: Enclosed Space Demolition and Entry", S133 assesses his "current training status" as "fatigue or lack of concentration"; the system collects the current subject as "Subject C: Enclosed Space Demolition and Entry"; the system queries the training knowledge base and obtains the standard DAG for this subject: Node1: "Prepare demolition tools"; Node2: "Demolish simulated door" (dependent on Node1); Node3: "Crawl through narrow passage" (dependent on Node2); Node4: "Internal target identification" (dependent on Node3); the system obtains metadata for each node, such as Node3: {Project_Name: "Crawl", Physical_Demand: "High", Difficulty_Level: "Medium"}.

[0068] The decision engine receives input: a standard DAG and the current state "fatigue"; the engine analyzes the DAG and finds that firefighter A is about to execute Node3: "crawl forward", the Physical_Demand of which is "High", which will exacerbate fatigue; the engine's rule base contains: IF(State=="Fatigue")AND(Next_Node.Physical_Demand=="High")THEN(Insert_Recovery_Node); The engine searches the knowledge base for nodes with Physical_Demand "Low" that are relevant to the current scene, and finds Node_R: "Kneeling observation and adjustment"; The engine modifies the standard DAG: inserting Node_R between Node2 and Node3; The new dependency relationship becomes: Node3 now depends on Node_R.

[0069] The system generates "updated training content," which is a new DAG: Node1: "Prepare breaching tools"; Node2: "Breaching the simulated door"; Node_R: "Kneeling posture observation and adjustment" (newly inserted, dependent on Node2); Node3: "Crawling forward through a narrow passage" (now dependent on Node_R); Node4: "Internal target identification." The system immediately converts this updated training content into instructions and sends them through firefighter A's communication equipment: "A, note that after breaching, pause crawling; adopt a kneeling posture, adjust breathing, observe the internal environment, and wait for the next instruction."

[0070] Furthermore, the system collects the firefighters' current training movements and determines the first-level state diagram based on these movements and the updated training content. Simultaneously, it collects facial images of the firefighters during training and determines the second-level state diagram based on the updated training content and these facial images. A dynamic state diagram of the firefighters is then determined based on both the first and second-level state diagrams, taking into account the overall considerations of both diagrams and ensuring the accuracy of the dynamic state diagram of the firefighters.

[0071] At this time, the system will collect and identify the firefighter's "current training action" in real time, and at the same time obtain the target sub-training items in the "updated training content" generated by S141. The system will perform in-depth behavioral semantic consistency verification on the two, and by calculating the posture matching degree, action smoothness and key node completion degree, a behavior execution deviation degree will be obtained. This deviation degree will be transformed into a visual "first-level state diagram" in real time, which will intuitively reflect the compliance of the firefighter's external behavior.

[0072] The system continuously collects facial image sequences of firefighters through a dedicated camera and inputs them into a multimodal emotion computing model. The model uses technologies such as micro-expression recognition, remote heart rate estimation (rPPG), and head posture and gaze estimation to output a psychophysiological state vector containing dimensions such as stress, fatigue, and concentration. This vector is also converted into a "secondary state diagram" in real time, which intuitively reflects the fluctuations in the firefighter's internal state.

[0073] The system employs an attention fusion network to achieve this goal. This network can dynamically adjust the attention weights for behavioral biases and psychological states based on the context of the current training content. For example, it pays more attention to the "stress index" during "high-risk tasks" and to the "fatigue index" during "rest and adjustment".

[0074] The network ultimately outputs a comprehensive state representation and maps it to predefined final state categories such as "efficient execution" and "fatigue but perseverance". This "state dynamic graph" that evolves over time not only shows what the firefighters are doing and feeling, but more importantly, it reveals the relationship between the two, providing the most comprehensive and profound basis for the final decision.

[0075] Specifically, firefighter A's training content was adjusted by S141 due to "fatigue," and he is currently performing a newly inserted sub-item: "Kneeling Posture Observation and Adjustment." The system has detected that A's current training action is "Kneeling Posture Still." The updated training content requires the target item to be "Kneeling Posture Observation and Adjustment." The system comparison found that A's kneeling posture is very standard, but his head did not perform the expected "left and right scanning observation" action. The behavior execution deviation was calculated as 0.3 (moderate deviation, mainly missing observation action). The "First Level State Diagram" shows a yellow pointer pointing to "partial match."

[0076] Meanwhile, the facial camera captured A's facial image; the micro-expression model detected that his brows were relaxed and his mouth was relaxed; the rPPG algorithm estimated his heart rate to be 75 bpm, and his HRV was within the normal range; the psychophysiological state vector was output as {Stress:0.1, Fatigue:0.3, Focus:0.6}; the "second state diagram" showed that the stress index decreased, the fatigue index decreased slightly, and the focus improved.

[0077] The attention fusion network receives two inputs: a behavioral deviation of 0.3 and a mental state vector. The network analyzes the context: the "adjustment and rest" project. It concludes that in this scenario, the recovery of mental state is more important than the perfect execution of actions, thus giving higher attention weight to the "secondary state." The network fusion analysis shows that although there are slight deficiencies in behavior (not observed), the mental state has significantly improved (stress reduction, fatigue relief), indicating that the intervention is effective and A is actively recovering. On the "state dynamic graph," A's state point is marked as "recovering," with the annotation: "Behavior is basically compliant, mental state has significantly improved, and it is recommended to continue the current adjustment project."

[0078] refer to Figure 6 In step S15, the specific steps are as follows: S151: Collect a dynamic state map, determine multiple state areas based on the detection of the dynamic state map, determine multiple training state levels of firefighters based on the identification of each state area, and determine the corresponding training upgrade node based on the comparison of multiple training state levels. S152: Collect the firefighter's current training subjects, determine the first training adjustment factor based on the training upgrade node and the firefighter's current training subjects, and determine the second training adjustment factor based on the training upgrade node and the firefighter's past training events; S153: Determine the next training content based on the mapping relationship between the first training adjustment factor, the second training adjustment factor, and the training content; mark the training level of the next training content and the training level of the updated training content; the training level of the next training content is greater than the training level of the updated training content.

[0079] In the embodiments of this application, a dynamic state map is collected, multiple state regions are determined based on the detection of the dynamic state map, multiple training state levels of firefighters are determined based on the identification of each state region, and the corresponding training upgrade node is determined based on the comparison of multiple training state levels. This approach takes into account the overall consideration of comparing multiple training state levels and ensures the accuracy of the corresponding training upgrade node.

[0080] At this point, the high-dimensional time series data of the continuous "state dynamic graph" generated by S142 is divided into several continuous time segments with relatively consistent internal features, namely "state regions". The system adopts a sliding window mechanism to calculate the statistical features of key state indicators within each window and applies change point detection algorithms (such as CUSUM or Bayesian change point detection) to identify the boundaries of the state regions. When the algorithm detects a significant change point, it "cuts" the time series at this point, and the time segment between two consecutive change points is defined as a "state region".

[0081] For each state region, the system aggregates all state dynamic graph data within that region to form a region state feature vector containing dimensions such as average behavior matching degree, average stress index, and state stability score. The system inputs this vector into a pre-trained multi-classifier, which is trained to map multi-dimensional performance onto a discrete training state level scale, such as level 1 (struggling), level 2 (mastering), level 3 (proficient), and level 4 (automating). After this step, each state region is labeled with a specific level.

[0082] The system arranges all state regions in chronological order to form a level sequence. A set of upgrade triggering rules is applied to scan this sequence. These rules typically include stability rules, which require a high level to be sustained for a certain period of time to be considered valid, thus filtering out occasional exceptional performances; and transition rules, which look for the turning point from a lower level to a higher level. When the sequence meets these rules, the system marks this transition moment as a "training upgrade node." This node is a timestamp that indicates that from that moment on, the system considers the firefighter to have "graduated" from the current level of difficulty and can enter the next learning stage.

[0083] Specifically, firefighter A is performing the project, and the status dynamic graph of S142 shows that his status is continuously improving; the system collects the status dynamic graph data of A for the most recent 10 minutes; the CUSUM algorithm is used to analyze the weighted time series of his "behavioral matching degree" and "stress index". The algorithm detects a significant change point at T+5 minutes: before this, the stress index is high and fluctuates greatly; after this, the stress index stabilizes at a low level, and the behavioral matching degree stabilizes at a high level; the system divides the time series into two status regions: Region_A:[T,T+5min] and Region_B:[T+5min,T+10min].

[0084] For Region A, the system calculates its region state feature vector as {Avg_Behavior_Match:0.7,Avg_Stress_Level:0.6,Stability_Score:0.4}; the classifier maps it to Level 2 (Mastery); for Region B, the system calculates its region state feature vector as {Avg_Behavior_Match:0.95,Avg_Stress_Level:0.1,Stability_Score:0.9}; the classifier maps it to Level 3 (Proficiency).

[0085] The system obtains the level sequence: […2,2,3,3,…], and the transition occurs at T+5 minutes. The system applies the upgrade trigger rule, assuming the rule is "Level 3 must last for at least 3 minutes". The system checks the duration of Region_B and finds that from T+5 minutes to the current time T+10 minutes, it has lasted for 5 minutes, exceeding the 3-minute threshold. The condition is met, and the system officially marks the time point T+5 minutes as a "training upgrade node".

[0086] Furthermore, the current training subjects of firefighters are collected, and a first training adjustment factor is determined based on the training upgrade node and the current training subjects of firefighters. A second training adjustment factor is determined based on the training upgrade node and the firefighters' past training events. This approach takes into account both the training upgrade node and the firefighters' past training events, ensuring the accuracy of the second training adjustment factor.

[0087] At this point, the system obtains the timestamp corresponding to the "training upgrade node" identified in S151, as well as the "current training subject" that the firefighter is currently performing. Using this timestamp, the system locates the firefighter's current position in the training knowledge base (DAG) used in S141, finding the sub-training project node that the firefighter has just completed or reached mastery. The system executes a graph traversal algorithm on this DAG to find all direct successor nodes of this mastered node. These successor nodes represent the training content that logically follows in the standard training path. The set of attributes of these successor nodes constitutes the "first training adjustment factor," which represents the most direct and logical next challenge direction that follows the current subject's syllabus.

[0088] The system takes into account "training upgrade nodes" and a database of all the firefighter's past training events. It then activates a global skills profiling engine to aggregate and mine historical data from multiple dimensions, constructing a skills proficiency matrix with "skill type" as rows and "proficiency score" as columns. Statistical analysis or ranking algorithms are applied to identify the skill areas with the lowest scores, slowest progress, or highest frequency of anomalies; these areas are marked as individual skill weaknesses. The system outputs one or more labels for the identified "weak skills" or "strong skills." These labels constitute the "second training adjustment factor," representing a strategic direction for addressing gaps in the firefighter's individual skills profile.

[0089] Specifically, when firefighter A is performing "Subject C", S151 identifies a "training upgrade node" at T+5 minutes because he has just mastered the "Kneeling Posture Observation and Adjustment" item to the "Proficiency" level. The system obtains the upgrade node timestamp T+5 minutes and the current subject "Subject C". The system locates the node in the DAG of "Subject C" and finds that the node that A just mastered at T+5 minutes is Node_R: "Kneeling Posture Observation and Adjustment". The system searches for the direct successor node of Node_R in the DAG and finds only one: Node3: "Crawling through a narrow passage". The system extracts the attributes of Node3: {Difficulty:4, Target_Skill: "Complex Environment Movement", Physical_Demand: "High"}. Therefore, the first training adjustment factor is determined to be "Complex Environment Movement", specifically pointing to the "Crawling" item.

[0090] The system acquires the upgrade node and accesses the database of all historical training events for firefighter A. The skills profiling analysis engine is activated, scanning all of A's training records from the past six months, including multiple subjects such as "high-altitude rescue" and "chemical spill response." The skills proficiency matrix generated by the analysis engine shows that A's "demolition" skills score is 9.2 / 10, "equipment operation" is 8.8 / 10, but the "complex environment movement" skills score is only 6.5 / 10. Moreover, in multiple training sessions involving narrow spaces and low visibility environments, there have been abnormal records of "route deviation" or "unstable posture." The system identifies "complex environment movement" as a significant weakness in firefighter A's capabilities. Therefore, the second training adjustment factor is also determined to be "complex environment movement."

[0091] Therefore, the next training content is determined based on the mapping relationship between the first training adjustment factor, the second training adjustment factor, and the training content. The training level of the next training content and the training level of the updated training content are marked. The training level of the next training content is greater than the training level of the updated training content. This approach takes into account the overall consideration of the mapping relationship between the first training adjustment factor, the second training adjustment factor, and the training content, ensuring the accuracy of the next training content. At the same time, a dynamic status graph of the firefighter is introduced to further control the training upgrade node. This approach achieves an overall consideration of the training upgrade node, the firefighter's current training subject, and the firefighter's past training events, thereby improving the accuracy of the next training content.

[0092] At this point, the system integrates the two adjustment factors, "task-oriented" and "ability-oriented," derived from S152, and selects the most suitable "next training content" from the training knowledge base. The system processes these two factors through a decision fusion engine, which can employ strategies such as weighted voting, intersection maximization, or priority. For example, when the two factors point to the same skill domain, the system will directly select a project of moderate difficulty within that domain. The engine will query the training knowledge base, match the attributes of the candidate projects with the adjustment factors, and finally output a unique, selected "next training content" object.

[0093] The system executes a rigorous dual query and comparison process: it queries and marks the inherent difficulty level of the "next training content"; it queries and marks the difficulty level of the "updated training content" that the firefighter has just mastered; the system executes a verification logic: the decision is considered valid only when the level of the "next training content" is strictly higher than the level of the "current training content". This verification is the cornerstone of S153. It ensures that the system's decisions are "upward", avoiding assigning trainees repetitive tasks of lower or equal difficulty, thereby ensuring the efficiency and challenge of training.

[0094] Specifically, firefighter A reaches the upgrade node in "Subject C". S152 analysis shows that the first and second training adjustment factors are both "movement in complex environments". The decision fusion engine receives the two adjustment factors and finds that they are highly consistent, both pointing to "movement in complex environments". The system queries the training knowledge base for sub-items related to "Subject C" and belonging to this skill, and obtains a candidate list: ["crawl", "side crawl", "roll forward"]. Since firefighter A has never carried out movement training under this subject before, the system selects the most basic "crawl" as the most suitable next step according to the DAG order. The system determines that the "next training content" is "crawl through a narrow passage".

[0095] The system queries the knowledge base. The "Crawling Forward" project requires higher physical fitness and more complex posture control. It is also the successor to "Kneeling Observation". Its Training_Level is marked as 4. The system reviews the project that A has just mastered, namely "Kneeling Observation and Adjustment" inserted in S141. This project is a low-intensity recovery project. Its Training_Level is marked as 3. The system compares the two levels. Next_Training_Level(4)>Current_Training_Level(3). The verification is successful and the decision is valid.

[0096] Final output: The system will issue the following instructions to firefighter A through his AR glasses or headphones: "Status assessment completed, excellent performance; prepare to begin the next training exercise: crawling through a narrow passage; training level: 4; difficulty increased, please pay attention to proper technique."

[0097] Please see Figure 7 , Figure 7 This is a schematic diagram of the structural composition of an image recognition-based firefighter training status identification system according to an embodiment of the present invention; the image recognition-based firefighter training status identification system includes: The training animation module 21 is used to determine multiple training images of firefighters in the training area based on multiple cameras and the Internet of Things, and to determine the training animation of firefighters based on multiple training images, the corresponding image shooting time and the shape of the training area. Training event module 22 is used to determine multiple sub-training behaviors based on image recognition of the firefighter's training animation, and to determine the training events of the firefighter in each training subject based on the behavior content, corresponding behavior path and corresponding training subject of each sub-training behavior. The current training status module 23 is used to determine the abnormal training behavior of firefighters based on the identification of the training event, and to determine the current training status of firefighters based on the content of the abnormal training behavior, the corresponding area where the behavior occurred, and the firefighters' previous training events. The status animation module 24 is used to determine the updated training content based on the firefighter's current training subject and corresponding current training status, and to determine the firefighter's status animation based on the updated training content, the firefighter's current training action and corresponding facial image. The training content module 25 is used to determine the corresponding training upgrade node based on image recognition of the state dynamic graph. The next training content is determined based on the training upgrade node, the firefighter's current training subject and the firefighter's past training events. At this time, the training level of the next training content is higher than that of the updated training content.

[0098] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for identifying the training status of firefighters based on image recognition, characterized in that, include: In the firefighters' training ground, multiple training images of firefighters are determined based on multiple cameras and the Internet of Things in the training ground. Based on multiple training images, the corresponding image shooting time and the shape of the training ground, a dynamic training map of the firefighters is determined. Multiple sub-training behaviors are determined by image recognition of firefighters' training dynamics. The training events of firefighters in each training subject are determined based on the content of each sub-training behavior, the corresponding behavior path, and the corresponding training subject. Based on the identification of this training event, the abnormal training behavior of the firefighter is determined, and the current training status of the firefighter is determined based on the content of the abnormal training behavior, the corresponding area where the behavior occurred, and the firefighter's past training events. The updated training content is determined based on the firefighter's current training subjects and corresponding current training status. The firefighter's status animation is determined based on the updated training content, the firefighter's current training actions, and the corresponding facial images. The corresponding training upgrade node is determined by image recognition based on the state dynamic graph. The next training content is determined based on the training upgrade node, the firefighter's current training subject and the firefighter's past training events. At this time, the training level of the next training content is higher than that of the updated training content.

2. The method for identifying the training status of firefighters based on image recognition according to claim 1, characterized in that, In the firefighter training area, multiple training images of firefighters are determined based on multiple cameras and the Internet of Things (IoT). A dynamic training diagram of the firefighters is then determined based on these multiple training images, the corresponding image capture time, and the shape of the training area, including: The system marks the training sites of firefighters and identifies multiple cameras in the training sites based on the detection of the training sites. Based on the positions of the multiple cameras, the current position of the firefighters, and the Internet of Things, it determines multiple training images of the firefighters. The multiple training images present different dimensions of the firefighters' training actions. The training site's shape is collected, and a first training image is determined based on multiple training images and the shape of the training site. A second training image is determined based on multiple training images and the corresponding image capture time. A dynamic training image of the firefighters is determined based on the first and second training images.

3. The method for identifying the training status of firefighters based on image recognition according to claim 1, characterized in that, The process involves determining multiple sub-training behaviors based on image recognition of firefighters' training dynamics, and identifying training events for each training subject based on the content, corresponding behavioral path, and corresponding training subject of each sub-training behavior. This includes: Based on the detection of firefighters' training dynamic map, the corresponding training action area is determined, and the corresponding sub-training behavior is determined based on the identification of each training action area. Multiple sub-training behaviors are collected, and the behavioral content of each sub-training behavior is marked. Based on the identification of each sub-training behavior, the corresponding behavior path is determined. Based on the behavior content of each sub-training behavior, the corresponding behavior path, and the corresponding training subject, the training events of firefighters in each training subject are determined.

4. The method for identifying the training status of firefighters based on image recognition according to claim 1, characterized in that, The process of identifying abnormal training behaviors of firefighters based on the recognition of the training event, and determining the current training status of firefighters based on the content of the abnormal training behaviors, the corresponding areas where the behaviors occurred, and the firefighters' past training events, includes: The training event is collected, and multiple sub-abnormal actions of the firefighter are identified based on the identification of the training event. The abnormal training behavior of the firefighter is determined according to the action content, corresponding action trajectory and corresponding action time of the multiple sub-abnormal actions, and the action content of the abnormal training behavior of the firefighter is marked.

5. The method for identifying the training status of firefighters based on image recognition according to claim 4, characterized in that, The method of identifying abnormal training behavior of firefighters based on the recognition of the training event, and determining the current training status of firefighters based on the content of the abnormal training behavior, the corresponding area where the behavior occurred, and the firefighters' past training events, further includes: The corresponding behavior occurrence area is determined by tracing the abnormal training behavior of firefighters, and the first state coefficient is determined based on the content of the abnormal training behavior and the corresponding behavior occurrence area. Collect past training events of firefighters, determine the second state coefficient based on the content of the firefighter's abnormal training behavior and the firefighter's past training events, and determine the firefighter's current training state based on the mapping relationship between the first state coefficient, the second state coefficient and the current training state.

6. The method for identifying the training status of firefighters based on image recognition according to claim 1, characterized in that, The process involves determining updated training content based on the firefighter's current training subject and corresponding current training status, and determining a dynamic status image of the firefighter based on the updated training content, the firefighter's current training actions, and the corresponding facial image, including: The system collects the firefighters' current training subjects, identifies multiple sub-training items based on the identification of the firefighters' current training subjects, and determines the updated training content based on the content of the multiple sub-training items, the corresponding order of the items, and the firefighters' current training status.

7. The method for identifying the training status of firefighters based on image recognition according to claim 6, characterized in that, The process of determining updated training content based on the firefighter's current training subject and corresponding current training status, and determining a dynamic status image of the firefighter based on the updated training content, the firefighter's current training actions, and corresponding facial images, also includes: The system collects the firefighters' current training actions and determines the first-level state diagram based on the firefighters' current training actions and the updated training content. At the same time, it collects facial images of the firefighters during training and determines the second-level state diagram based on the updated training content and the facial images of the firefighters during training. Based on the first-level state diagram and the second-level state diagram, a dynamic state diagram of the firefighters is determined.

8. The method for identifying the training status of firefighters based on image recognition according to claim 1, characterized in that, The image recognition based on the state dynamic graph determines the corresponding training upgrade node, and the next training content is determined based on the training upgrade node, the firefighter's current training subject, and the firefighter's past training events, including: The system collects dynamic state maps, identifies multiple state regions based on the detection of these maps, determines multiple training state levels for firefighters based on the identification of each state region, and identifies corresponding training upgrade nodes based on the comparison of multiple training state levels.

9. The method for identifying the training status of firefighters based on image recognition according to claim 8, characterized in that, The method of determining the corresponding training upgrade node based on image recognition of the state dynamic graph, and determining the next training content based on the training upgrade node, the firefighter's current training subject, and the firefighter's past training events, also includes: Collect the firefighters' current training subjects, determine the first training adjustment factor based on the training upgrade node and the firefighters' current training subjects, and determine the second training adjustment factor based on the training upgrade node and the firefighters' past training events; The next training content is determined based on the mapping relationship between the first training adjustment factor, the second training adjustment factor, and the training content. The training level of the next training content and the training level of the updated training content are marked, and the training level of the next training content is greater than the training level of the updated training content.

10. A system for recognizing the training status of firefighters based on image recognition, characterized in that, The image recognition-based firefighter training status identification system is applied to the image recognition-based firefighter training status identification method as described in any one of claims 1-9, wherein the image recognition-based firefighter training status identification system comprises: The training animation module is used to determine multiple training images of firefighters in the training area based on multiple cameras and the Internet of Things. Based on multiple training images, the corresponding image capture time and the shape of the training area, the training animation of the firefighters is determined. The training event module is used to determine multiple sub-training behaviors based on image recognition of firefighters' training animations, and to determine the training events of firefighters in each training subject based on the behavior content, corresponding behavior path and corresponding training subject of each sub-training behavior. The current training status module is used to determine the abnormal training behavior of firefighters based on the identification of the training event. The current training status of firefighters is determined based on the content of the abnormal training behavior, the corresponding area where the behavior occurred, and the firefighters' previous training events. The status animation module is used to determine the updated training content based on the firefighter's current training subject and corresponding current training status, and to determine the firefighter's status animation based on the updated training content, the firefighter's current training action and corresponding facial image. The training content module is used to determine the corresponding training upgrade node based on image recognition of the state dynamic graph. Based on the training upgrade node, the firefighter's current training subject and the firefighter's past training events, the next training content is determined. At this time, the training level of the next training content is higher than that of the updated training content.

Images