Firefighter competence level assessment method and system based on fire scene environment

Abstract

The present invention discloses a method and system for evaluating the ability level of firefighters based on a fire scene environment. The present invention relates to the technical field of ability level evaluation methods. The sub-ability level of each fire drill node is determined based on the node position of each fire drill node, fire behavior events, and the firefighter's previous fire drill records. The firefighter's ability level is evaluated based on the node position of each fire drill node, the sub-ability level, and the physical data set corresponding to each fire drill node, thereby improving the accuracy of the firefighter's ability level evaluation. Based on the detection of each sub-ability level, the sub-ability level in an abnormal state is determined to determine the corresponding abnormal fire behavior; in each fire drill node, multiple abnormal fire action features are determined based on the identification of abnormal fire behavior, and abnormal fire factors are determined based on the multiple abnormal fire action features and the corresponding physical data set, thereby improving the accuracy of identifying abnormal fire factors.

Description

Technical Field

[0001] The present invention relates to the technical field of capability level assessment methods, and in particular to a method and system for assessing the capability level of firefighters based on a fire scene environment. Background Art

[0002] With the development of science and technology, AR glasses are gradually applied to people's lives and can be used in the daily drills of firefighters. Firefighters wear AR glasses and enter the fire environment constructed by AR glasses. In the existing technology, AR glasses enter the fire environment as virtual characters, and conduct corresponding firefighting training in the fire environment, collect corresponding firefighting training events, and conduct capability level assessment based on the firefighting actions of the training events, ignoring the capability level of each firefighting drill node and the physical data set corresponding to each firefighting drill node, resulting in low accuracy in the existing firefighter capability level assessment. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a method and system for evaluating the ability level of firefighters based on a fire scene environment.

[0004] An embodiment of the present invention provides a method for evaluating the ability level of firefighters based on a fire scene environment, comprising: creating a fire scene environment in which the firefighters are located based on AR glasses, and collecting fire drill images of the firefighters in the fire scene environment; determining the fire drill path of the firefighters based on the detection of the fire drill images, and marking the corresponding firefighters' fire behavior events at each fire drill node in the fire drill path; determining the sub-ability level of each fire drill node based on the node position, fire behavior events and previous fire drill records of each fire drill node, and evaluating the firefighters' ability level based on the node position, sub-ability level and physical data set corresponding to each fire drill node; if the firefighters' ability level is lower than a preset ability level threshold, determining the sub-ability level in an abnormal state based on the detection of each sub-ability level to determine the corresponding abnormal fire behavior; in each fire drill node, determining multiple abnormal fire action features based on the identification of abnormal fire behavior, determining abnormal fire factors based on the multiple abnormal fire action features and the corresponding physical data set, and matching corresponding fire optimization actions.

[0005] An embodiment of the present invention provides a firefighter capability level assessment system based on a fire scene environment. The firefighter capability level assessment system based on a fire scene environment is applied to the firefighter capability level assessment method based on a fire scene environment. The firefighter capability level assessment system based on a fire scene environment includes:

[0006] The fire drill screen module is used to create a fire scene environment in which firefighters are located based on AR glasses and collect fire drill images of firefighters in this fire scene environment;

[0007] A fire behavior event module is used to determine the fire drill path of the firefighter based on the detection of the fire drill screen, and mark the corresponding fire behavior event of the firefighter at each fire drill node in the fire drill path;

[0008] A capability level module, configured to determine the sub-ability level of each fire drill node based on the node position of each fire drill node, fire behavior events, and the firefighter's previous fire drill records, and to evaluate the firefighter's capability level based on the node position of each fire drill node, the sub-ability level, and the physical data set corresponding to each fire drill node;

[0009] An abnormal firefighting behavior module is used to determine the sub-ability level in an abnormal state based on the detection of each sub-ability level if the firefighter's ability level is lower than a preset ability level threshold, so as to determine the corresponding abnormal firefighting behavior;

[0010] The abnormal fire factor module is used to determine multiple abnormal fire action features based on the identification of abnormal fire behavior in each fire drill node, determine abnormal fire factors based on multiple abnormal fire action features and corresponding body data sets, and match corresponding fire optimization actions.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] In an embodiment of the present invention, through the method in the embodiment of the present invention, the sub-capability level of each fire drill node is determined according to the node position of each fire drill node, fire behavior events and previous fire drill records of firefighters, and the ability level of firefighters is evaluated based on the node position of each fire drill node, the sub-capability level and the physical data set corresponding to each fire drill node. Fire behavior events are introduced, and the overall consideration of the node position of each fire drill node, the sub-capability level and the physical data set corresponding to each fire drill node is compatible, thereby improving the accuracy of the evaluation of the firefighter's ability level.

[0013] Therefore, if the firefighter's ability level is lower than the preset ability level threshold, the sub-ability level in an abnormal state is determined based on the detection of each sub-ability level to determine the corresponding abnormal firefighting behavior; in each fire drill node, multiple abnormal firefighting action features are determined based on the identification of abnormal firefighting behavior, and abnormal firefighting factors are determined based on multiple abnormal firefighting action features and corresponding physical data sets. Abnormal firefighting behavior is introduced, and the overall consideration of multiple abnormal firefighting action features and corresponding physical data sets is realized, thereby improving the identification accuracy of abnormal firefighting factors and matching corresponding firefighting optimization actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 1 is a flow chart of a method for evaluating the ability level of firefighters based on a fire scene environment in an embodiment of the present invention;

[0015] Figure 2 1 is a flow chart of step S11 in the fire scene environment-based firefighter competence level assessment method in an embodiment of the present invention;

[0016] Figure 3 1 is a flow chart of step S12 in the method for evaluating the ability level of firefighters based on the fire scene environment in an embodiment of the present invention;

[0017] Figure 4 1 is a flow chart of step S13 in the fire scene environment-based firefighter competence level assessment method in an embodiment of the present invention;

[0018] Figure 5 1 is a flow chart of step S14 in the fire scene environment-based firefighter competence level assessment method in an embodiment of the present invention;

[0019] Figure 6 1 is a flow chart of step S15 in the fire scene environment-based firefighter competence level assessment method in an embodiment of the present invention;

[0020] Figure 7 Schematic diagram of the structure of a firefighter's ability level assessment system based on a fire scene environment in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0022] See also Figures 1 to 7 A firefighter competence level assessment method based on a fire scene environment is applied to a firefighter competence level assessment scenario. The firefighter competence level assessment method based on a fire scene environment includes:

[0023] Step S11: creating a fire scene environment where firefighters are located based on AR glasses, and collecting fire drill images of firefighters in the fire scene environment;

[0024] Step S12: determining the fire drill path of the firefighter based on the detection of the fire drill screen, and marking the corresponding firefighting behavior event of the firefighter at each fire drill node in the fire drill path;

[0025] Step S13: Determine the sub-ability level of each fire drill node based on the node position of each fire drill node, fire behavior events, and the firefighter's previous fire drill records, and evaluate the firefighter's ability level based on the node position of each fire drill node, the sub-ability level, and the physical data set corresponding to each fire drill node;

[0026] Step S14: If the firefighter's ability level is lower than the preset ability level threshold, the sub-ability level in an abnormal state is determined based on the detection of each sub-ability level to determine the corresponding abnormal firefighting behavior;

[0027] Step S15: At each fire drill node, multiple abnormal firefighting action features are determined based on the identification of abnormal firefighting behaviors, abnormal firefighting factors are determined based on the multiple abnormal firefighting action features and the corresponding body data sets, and corresponding firefighting optimization actions are matched;

[0028] refer to Figure 2 In step S11, the specific steps are:

[0029] S111: When the firefighter wears the AR glasses, a fire drill type is determined according to the fire drill category selected by the firefighter and the corresponding fire drill progress, and a fire scene environment in which the firefighter is located is created based on the fire drill type and the surrounding environment detected by the AR glasses;

[0030] S11: The firefighter enters the fire scene environment as a virtual character, and the virtual character synchronizes the on-site actions of the firefighter. According to the firefighter's fire drill in the fire scene environment, the firefighter's fire drill picture in the fire scene environment is collected.

[0031] In an embodiment of the present application, after the firefighter puts on the AR glasses, he needs to use some interactive method (for example, the glasses' built-in touchpad, voice commands) to select the specific category of this drill. This category refers to the type of scene in which the fire occurs, such as "multi-story residential building fire", "underground shopping mall fire", "chemical plant leakage fire", etc.; at the same time, the system also needs to know the firefighter's "fire drill progress" information, which can be understood as the firefighter's training stage or proficiency level, such as "basic operation for novice", "intermediate rescue skills", "advanced complex environment handling", etc.

[0032] After the system receives the category and progress information, the internal "drill type library" or algorithm will be parsed. This library stores the specific drill modes, difficulty parameters, key training points, etc. of various categories (such as residential building fires) at different progress levels (such as novice, intermediate, and advanced); the system matches the most appropriate "drill type" based on the input; for example, "multi-story residential building fire" at the "novice basic operation" progress corresponds to "familiarity with the structure, basic fire extinguisher use, and simple guidance of trapped people"; and at the "advanced complex environment handling" progress, it corresponds to "multi-floor coordinated firefighting, search and rescue of trapped people in complex obstacle areas, and handling of secondary disaster risks."

[0033] AR glasses have multiple built-in sensors for real-time perception of the firefighters' physical environment, including: cameras: capturing real-time video streams in front of them to identify objects, planes, boundaries, etc. in space; depth sensors (such as LiDAR or structured light): measuring the distance to objects in the environment and building a three-dimensional space map; inertial measurement unit (IMU): detecting the posture, acceleration, and angular velocity of the head for positioning and stabilizing the image; other sensors: such as microphones (detecting ambient sounds and assisting in scene simulation), and even combining with GPS or indoor positioning systems (such as UWB) to obtain more accurate location information.

[0034] The system dynamically generates and configures the virtual fire scene environment based on the scene elements (such as flame location, smoke concentration, obstacles, location of trapped persons, etc.) and difficulty parameters contained in the determined "drill type" and combined with the "surrounding environment" data obtained in step 3 (such as the actual size of the room, wall location, door and window location, table and chair placement, etc.).

[0035] At this point, the system will accurately superimpose virtual flames, smoke, traces of destruction, etc. onto corresponding positions in the real environment; for example, if the reality is a simulated room for training, the system will generate flames from a certain window and smoke filling a specific area at the virtual level based on the actual layout of the room; key elements in the virtual environment will be configured according to the type of drill; for example, if it is a "search and rescue of trapped persons" task, the system will set the positions of several "trapped persons" at the virtual level and assign them different states (such as minor injuries, inability to move, etc.); if it is a "fire extinguishing" task, the size, intensity, and spread speed of the fire source will be set, and these parameters will adjust the difficulty according to the progress of the drill; the environment contains some virtual elements that can interact with firefighters, such as virtual valves that can be closed, virtual obstacles that can be destroyed, etc., which will be created according to the type of drill; in the physical space of the firefighters, a highly realistic virtual fire scene environment is created that conforms to real-life constraints and contains specific training objectives and challenges.

[0036] Specifically, suppose there is a firefighter named "Xiao Wang" whose training progress is evaluated by the system as "intermediate rescue skills". He chooses to conduct a drill in the "multi-story residential building fire" category; Xiao Wang puts on AR glasses, selects the "multi-story residential building fire" category through the connected mobile phone APP, and confirms the "intermediate rescue skills" progress displayed by the system (this progress is automatically set based on the training tasks he has completed before).

[0037] After system analysis, the drill type corresponding to "Multi-story residential building fire - intermediate rescue skills" is matched from the drill type library. This type includes the following elements: Mission objective: Search and rescue trapped people on the designated floor within a limited time, and control the spread of fire to the area; Scene elements: Virtual flames appear in a specific room, the smoke concentration is medium, some virtual doors and windows are blocked by virtual "obstacles" (such as virtual collapsed furniture), and there are 1-2 virtual "trapped people" who need to be found and "rescued" (for example, you need to walk to their virtual position and perform a "comfort" or "mark found" action); Difficulty parameters: The fire spreads at a moderate speed, smoke visibility affects vision, and it is necessary to operate a virtual fire hose to extinguish the fire, and the operation is relatively complex.

[0038] Xiao Wang enters a preset training room (the layout information of this room has been pre-entered into the system, or scanned in real time by AR glasses); the glasses' camera and depth sensor start working, identifying the walls, floor, ceiling, and training tables and chairs placed in the room; the IMU ensures that the image is stable and follows Xiao Wang's head movement; the system confirms that Xiao Wang is at the entrance of the room.

[0039] The system starts to create a virtual fire scene based on the drill type of "Intermediate Rescue Skills" and the detected room environment: the system determines that the layout of the room is suitable for simulating a standard apartment in a residential building, and it sets the virtual flame in a "virtual room" near the inside of the room (corresponding to a corner in reality or an area defined by a curtain / marker), and uses AR effects to make the flames look like they are coming out from behind the walls or furniture in that area; the smoke effect is rendered, filling the upper half of the room, reducing visibility but not completely blocking the view, simulating a medium smoke concentration; the system is at the other end of the room (according to the "trapped person" set according to the drill type) A virtual "trapped person" mark (a translucent virtual human outline or a specific icon) is generated at the "trapped person" location; a virtual "obstacle" is placed on the way to the "trapped person" location (for example, a virtual fallen cabinet model that blocks part of the path); a virtual fire hose is placed near the entrance of the room; now, what Xiao Wang sees through the AR glasses is a residential building fire scene that seems to have actually occurred. He needs to complete search, firefighting, rescue and other tasks in this environment that is a fusion of real and virtual elements, and the system begins to collect his action data in preparation for subsequent evaluation.

[0040] Furthermore, the firefighter enters the fire scene environment as a virtual character, and the virtual character synchronizes the firefighter's on-site actions. Based on the firefighter's fire drill in the fire scene environment, the firefighter's fire drill pictures in the fire scene environment are collected. The firefighter's fire drill in the fire scene environment is introduced and the firefighter's fire drill pictures in the fire scene environment are collected.

[0041] At this point, when the fire scene environment is created, the system will generate a "virtual character" (Avatar) representing the firefighter in the virtual environment. This virtual character is the firefighter's incarnation in the virtual world; the key is that this virtual character is set to be "bound" to the real firefighter; visually, what the firefighter sees through the AR glasses is: he seems to be wearing equipment with AR display effects, in a real (or simulated real) physical space, while virtual fire scene elements (flames, smoke, trapped people, etc.) are superimposed around him; the virtual character is usually located at the approximate position of the firefighter in the physical space and follows the firefighter's physical movements. This virtual character is the basis for subsequent action synchronization and behavior analysis.

[0042] The system needs to capture the various actions of firefighters in the physical world in real time and transmit this action information to the virtual character so that it can make corresponding actions in the virtual environment; action synchronization relies on multiple sensors and data sources, including: AR glasses themselves: capturing head posture (rotation, tilt), line of sight direction; Inertial Measurement Unit (IMU) sensor: integrated in AR glasses, fire suits, gloves, boots and other equipment, used to capture the posture, acceleration and angular velocity of body parts (such as hands, arms, torso, legs); Camera: The camera on the AR glasses is used to capture the approximate position and posture of the hands or other key parts.

[0043] After the system receives data from the sensor, it needs to parse it and map it to the corresponding skeletal joints or action sequences of the virtual character; for example, when a firefighter raises his right hand, the IMU sensor on his hand will record this action. After the system parses it, the virtual character's right hand will also be raised in the virtual environment. This process requires very low latency and high precision to avoid a "sense of disconnection" and ensure that the virtual character's movements appear natural and smooth, and can accurately reflect the firefighter's actual operations.

[0044] During the firefighters' drills, the system needs to continuously record everything. The collected "images" not only refer to the screen output of the AR glasses (that is, the scenes seen by the firefighters themselves), but more importantly, the data recorded by the system from the perspective of the "evaluator", including the virtual character's actions, the state of the virtual environment, and the interaction process between the firefighters and the environment. The collected data usually includes: the virtual character's skeletal animation data: a detailed record of the position and posture of each joint of the virtual character over time, which is the basis for analyzing the action; the virtual character's position and orientation data: recording the coordinates and orientation of the virtual character in the virtual environment for subsequent path analysis; the virtual environment's status data: such as changes in flame intensity, smoke concentration, and the status of trapped people over time; interaction event data: recording the interaction between firefighters and objects in the virtual environment, such as picking up objects, using tools, triggering alarms, etc.

[0045] Specifically, firefighter Xiao Wang is conducting an "intermediate rescue skills" drill for a "multi-story residential building fire"; Xiao Wang puts on AR glasses and selects the drill category and progress; the system creates a virtual fire scene based on the room he is in (physical environment) and the drill requirements; Xiao Wang sees through the glasses the appearance of virtual flames, smoke, and a virtual marker representing the trapped person in the room.

[0046] Xiao Wang starts to act: he moves forward a few steps; the IMU on his feet and the head tracking sensor on the glasses capture his movements and position changes, and the virtual character also moves forward in the virtual environment accordingly; he raises his right hand and picks up the virtual water gun from the virtual water gun stand; the IMU sensor on his hand captures the posture of his hand and arm, and the virtual character's right hand also accurately raises and "picks up" the water gun model; he turns his body and aims in the direction of the virtual flame; the rotation of his head is captured by the glasses, and the virtual character also turns its head; he simulates pulling the trigger (through a handle button or gesture recognition); the system records this interaction event, and the virtual water gun model shows a water column effect; the system continuously transmits Xiao Wang's physical action data (movement, posture, interaction) to the virtual character in real time, ensuring that the virtual character is highly consistent with his own movements from Xiao Wang's perspective and from the perspective of the system recording.

[0047] Image acquisition: During Xiao Wang's entire drill (for example, from entering the room, observing the environment, moving, and using the water gun): the system continuously records the virtual character's skeletal animation data (how he moves, how he holds the gun, and how he turns); the system records the virtual character's position coordinates (where he walks and what path he forms); the system records interactive events such as when he "picks up" the water gun, when he "aims" at the flames, and when he "discovers" the trapped person; the system also records the camera image of the AR glasses, so that Xiao Wang's actual operations and expressions can be seen; all of this data is time-stamped to form a complete record of the drill process.

[0048] refer to Figure 3 In step S12, the specific steps are:

[0049] S121: Collecting a fire drill screen, determining multiple drill detection areas based on the division of the fire drill screen, determining corresponding sub-fire drill paths based on the simultaneous identification of the multiple drill detection areas, collecting multiple sub-fire drill paths, and determining a firefighter's fire drill path based on the synthesis of the multiple sub-fire drill paths;

[0050] S122: Determine multiple fire drill nodes based on the detection of the fire drill path, where the multiple fire drill nodes are distributed at different locations of the fire drill path;

[0051] S123: Determine multiple firefighting behaviors of firefighters in the firefighting nodes based on the node positions, times, and firefighting pictures of the multiple firefighting nodes, and determine corresponding firefighting behavior events according to the synthesis of the multiple firefighting behaviors.

[0052] In an embodiment of the present application, fire drill footage is collected, and the system cannot directly process infinite continuous space and needs to discretize it; there are many ways to divide the detection area, which are usually combined with the design of the virtual fire scene and the typical process of the fire drill: based on the spatial structure: the fire scene is divided into rooms, corridors, corners, specific functional areas (such as bedrooms, kitchens, living rooms), etc.; based on grid division: the entire drill area is divided into grid units of uniform size; based on key nodes: key positions in the fire scene, such as entrances, exits, fire sources, positions of trapped persons, and areas around important obstacles, are defined as independent detection areas; the result of the division is a "map" composed of multiple clear "grids" or "areas", each of which has a unique identifier (ID).

[0053] The system begins to track the firefighter's movements in real time. By analyzing the collected images and sensor data, the system can determine which detection area the firefighter's current virtual character (Avatar) or physical body is in. When a firefighter moves from one detection area to another, a "movement segment" is formed. Connecting continuous movement segments between adjacent detection areas forms a "sub-fire drill path," which describes a series of detection areas that the firefighter passes through continuously at the fire scene. For example, a sub-path can be formed from the entrance area > corridor area 1 > living room area.

[0054] During the entire drill, firefighters will continue to move through different combinations of detection areas; the system will continuously identify new sub-fire drill paths, which vary in length and include straight-line movements, turns, detours, etc.; the system needs to record all identified sub-paths to form a sub-path collection.

[0055] When the drill is over, the system stitches together all the collected sub-fire drill paths in the order in which they occurred and in spatial continuity. This stitching process ensures that the connections between the sub-paths are reasonable (for example, the end detection area of ​​sub-path A is the starting detection area of ​​sub-path B). Ultimately, all the sub-paths are connected to form a complete "fire drill path" that describes the movement trajectory of the firefighters throughout the drill. This path can be represented by a series of detection area IDs arranged in sequence, or drawn as a visual trajectory line.

[0056] Specifically, suppose firefighter Xiao Wang is conducting a fire drill simulating a residential building fire. The fire scene environment has been created by S111, including areas such as the entrance, corridor, living room, kitchen, master bedroom, and second bedroom. Xiao Wang puts on AR glasses and sees the real environment (or a purely virtual environment) superimposed with virtual flames, smoke, and obstacles. The front camera of the AR glasses continuously captures the image and records Xiao Wang's head posture.

[0057] Based on the fire scene design, the system divides the environment into the following detection zones (denoted by letters and numbers): A1: Entrance; B1: Corridor start; B2: Corridor corner; C1: Living room; D1: Kitchen entrance; D2: Kitchen operation area; E1: Master bedroom entrance; E2: Master bedroom center; F1: Second bedroom entrance; F2: Second bedroom center. Xiao Wang begins his journey: He starts at A1 (entrance) and walks to B1 (corridor start). The system identifies subpath 1 as A1 > B1. He continues down the corridor to B2 (corridor corner). The system identifies subpath 2 as B1 > B2. He enters C1 (living room). The system identifies subpath 3 as B2 > C1.

[0058] After a brief observation in the living room, he walked towards D1 (kitchen entrance); the system identified: sub-path 4 = C1>D1; he entered D2 (kitchen operation area) to extinguish the fire; the system identified: sub-path 5 = D1>D2; after extinguishing the fire, he left the kitchen and walked towards E1 (master bedroom entrance); the system identified: sub-path 6 = D2>E1; he entered E2 (center of the master bedroom) to search; the system identified: sub-path 7 = E1>E2.

[0059] After he found the "trapped person", he left the master bedroom and walked towards F1 (the entrance to the second bedroom) to continue searching; the system identified: sub-path 8 = E2>F1; he entered F2 (the center of the second bedroom) to search; the system identified: sub-path 9 = F1>F2; after the search was completed, he returned along the same route, passing through F1, E1, D1, B2, B1, and A1 to leave; during the return process, the system will continue to identify corresponding sub-paths, such as F2>F1, F1>E1, E1>D1, D1>B2, B2>B1, B1>A1.

[0060] The system records all the identified subpaths (at least nine outbound subpaths, plus several return subpaths). After the drill, the system connects all the subpaths in chronological order and spatial continuity. For example, the complete outbound path is: A1>B1>B2>C1>D1>D2>E1>E2>F1>F2; the return path is: F2>F1>E1>D1>B2>B1>A1. By connecting the outbound and return paths, we get Xiao Wang's complete fire drill path throughout the entire drill.

[0061] The system needs to use the structured fire drill path data that has been generated in S121; the system will "scan" or "analyze" this path to find points that meet specific conditions. This "detection" process can be based on a variety of rules or algorithms: Based on spatial features: Identify key geometric feature points on the path, such as the starting / end point of the corridor, the entrance / exit of the room, corners, stairs / ladders, virtual fire sources / smoke centers, virtual trapped person locations, etc.; Based on preset task points: According to the drill task design, mark the locations on the path that must be passed or specific operations need to be performed, such as designated checkpoints, fire extinguishing points, rescue points, etc.

[0062] If the system detects certain specific behaviors in real time (such as using a fire extinguisher, opening / closing doors and windows, and interacting with virtual humans), a node is set at the location where the behavior occurs; based on time / distance thresholds: a node is automatically set at a certain distance or time interval on the path for time series analysis. These detection rules can be used in combination to fully cover the points that need attention.

[0063] Based on the detection results of the previous step, the system clearly marks the specific node locations on the fire drill path. These nodes usually have the following properties: unique identifier: each node has a unique number or name; spatial coordinates: the specific location of the node in the virtual fire environment (for example, the specific coordinates of the center of the room, the corner of the corridor); node type: classified according to the function or meaning of the node, such as "entry node", "search node", "fire extinguishing node", "rescue node", "exit node", "critical corner node", etc.

[0064] Fire drill nodes will not be concentrated in only one section of the path, but will be scattered at various key or representative locations of the path according to the fire environment and drill tasks; for example, one node is located at the entrance, another at the first corner, another near the virtual fire source, and another at the virtual trapped person's location.

[0065] Specifically, assuming that through S121, we have determined Xiao Wang’s complete drill path, such as: A1 (entrance)>B1>B2>C1>D1>D2 (master bedroom entrance)>E1>E2 (master bedroom center / trapped person position)>F1 (secondary bedroom entrance)>F2 (secondary bedroom center)>F1>E1>D1>B2>B1>A1 (exit).

[0066] The system first scans the path and identifies spatial features: entrance A1, corridor B1-B2, master bedroom entrance D2, master bedroom center E2 (virtual trapped person location), second bedroom entrance F1, second bedroom center F2, exit A1; based on the drill mission (search and rescue), the system identifies key mission points: virtual trapped person location E2; the system also sets nodes at major corners or room entrances according to preset rules, such as B2 (the first corner of the corridor), D2 (master bedroom entrance), and F1 (second bedroom entrance); the system also sets nodes at entrance A1 and exit A1 to record the start and end of the drill.

[0067] The system marks the following nodes on the path: Node 1: Position A1, Type: "Entrance Node", Expected Behavior: Start Search; Node 2: Position B2, Type: "Key Corner Node"; Node 3: Position D2, Type: "Room Entrance Node" (Master Bedroom); Node 4: Position E2, Type: "Rescue Node" (Position of Virtual Trapped Person); Node 5: Position F1, Type: "Room Entrance Node" (Second Bedroom); Node 6: Position F2, Type: "Search Node" (Center of Second Bedroom); Node 7: Position A1, Type: "Exit Node", Expected Behavior: End Search / Rescue; As shown above, these nodes are distributed at different locations along the drill path: entrance, corridor corner, room entrance, room center (rescue point), and exit. They cover the key positions in Xiao Wang's entire process from entering the fire scene to completing the search and rescue and then leaving.

[0068] Therefore, based on the node position, time and fire drill screen of multiple fire drill nodes, multiple firefighting behaviors of firefighters in the fire drill nodes are determined, and the corresponding firefighting behavior events are determined based on the synthesis of multiple firefighting behaviors, which is compatible with the overall consideration of the synthesis of multiple firefighting behaviors and ensures the accuracy of the corresponding firefighting behavior events.

[0069] At this time, the system will retrieve the fire drill screen data near the location of each determined fire drill node (including a short period of time before and after reaching the node). These screen data are associated with the specific location coordinates and precise timestamps of the node; the system uses computer vision, behavior recognition and other technologies to analyze the firefighters' posture, movements, and interactions with the virtual environment in the picture; for example, the system can identify whether the firefighters look up to observe, whether they bend over, whether they raise their hands (to operate tools or switches), their movement speed, whether they wear equipment (such as whether the respirator mask is worn properly), etc.; time and location information help the system accurately locate the observed actions within specific nodes and time periods.

[0070] After analyzing the image, the system will classify the identified fine-grained actions into specific firefighting behaviors, which are usually more basic and atomic action units; for example, in the "rescue node" (node ​​4, E2).

[0071] Xiao Wang walked to position E2, saw the virtual trapped person, stopped, lowered his head to observe the trapped person's condition, then squatted down, stretched out his hands, simulated grabbing the trapped person's shoulders, and began to drag him backwards.

[0072] Multiple firefighting behaviors were identified: reaching the designated location (E2) - stopping - looking down - squatting - extending both hands - grasping (simulation) - dragging backward (simulation). These behaviors are atomic actions completed by Xiao Wang at the specific location of node 4 and can be recognized by the system.

[0073] The multiple atomic firefighting behaviors identified in the previous step are combined into one or more more meaningful and macro-level "firefighting behavior events" based on the order of their occurrence, relevance, and the context of the drill. This synthesis process relies on a preset behavior pattern library, task rules, or pattern recognition through a machine learning model; for example, in node 4: synthesized firefighting behavior events: Event 1: Discovering and confirming the trapped person (synthesized by "reaching the designated location", "stopping moving", and "looking down"); Event 2: Implementing rescue dragging (synthesized by "squatting", "extending both hands", "grasping (simulation)", and "backward dragging (simulation)").

[0074] The system converts firefighters' detailed actions at key nodes into explainable and categorizable behavioral events. These behavioral events (whether positive or negative) are the key basis for judgment and scoring in subsequent steps (such as S13 capability level assessment and S14 anomaly detection). For example, "implementing a rescue drag" is a positive event that complies with regulations, while "rapidly passing a corner without observation" is a negative event that requires point deduction or marking as a risk.

[0075] Specifically, at node 4 (E2, rescue point), the system identified multiple atomic behaviors of Xiao Wang (arrival, stop, observation, squatting, reaching out, grasping, dragging), and synthesized them into two behavioral events: "discovering and confirming the trapped person" and "implementing rescue and dragging", which indicates that Xiao Wang executed the correct rescue process at this node; at node 2 (B2, critical corner), the system identified multiple atomic behaviors of Xiao Wang (arrival, no deceleration, fast passing, no observation), and synthesized them into the behavioral event "fast passing the corner without observation", which indicates that Xiao Wang had operational risks or standard issues at this node.

[0076] refer to Figure 4 In step S13, the specific steps are:

[0077] S131: collecting the node position of each fire drill node and matching it with the corresponding fire behavior event, and determining the first fire capability coefficient of each fire drill node according to the node position of each fire drill node and the corresponding fire behavior event;

[0078] S132: Determine a second firefighting capability coefficient based on the firefighting behavior event and the firefighter's previous firefighting drill records; determine a sub-capability level for each firefighting drill node based on a mapping relationship among the first firefighting capability coefficient, the second firefighting capability coefficient, and the sub-capability levels;

[0079] S133: Collect the physical data of the firefighters in each fire drill node, and determine the physical data set corresponding to each fire drill node based on the synthesis of multiple physical data, and evaluate the ability level of the firefighters according to the node position, sub-ability level and physical data set corresponding to each fire drill node.

[0080] In an embodiment of the present application, the node position of each fire drill node is collected and matched with the corresponding fire behavior events. The system needs to calculate a capability coefficient based on the behavior events at each node and the importance or danger of the node in the overall drill. This coefficient reflects the ability level of the firefighter at that specific node; the calculation method can be based on a preset rule base; the preset rule base is as follows: Node importance / danger assessment: A1 (entrance): high importance (starting point), medium danger; B2 (critical corner): high importance (key point on the path), high danger (blind spots in sight, structural risks); C1 (master bedroom entrance): medium importance, medium danger; E2 (rescue point): extremely high importance (core task), high danger (rescue operation); F1 (secondary bedroom entrance): medium importance, low danger.

[0081] Behavioral event assessment (positive / negative): Entering the fire environment and checking personal equipment: positive (basic operation); passing quickly without observation: negative (safety risk); stopping, observing the internal situation, and scanning with a thermal imager: positive (standard operation); discovering trapped people and implementing rescue and towing: positive (core task completed); stopping and observing the internal situation: positive (standard operation); no abnormalities found, continuing to move forward: neutral / slightly positive.

[0082] The system assigns a basic score to each behavioral event (e.g., positive +1, negative -2, neutral 0), and then multiplies it by a weight factor based on the importance / danger of the node (e.g., extremely important nodes have a weight of 1.5, high importance 1.2, medium 1.0, low 0.8); finally, the weighted scores of all behavioral events within the node are summed up and normalized (e.g., mapped to the range of 0-1) to obtain the first firefighting capability coefficient; the system generates a first firefighting capability coefficient for each fire drill node, which directly reflects the immediate performance and capability level of the firefighter at that specific location and when performing a specific task; for example, node 4 (rescue point) has the highest coefficient (1.0), indicating that Xiao Wang performed well in the core rescue task; while node 2 (critical corner) has the lowest coefficient (0.1), indicating that he has obvious deficiencies in safety awareness or operating specifications at that critical location.

[0083] Specifically, node 1 (A1): behavior 1 (+1) × importance weight 1.2 + behavior 2 (+1) × importance weight 1.2 = 2.4; assuming that after normalization, the first capability coefficient is 0.9; node 2 (B2): behavior 3 (-2) × importance weight 1.2 × risk weight 1.2 = -2.88; assuming that after normalization, the first capability coefficient is 0.1; node 3 (C1): behavior 4 (+1) × importance weight 1.0 + behavior 5 (+1) × importance weight 1.0 = 2.0; assuming that after normalization, the first capability coefficient is 0.8.

[0084] Node 4 (E2): Behavior 6 (+1) × importance weight 1.5 × criticality weight 1.2 + Behavior 7 (+1) × importance weight 1.5 × criticality weight 1.2 = 3.6; assuming that after normalization, the first capability coefficient is 1.0; Node 5 (F1): Behavior 8 (+1) × importance weight 1.0 + Behavior 9 (0) × importance weight 1.0 = 1.0; assuming that after normalization, the first capability coefficient is 0.7.

[0085] Furthermore, the second firefighting capability coefficient is determined based on firefighting behavior events and firefighters' previous firefighting drill records; the sub-capability level of each firefighting drill node is determined based on the mapping relationship between the first firefighting capability coefficient, the second firefighting capability coefficient and the sub-capability level, which is compatible with the overall consideration of the mapping relationship between the first firefighting capability coefficient, the second firefighting capability coefficient and the sub-capability level, and ensures the accuracy of the sub-capability level of each firefighting drill node.

[0086] At this time, firefighting behavioral events and firefighters' previous fire drill records are introduced to identify specific behavioral events exhibited by firefighters in the current drill (such as "passing quickly without observation", "using fire extinguishers correctly", "effective rescue dragging", etc.); query the firefighter's performance in similar behavioral events or tasks in previous drills or training records, which include: frequency: how many times the firefighter has performed similar behaviors; consistency: whether the execution effect is stable; whether it is good or bad; improvement trend: compared with the past, has it improved or regressed; specific weaknesses: whether there are recurring problem behaviors; based on a comprehensive analysis of current behavioral events and historical performance, the second firefighting capability coefficient is calculated; the second firefighting capability coefficient can reflect the firefighter's proficiency, stability, improvement potential or inherent weaknesses in this type of behavior; the calculation method involves: weighted average: combining the performance score of the current behavior with the historical average score according to a certain weight; trend analysis: if the historical performance shows an upward trend, the coefficient increases; otherwise it decreases.

[0087] Specifically, the current behavioral event: "passing quickly without observing" (score: -2, importance weight: 1.2, danger weight: 1.2, preliminary calculation of the first coefficient: 0.1); previous record retrieval: the system found that in the past 5 similar corner handling drills, Xiao Wang: 3 times were "passing quickly without observing" (similar to the current error); 1 time was "passing slowly and observing" (correct behavior); 1 time was "passing quickly but with a quick glance" (partially correct); the coach had conducted 2 individual coaching sessions on the "passing quickly without observing" behavior; the system analysis believed that Xiao Wang had recurring habitual mistakes in this type of behavior, and although there was coaching, the effect was not good; therefore, the system assigned a lower coefficient, such as 0.2, indicating that he had long-term stability problems in the skill of "safe corner handling". This coefficient is slightly higher than the current behavior's instant score (corresponding to the first coefficient 0.1), but it is still very low, reflecting that this is an area that needs to be improved.

[0088] Get the first firefighting capability coefficient (FC) calculated by S131 and the second firefighting capability coefficient (SC) calculated by S132; usually these two coefficients are combined according to the preset weights to obtain a comprehensive capability coefficient (CC); for example: CC=w1×FC+w2×SC, where w1 and w2 are weights, usually w1+w2=1; the weights can be adjusted according to the evaluation objectives, for example, if immediate performance is more important, w1 can be set higher; if long-term stability is more important, w2 can be set higher; assuming we simply average: CC=(FC+SC) / 2; match the calculated comprehensive capability coefficient (CC) with the pre-defined "sub-capability level mapping relationship", which is usually a threshold table or function. The numerical range is mapped to a specific grade name; for example: CC ≥ 0.9: Excellent; 0.7 ≤ CC < 0.9: Good; 0.5 ≤ CC < 0.7: Pass; CC < 0.5: Through this step, the system assigns a clear and easy-to-understand sub-ability grade to each fire drill node; the system not only evaluates the firefighter's immediate performance at each node, but also combines its historical performance to give a more comprehensive and stable sub-ability grade; for example, the grade of node 2 (B2) is "needs improvement", which reflects both the errors in the current drill and his past persistent weaknesses in this type of behavior. These sub-ability grades provide key inputs for the final overall capability grade assessment (S133).

[0089] Therefore, the physical data of firefighters in each fire drill node are collected, and the physical data set corresponding to each fire drill node is determined based on the synthesis of multiple physical data. The ability level of firefighters is evaluated according to the node position, sub-ability level and physical data set corresponding to each fire drill node of each fire drill node. This is compatible with the overall consideration of the synthesis of multiple physical data, and ensures the accuracy of the physical data set corresponding to each fire drill node. At the same time, fire behavior events are introduced, which is compatible with the node position, sub-ability level and physical data set corresponding to each fire drill node of each fire drill node, and improves the accuracy of the assessment of firefighters' ability level.

[0090] At this time, the physiological and motion data of firefighters during the drill are obtained from wearable devices (such as smart vests, bracelets, and helmet sensors) or on-site monitoring equipment. These data need to correspond precisely to the timestamp so that they can be accurately mapped to specific fire drill nodes. The data that need to be collected include: heart rate: reflecting cardiovascular load and fatigue level; respiratory rate: reflecting the load and tension of the respiratory system; body surface temperature: reflecting heat stress and physical exertion; gait: reflecting movement efficiency and fatigue effects; number of hand waves: reflecting tension or specific operations; and fulcrum: reflecting movement mode or weight effects.

[0091] The collected raw body data is usually multi-dimensional and multi-valued. The goal of this step is to integrate and process these raw data to form a comprehensive indicator or data set that can represent the physiological state of the node; the processing methods include: calculating statistics such as average, maximum, minimum, standard deviation, etc.; for example, calculating the average heart rate and maximum respiratory rate of node E2; comparing the physiological data with the preset threshold or health model to determine whether the physiological state is normal, fatigued, over-stressed or dangerous; calculating the comprehensive physiological load index, for example, combining indicators such as heart rate and respiratory rate to generate a physiological load coefficient between 0 and 1; combining the processed features, state assessment results or quantitative indexes into a structured data set, marked as the body data set of the node.

[0092] The system needs to comprehensively consider the following three aspects of information to ultimately determine the overall ability level of the firefighter: Node location: The difficulty and importance of tasks at different locations are different; for example, making a mistake at the entrance and making a mistake at the core rescue point have different impact weights; the system will consider the impact of the node location on the final evaluation based on preset weights or rules; Sub-ability level: This is the result of S132, which represents the firefighter's ability level of comprehensive technical, tactical and historical performance at each node (such as excellent, good, qualified, needs improvement).

[0093] Physical data set: This reflects the firefighter's physiological state and load when performing tasks; the system will evaluate: Physiological load rationality: Is the current physiological load reasonable when completing a specific task (corresponding node position) and reaching a certain ability level (sub-ability level)? For example, when performing a rescue at node 4 (E2) (sub-ability level is excellent), a heart rate of 130bpm (physical data set) is reasonable; but if a firefighter passes through node 2 (B2) quickly without observation (sub-ability level needs improvement), a heart rate of 110bpm (physical data set) appears too high, suggesting tension or excessive exertion; whether a higher sub-ability level can be maintained under higher physiological stress or fatigue; conversely, whether the sub-ability level is too low under lower physiological load; potential risks: whether the physiological state has reached the dangerous threshold, which will affect the final evaluation even if the sub-ability level is acceptable.

[0094] refer to Figure 5 In step S14, the specific steps are:

[0095] S141: Collecting the fire database of the AR glasses, and determining a preset ability level threshold based on a match between the fire database and the fire drill footage of the firefighter, and comparing the firefighter's ability level with the preset ability level threshold;

[0096] S142: If the firefighter's ability level is lower than the preset ability level threshold, an abnormality detection is performed on each sub-ability level, and the sub-ability levels in an abnormal state are screened out. The corresponding abnormal firefighting behavior is determined based on the tracing back of the sub-ability levels in the abnormal state;

[0097] In an embodiment of the present application, a fire protection database of AR glasses is collected, and this database is the basis for evaluation. The specific data included depends on the design of the system, but generally includes: standard capability level threshold: this is the core part; the database will store capability level standards set for different drill categories (such as "multi-story residential fire rescue", "warehouse cargo fire fighting", "high-rise building trapped people search and rescue"), different firefighter levels (such as "junior firefighter", "intermediate firefighter", "senior firefighter" or "squad leader", "combatant"), and even different drill stages (such as "basic skills", "tactical application", "comprehensive drill"). These standards define the conditions that must be met to reach a certain level (such as "qualified", "good", "excellent").

[0098] For example, for a "multi-story residential fire rescue" drill conducted by "junior firefighters," the standard is set as "the overall capability level must reach 'qualified'"; drill scenario information: The database will associate the specific scenario information of the current drill, including the simulated building structure, fire conditions, smoke concentration, potential danger points (such as the location of flammable materials, structural weaknesses), etc., which enables the system to call the standard that best matches the current scenario; mission objectives and key nodes: The database also contains the specific mission objectives of the current drill and preset key evaluation nodes (these nodes have been identified in S12, but the standard will define the performance expected at these nodes).

[0099] The system needs to filter out the most applicable preset capability level threshold from the database based on the specific circumstances of the current drill; the key to matching lies in the "fire drill screen" (or more broadly, the contextual information of the drill, such as the scene, task, personnel level, etc.); the system will analyze the metadata of the current drill screen, such as the drill category, simulated building type, firefighter level, etc.; then, the system will search the database for the standard threshold that fully matches or is closest to these metadata; once a match is successful, the system determines the specific preset capability level threshold based on which this assessment is based. This threshold is a single level (such as "qualified") or a composite standard that includes multiple sub-ability dimensions and their corresponding level requirements (such as "fire scene search: good; rescue skills: qualified; teamwork: qualified").

[0100] The system obtains the firefighter's final ability level assessed in S13 (for example, calculated by S133), and then directly compares it with the preset ability level threshold determined in the second step; the comparison can be an overall level comparison (such as the firefighter's final level "good" vs. the preset threshold "qualified"), or it can be a sub-item comparison (such as the firefighter's "fire scene search" sub-ability level "needs improvement" vs. the preset threshold "good"); there are only two results of the comparison: meet the standard or fail to meet the standard; if the firefighter's ability level is equal to or higher than the preset threshold, it is judged as "meeting the standard"; if the firefighter's ability level is lower than the preset threshold, it is judged as "failure to meet the standard".

[0101] Specifically, the AR glasses continuously captured footage of Xiao Wang practicing at the fire scene. The system not only saw the footage but also parsed its context: confirming that it was a "multi-story residential fire rescue" operation and that the firefighter was a "junior firefighter." The system matched the two key pieces of information, "multi-story residential fire rescue" and "junior firefighter," with the database. The database contained many standards, but only the standard corresponding to the combination of "junior firefighter-multi-story residential fire rescue" was selected. The system determined that the preset capability level threshold for this assessment was "the overall capability level must reach 'qualified'."

[0102] Assume that after the calculation of S133 of S13, the final overall ability level of firefighter Xiao Wang is assessed as "qualified"; the system now compares this assessment result "qualified" with the previously determined preset ability level threshold "qualified"; Xiao Wang's assessment level "qualified" is equal to the preset threshold "qualified"; the system determines that Xiao Wang has "met the standard" in this exercise.

[0103] Suppose another junior firefighter, Xiao Li, is ultimately assessed as having an overall competency level of "needs improvement" in the same "multi-story residential fire rescue" drill. The system compares "needs improvement" with the preset threshold of "qualified." Xiao Li's assessment level of "needs improvement" is lower than the preset threshold of "qualified." The system determines that Xiao Li's drill "does not meet the standards," and this result triggers the S142 process.

[0104] Furthermore, if the firefighter's ability level is lower than the preset ability level threshold, anomaly detection is performed on each sub-ability level, and the sub-ability levels in abnormal state are screened out. The corresponding abnormal firefighting behavior is determined based on the tracing of the sub-ability levels in abnormal state, which is compatible with the overall consideration of the tracing of the sub-ability levels in abnormal state, and ensures the accuracy of the corresponding abnormal firefighting behavior.

[0105] At this time, when the firefighter's ability level is lower than the preset ability level threshold, the system no longer looks at the overall level, but instead focuses on the various "sub-ability levels" that constitute the overall ability level (these sub-ability levels are determined in S132 and S133, such as fire scene search, fire extinguishing operations, rescue skills, safety awareness, etc.); the system will compare each sub-ability level with the preset threshold corresponding to the sub-ability (this threshold is usually obtained in the first step of S141 "collecting the fire protection database of AR glasses", or stored in the database in association with the overall threshold).

[0106] Usually, a simple threshold comparison is used; that is, for each sub-capability, check whether the sub-capability level is lower than its corresponding preset sub-capability threshold; if it is lower, the sub-capability level is judged to be "abnormal" or "poor performance"; the system will traverse all sub-capability levels and filter out those sub-capabilities judged to be "abnormal" to form an "abnormal sub-capability list"; accurately locate the problem; instead of saying "lack of ability" in general, it specifically points out "poor fire scene search ability" or "lack of proficiency in firefighting operations"; we have determined which sub-capability levels are below the threshold (i.e., abnormal) through comparison, and this step is to clearly separate these detected abnormal sub-capability levels from all sub-capability levels to form a clear list; at this time, the sub-capabilities that meet the conditions (sub-capability level < preset sub-capability threshold) are identified and added to a set or list.

[0107] Review each drill node and its corresponding behavioral events recorded in S12; check at which nodes the firefighters performed behaviors related to the abnormal sub-capability and whether these behaviors did not meet the specifications or standards; for example, if the "fire scene search" sub-capability is abnormal, review the firefighters' behavioral records at each search node (such as rooms and corridor corners).

[0108] Review the set of physical data recorded in S13; some abnormal physical data (such as too fast heart rate, slow movements) are related to specific abnormal behaviors or insufficient capabilities; for example, a persistently high heart rate at a node requiring quick decision-making reflects insufficient decision-making ability or psychological quality; there is a scoring rule library within the system that defines which behavioral events correspond to which sub-capability scores; through reverse search, the specific behavioral events that lead to low scores for the sub-capability can be found; through the above tracing, the system can clearly point out which specific behaviors lead to abnormal sub-capability levels; for example, instead of saying "poor fire scene search capability" in general, it specifically points out that "the 'six-step search method' was not carried out as required at node B2" and "the potential danger point was not marked at node C3."

[0109] Specifically, suppose firefighter Xiao Li is judged as "not meeting the standard" in S141 (the overall ability level "needs improvement" is lower than the preset threshold "qualified"); the system checks Xiao Li's various sub-ability levels: fire scene search: level "needs improvement" (preset threshold "good"); fire extinguishing operation: level "good" (preset threshold "good"); rescue skills: level "qualified" (preset threshold "good"); safety awareness: level "good" (preset threshold "good"); after comparison, it is found that the two sub-ability levels of "fire scene search" and "rescue skills" are lower than their corresponding preset thresholds; the system filters out the sub-ability levels that are in abnormal status: "fire scene search" and "rescue skills".

[0110] Regarding the "Fire Scene Search" anomaly: The system reviewed the data from S12 and found that Xiao Li's behavior record at the key node B2 (a corner room) was "quickly passing through without stopping to observe." According to the scoring rules, "quickly passing the key observation point" will result in a point deduction and seriously affect the score of the "Fire Scene Search" sub-ability. The system also reviewed the physical data from S13 and found that Xiao Li's heart rate increased slightly at point B2, indicating a nervous or impatient mentality. Determining the abnormal behavior: The system determined that the specific behavior that caused the abnormality in the "Fire Scene Search" sub-ability was "failure to stop, observe, and mark at the key corner point B2 as required."

[0111] Regarding the abnormality of "rescue skills": the system reviewed the data of S12 and found that Xiao Li's behavior record at node C5 (simulating the position of trapped people) was "trying to move the trapped person (simulated dummy) with bare hands, without using auxiliary tools or correct force posture"; according to the scoring rules, this does not comply with the regulations and will be deducted points, affecting the score of the "rescue skills" sub-ability; determine the abnormal behavior: the system determines that the specific behavior that caused the abnormality of the "rescue skills" sub-ability is "not using standard techniques and auxiliary tools when moving trapped people at node C5."

[0112] Assessment results: Not up to standard; Main issues: Fire scene search capabilities and rescue techniques need improvement; Specific abnormal behaviors: 1. At the critical corner B2, the personnel did not stop to observe and mark according to regulations; 2. When relocating trapped personnel at node C5, they did not use standard techniques and auxiliary tools; Recommendations: Strengthen training on fire scene search procedures, especially observation of key nodes; Review and practice correct rescue and relocation techniques.

[0113] refer to Figure 6 In step S15, the specific steps are:

[0114] S151: Real-time monitoring of each fire drill node, collecting abnormal firefighting behaviors, identifying multiple abnormal action areas based on the abnormal firefighting behaviors, and determining corresponding abnormal firefighting action features based on the detection of each abnormal action area. In this case, one abnormal action area matches one abnormal firefighting action feature, and the abnormal firefighting behavior has multiple abnormal firefighting action features.

[0115] S152: At each fire drill node, multiple body data of firefighters performing abnormal firefighting behaviors are collected, a corresponding body data set is determined based on a combination of the multiple body data, and multiple body features are determined based on the recognition of the body data set;

[0116] S153: Determine the abnormal firefighting factors according to the abnormal firefighting action characteristics, the corresponding body characteristics and the mapping relationship between the abnormal firefighting factors, and determine the corresponding firefighting optimization actions based on the abnormal firefighting factors, the firefighter's morphological information and the firefighting database of the AR glasses.

[0117] In an embodiment of the present application, each fire drill node is monitored in real time, and abnormal firefighting behavior is collected. The system already knows that this is an abnormal behavior, and now needs to understand how this behavior occurs and what specific action steps it includes; the system will decompose this continuous abnormal behavior process into several key time periods or spatial areas, each area represents a sub-stage of the behavior, and these areas are divided based on behavioral logic or key action points; for example, the behavior of "detection without using tools" can be decomposed into areas such as "reaching the door", "preparing for detection (should happen but did not happen)", "opening the door directly", and "entering the room"; the system will clearly mark the starting and ending points of these areas on the timeline or space.

[0118] The system focuses on each identified abnormal motion area and uses the sensor data of AR glasses (images, posture, motion trajectory, etc.) for more detailed analysis; the system will extract specific deviation points from the standard specifications in each area. These features are quantifiable or descriptive details; for example, whether the posture is correct, whether the speed is too fast, whether the key steps are missing, whether the line of sight is deviated, etc. Each abnormal motion area usually corresponds to one or more specific abnormal features, and one abnormal motion area matches one abnormal firefighting motion feature.

[0119] Specifically, suppose firefighter Xiao Wang is conducting a search and rescue drill simulating a building fire. He is currently located at node 4 on the drill path: "E2 - Second bedroom door." Standard procedures require that before entering any enclosed room, a thermal imager must be used to observe the interior and tools (such as a crowbar) must be used to test whether the door lock or door panel is overheated or obstructed. At this time, the system monitors Xiao Wang's behavior in real time through the camera and sensors of the AR glasses: he directly pushed open the door of room E2 without any observation or testing. The system immediately recognizes that this is an abnormal firefighting behavior of "failure to observe and detect according to regulations" and begins collecting: timestamp "12 minutes and 30 seconds into the drill", location "node E2", and behavior classification "safety procedure violation".

[0120] Regarding Xiao Wang's "failure to observe and detect according to regulations" at node E2, the system breaks it down into the following abnormal action areas: Area A: Arrival at the door (time 12:28-12:29): Xiao Wang walked from the corridor to the door of room E2. Although this is not an anomaly itself, it is the background area where the abnormal behavior occurred. Area B: Preparation for detection phase (should have occurred but did not, time 12:29-12:29.5): The standard requires that at this stage, Xiao Wang should stop, take out the thermal imager for observation, and take out tools to prepare to detect the door. The system detected that Xiao Wang did not perform these actions, and this "missing" phase is also considered an abnormal action area.

[0121] Area C: Open the door directly (time 12:29.5-12:30): Xiao Wang pushed the door open without hesitation; Area D: Enter the room (time 12:30-12:31): Xiao Wang stepped over the threshold and entered the room; the system now clearly understands these four areas, especially the "missing" area B, which is the key to understanding the entire abnormal behavior.

[0122] The system detects the four abnormal motion areas mentioned above and determines the core abnormal characteristics:

[0123] Area A characteristic: "moving slightly faster than the standard speed" (although not directly abnormal, it is the cause of subsequent impatience); Area B characteristic: "key detection steps are missing" (the most core characteristic, a direct violation of regulations); Area C characteristic: "door opening is done directly by hand, without the use of tools" (further confirmation of the violation); Area D characteristic: "no immediate internal inspection after entering the room" (although it is an action after entering, it is also part of the abnormal behavior chain, characterized by "lack of preliminary environmental assessment").

[0124] We can select the most core feature of each area, or merge related areas; for example, if areas B and C are merged, the core feature is "failure to perform standard detection procedures (lack of observation and tool detection)"; the feature of area D is "lack of preliminary environmental assessment after entry"; the feature of area A, "slightly faster travel speed", is considered a potential cause of the anomalies in areas B and C, but is not the core action feature of this specific abnormal behavior.

[0125] Furthermore, at each fire drill node, multiple body data of firefighters during abnormal firefighting behaviors are collected, and the corresponding body data set is determined based on the combination of multiple body data. Based on the identification of the body data set, multiple body features are determined, which is compatible with the overall consideration of the identification of the body data set and ensures the accuracy of multiple body features.

[0126] At this time, in each fire drill node, multiple physical data of firefighters during abnormal firefighting behaviors are collected. Based on the combination of multiple physical data: the system will not view a certain data in isolation (such as heart rate), but will associate and combine all relevant physical data collected within the same time window (heart rate, respiration, posture, skin conductivity, etc.); determine the corresponding physical data set: the combined data will be classified according to the different stages or aspects of the abnormal behavior to form one or more structured data sets, each set representing the multi-dimensional physiological and motion state of the firefighter in a specific situation (when performing a certain abnormal behavior).

[0127] The system uses preset rules, thresholds or machine learning models to analyze these structured sets of physical data. It looks for indicators in the data that reflect specific physiological or psychological states; it refines the analysis results into a series of interpretable physical feature labels or descriptions. These features should be able to summarize the state of firefighters when performing abnormal behaviors, such as "fast heart rate", "rapid breathing", "unstable posture", "insufficient movement range", "muscle tension", etc.

[0128] Specifically, the system not only recorded the original physiological and motion data of Xiao Wang when he performed abnormal behavior at the E2 node, but also refined it into specific physical characteristics (such as a significantly increased heart rate, unstable posture, etc.). These physical characteristics are combined with the abnormal action characteristics identified in S151 (such as lack of observation and tool detection, and direct use of hands to open the door), providing S153 with a strong physiological basis for inferring the root cause of the abnormal behavior (whether it is lack of skills, excessive tension or fatigue); for example, a significantly increased heart rate and accelerated breathing strongly suggest that Xiao Wang ignored standard procedures due to nervousness.

[0129] Therefore, the abnormal firefighting factors are determined according to the mapping relationship between the abnormal firefighting action characteristics, the corresponding physical characteristics and the abnormal firefighting factors, and the corresponding firefighting optimization actions are determined based on the abnormal firefighting factors, the firefighter's morphological information and the firefighting database of the AR glasses. The overall consideration of the abnormal firefighting factors, the firefighter's morphological information and the firefighting database of the AR glasses is compatible to ensure the accuracy of the corresponding firefighting optimization actions. At the same time, abnormal firefighting behaviors are introduced to realize the overall consideration of multiple abnormal firefighting action characteristics and the corresponding physical data sets, thereby improving the recognition accuracy of abnormal firefighting factors and matching the corresponding firefighting optimization actions.

[0130] At this point, the mapping relationship between abnormal firefighting action features, corresponding physical features, and abnormal firefighting factors is introduced. Specific anomalies identified by S151, such as "missing key detection steps," "opening the door directly with hands," and "not looking around after entering," are used; firefighter status information extracted by S152, such as "significantly increased heart rate," "accelerated breathing rate," "physiological stress response," and "unstable posture," are used. The mapping relationship between abnormal firefighting factors is a pre-established knowledge base or rule base that defines different combinations of abnormal action features and physical features, and which potential root causes (abnormal firefighting factors) are usually associated with them. These factors include: Insufficient skills / knowledge: Lack of understanding of standard operating procedures (SOPs) or unfamiliarity with specific equipment / environment; excessive tension / anxiety: fear of high-risk situations leads to narrow attention and distorted movements; fatigue / poor physiological state: long hours of work or physical discomfort lead to slow reaction and decreased judgment; distraction: interference from environmental noise, calls from teammates or other interference; equipment problems: such as unskilled use of tools, or interference from information displayed by AR glasses; poor communication: failure to obtain or understand key information from teammates or command; the system inputs the results of S151 and S152 into the mapping relationship for matching and reasoning to derive the root cause, which is usually a judgment with a high probability or confidence level.

[0131] Specifically, at the E2 node, the abnormal action characteristics are: lack of observation and tool detection, direct use of hands to open the door, and no looking around after entering; physical characteristics: significantly increased heart rate, accelerated breathing rate, physiological stress response, and unstable posture; the system inputs this information into the "abnormal firefighting factor mapping relationship"; the mapping relationship contains the following rules: Rule 1: If the "key detection step is missing" and the "physiological stress response" is strong, the high-level factor is "excessive tension / anxiety"; Rule 2: If "the door is opened directly with hands" (non-standard procedure) and "the posture is unstable", the factor is "tension causing deformation of movement" or "fatigue"; Rule 3: If "no looking around after entering" and "the heart rate is significantly increased", the factor is "excessive tension causing narrow attention"; combined with all of Xiao Wang's characteristics, the system's most certain abnormal firefighting factor is: "excessive tension / anxiety leading to narrow attention, ignoring standard safety procedures, and deformed movements"; the system will give this factor a high confidence score (for example, 85%).

[0132] Abnormal firefighting factors, firefighters' morphological information, and AR glasses' firefighting database are introduced. Based on abnormal firefighting factors: this is the core starting point for optimizing actions; for example, if the factor is determined to be "excessive tension", then the optimization action should focus on helping firefighters relieve tension and restore calmness; if the factor is "lack of skills", it should focus on guiding the correct operating steps; firefighters' morphological information: this includes firefighters' personal profile information, such as experience level (novice, experienced, expert), training records, past performance, and even specific physical conditions (such as left-handedness / right-handedness, although AR glasses themselves do not directly record it, the system can infer it). The information helps to personalize recommendations; for example, the optimization suggestions for novices and experts have different focuses; the fire protection database of AR glasses is a database containing a large amount of fire protection knowledge, standard operating procedures (SOPs), best practices, common errors and correction methods, specific scenario response strategies, etc.; the system combines the above three aspects of information to retrieve or generate specific and executable guidance suggestions from the database. These suggestions need to directly target abnormal fire protection factors and take into account the individual circumstances of firefighters; optimization actions should be clear and specific, and it is best to combine the display capabilities of AR glasses (such as superimposing virtual markers, arrows, text prompts, playing short guidance videos, etc.).

[0133] Specifically, abnormal firefighting factors include: excessive tension / anxiety leading to narrow attention, ignoring standard safety procedures, and distorted movements; firefighter physiognomy information: assuming the system knows that Xiao Wang is an inexperienced firefighter (a novice); the firefighting database of AR glasses: containing standard search and rescue procedures, tension-relieving techniques, and action guidelines for different scenarios; the system now needs to determine the corresponding firefighting optimization actions, which it will do as follows: For "excessive tension": extract tension-relieving methods from the database and combine them with the characteristics of novices; for example, recommending "breathing adjustment methods" (such as the 4-7-8 breathing method) and "short pause methods" (forcing yourself to pause for 3-5 seconds at key nodes for evaluation); for "ignoring standard procedures": extract the standard operating procedures (SOPs) for the E2 node (near the rescue point) from the database, especially the door frame detection and entry procedures; for example, "When arriving at the door, hold a tool (such as a crowbar or demolition tool) in both hands and slowly feel the upper and sides of the door frame. After confirming that there is no heat or danger, use the tip of the tool to gently push the door gap and observe the internal situation. After confirming that it is safe, use the tool to prop the door open while entering sideways. Immediately after entering, conduct a 360-degree look around."

[0134] The system can overlay these optimization actions onto Xiao Wang's field of view in the form of visual cues: as he approaches the door, the AR glasses display a virtual timer, prompting him to take a deep breath for 5 seconds; at the door, virtual step markers are displayed (1. Touch the upper frame 2. Touch the left side 3. Touch the right side 4. Gently push the tool 5. Enter sideways); after entering the room, a slowly rotating arrow is displayed, prompting him to look around; a short animation or voice prompt about the importance of door frame detection is played in a small window on the side of the AR glasses; the final firefighting optimization actions are as follows:

[0135] "Xiao Wang, we detect that your heart rate is high and you are a little nervous. Please take a deep breath for 5 seconds and try to relax. When you reach the door, please follow the steps: 1. Use a tool to touch the upper part of the door frame. 2. Touch the left side. 3. Touch the right side. 4. Use a tool to gently push the door gap and observe. 5. Enter sideways after confirming it is safe. After entering the door, please slowly turn your head and take a 360-degree look to ensure the safety of the environment. (A small window pops up on the side) "Tip: Door frame detection is the most important safety step before entering an unknown area. It can detect high temperature and collapse risks in advance."

[0136] The system not only diagnosed the root cause behind Xiao Wang's abnormal behavior (excessive tension), but also generated very specific and personalized optimization action suggestions based on his experience level and standard knowledge base. These suggestions directly target the root cause of the problem and use the capabilities of AR glasses for intuitive display. They aim to help Xiao Wang correct errors immediately during drills, learn correct and safer operating methods, and thus improve his actual combat capabilities. The entire process embodies a closed-loop intelligent assistance from discovering problems to analyzing causes to providing solutions.

[0137] See also Figure 7 , Figure 7 : is a schematic diagram of the structure of a firefighter capability level assessment system based on a fire scene environment in an embodiment of the present invention; the firefighter capability level assessment system based on a fire scene environment includes:

[0138] The fire drill screen module 21 is used to create a fire scene environment in which firefighters are located based on AR glasses and collect fire drill screens of firefighters in the fire scene environment;

[0139] A fire behavior event module 22 is used to determine the fire drill path of the firefighter based on the detection of the fire drill screen, and mark the corresponding fire behavior event of the firefighter at each fire drill node in the fire drill path;

[0140] A capability level module 23 is configured to determine a sub-ability level of each fire drill node based on the node position of each fire drill node, fire behavior events, and the firefighter's previous fire drill records, and to evaluate the firefighter's capability level based on the node position of each fire drill node, the sub-ability level, and the physical data set corresponding to each fire drill node;

[0141] The abnormal firefighting behavior module 24 is configured to determine the sub-ability level in an abnormal state based on the detection of each sub-ability level if the firefighter's ability level is lower than a preset ability level threshold, so as to determine the corresponding abnormal firefighting behavior;

[0142] The abnormal fire factor module 25 is used to determine multiple abnormal fire action features based on the identification of abnormal fire behavior in each fire drill node, determine abnormal fire factors based on the multiple abnormal fire action features and the corresponding body data set, and match the corresponding fire optimization actions.

[0143] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A firefighter competence level assessment method based on a fire scene environment, characterized in that: include: Using AR glasses, the fire scene environment of the firefighters is created, and the fire drill footage of the firefighters in the fire scene environment is collected; Determine the fire drill path of the firefighter based on the detection of the fire drill screen, and mark the corresponding firefighting behavior event of the firefighter at each fire drill node in the fire drill path; Determine the sub-ability level of each fire drill node based on the node position of each fire drill node, fire behavior events, and previous fire drill records of firefighters, and evaluate the firefighter's ability level based on the node position of each fire drill node, the sub-ability level, and the physical data set corresponding to each fire drill node; If the firefighter's ability level is lower than the preset ability level threshold, the sub-ability level in an abnormal state is determined based on the detection of each sub-ability level to determine the corresponding abnormal firefighting behavior; In each fire drill node, multiple abnormal fire action features are determined based on the identification of abnormal fire behavior, abnormal fire factors are determined based on the multiple abnormal fire action features and the corresponding body data sets, and corresponding fire optimization actions are matched.

2. The firefighter competence level assessment method based on the fire scene environment according to claim 1 is characterized in that: The method of creating a fire scene environment in which firefighters are located based on AR glasses and collecting fire drill images of firefighters in the fire scene environment includes: When a firefighter wears AR glasses, the fire drill type is determined based on the fire drill category selected by the firefighter and the corresponding fire drill progress. The fire scene environment in which the firefighter is located is created based on the fire drill type and the surrounding environment detected by the AR glasses. The firefighter enters the fire scene environment as a virtual character, and the virtual character synchronizes the firefighter's on-site actions. According to the firefighter's fire drill in the fire scene environment, the firefighter's fire drill footage in the fire scene environment is collected.

3. The firefighter competence level assessment method based on the fire scene environment according to claim 1, characterized in that: The method of determining the fire drill path of the firefighter based on the detection of the fire drill screen and marking the corresponding firefighting behavior event of the firefighter at each fire drill node in the fire drill path includes: Collecting a fire drill screen, determining multiple drill detection areas based on the division of the fire drill screen, determining corresponding sub-fire drill paths based on the simultaneous identification of the multiple drill detection areas, collecting multiple sub-fire drill paths, and determining a firefighter's fire drill path based on the synthesis of the multiple sub-fire drill paths; Determining a plurality of fire drill nodes based on detection of a fire drill path, wherein the plurality of fire drill nodes are distributed at different positions of the fire drill path; A plurality of firefighting behaviors of firefighters in a firefighting node is determined based on node positions, time, and firefighting pictures of the plurality of firefighting nodes, and a corresponding firefighting behavior event is determined according to a synthesis of the plurality of firefighting behaviors.

4. The firefighter competence level assessment method based on the fire scene environment according to claim 1, characterized in that: The determining of the sub-ability level of each fire drill node based on the node position of each fire drill node, fire behavior events, and previous fire drill records of the firefighters, and the evaluating of the firefighter's ability level based on the node position of each fire drill node, the sub-ability level, and the physical data set corresponding to each fire drill node, include: Collecting the node position of each fire drill node and matching it with the corresponding fire behavior event, and determining the first firefighting capacity coefficient of each fire drill node according to the node position of each fire drill node and the corresponding firefighting behavior event; The second firefighting capability coefficient is determined based on firefighting behavior events and previous firefighting drill records of firefighters; the sub-capacity level of each firefighting drill node is determined based on the mapping relationship among the first firefighting capability coefficient, the second firefighting capability coefficient and the sub-capacity level.

5. The firefighter competence level assessment method based on the fire scene environment according to claim 4 is characterized in that: The determining of the sub-ability level of each fire drill node based on the node position of each fire drill node, fire behavior events, and previous fire drill records of the firefighters, and evaluating the firefighter's ability level based on the node position of each fire drill node, the sub-ability level, and the physical data set corresponding to each fire drill node, further includes: The physical data of firefighters in each fire drill node are collected, and the physical data set corresponding to each fire drill node is determined based on the synthesis of multiple physical data. The ability level of firefighters is evaluated according to the node position, sub-ability level and physical data set corresponding to each fire drill node.

6. The firefighter competence level assessment method based on the fire scene environment according to claim 1, characterized in that: If the firefighter's ability level is lower than the preset ability level threshold, the sub-ability level in an abnormal state is determined based on the detection of each sub-ability level to determine the corresponding abnormal firefighting behavior, including: The fire protection database of the AR glasses is collected, and a preset capability level threshold is determined based on the matching of the fire protection database and the fire drill picture of the firefighter, and the firefighter's capability level is compared with the preset capability level threshold.

7. The firefighter competence level assessment method based on the fire scene environment according to claim 6, characterized in that: If the firefighter's ability level is lower than the preset ability level threshold, the sub-ability level in an abnormal state is determined based on the detection of each sub-ability level to determine the corresponding abnormal firefighting behavior, and further includes: If the firefighter's ability level is lower than the preset ability level threshold, anomaly detection will be performed on each sub-ability level, and the sub-ability levels in abnormal state will be screened out. The corresponding abnormal firefighting behavior will be determined based on the tracing of the sub-ability levels in abnormal state.

8. The firefighter competence level assessment method based on the fire scene environment according to claim 1, characterized in that: The method of determining multiple abnormal firefighting action features based on the identification of abnormal firefighting behaviors at each firefighting drill node, determining abnormal firefighting factors based on the multiple abnormal firefighting action features and corresponding body data sets, and matching corresponding firefighting optimization actions includes: Monitor each fire drill node in real time, collect abnormal firefighting behaviors, determine multiple abnormal action areas based on the identification of abnormal firefighting behaviors, and determine the corresponding abnormal firefighting action features based on the detection of each abnormal action area. At this time, one abnormal action area matches one abnormal firefighting action feature, and there are multiple abnormal firefighting action features in the abnormal firefighting behavior.

9. The firefighter competence level assessment method based on the fire scene environment according to claim 8, characterized in that: The method further includes: determining a plurality of abnormal firefighting action features based on the identification of abnormal firefighting behaviors at each firefighting drill node; determining abnormal firefighting factors based on the plurality of abnormal firefighting action features and corresponding body data sets; and matching corresponding firefighting optimization actions. At each fire drill node, multiple body data of firefighters during abnormal firefighting behaviors are collected, a corresponding body data set is determined based on the combination of the multiple body data, and multiple body features are determined based on the recognition of the body data set; Abnormal firefighting factors are determined based on the mapping relationship between abnormal firefighting action characteristics, corresponding physical characteristics and abnormal firefighting factors, and corresponding firefighting optimization actions are determined based on abnormal firefighting factors, firefighters' morphological information and the firefighting database of AR glasses.

10. A firefighter capability level assessment system based on a fire scene environment, characterized in that: The firefighter capability level assessment system based on a fire scene environment is applied to the firefighter capability level assessment method based on a fire scene environment as claimed in any one of claims 1 to 9, and the firefighter capability level assessment system based on a fire scene environment includes: The fire drill screen module is used to create a fire scene environment in which firefighters are located based on AR glasses and collect fire drill images of firefighters in this fire scene environment; A fire behavior event module is used to determine the fire drill path of the firefighter based on the detection of the fire drill screen, and mark the corresponding fire behavior event of the firefighter at each fire drill node in the fire drill path; A capability level module, configured to determine the sub-ability level of each fire drill node based on the node position of each fire drill node, fire behavior events, and the firefighter's previous fire drill records, and to evaluate the firefighter's capability level based on the node position of each fire drill node, the sub-ability level, and the physical data set corresponding to each fire drill node; An abnormal firefighting behavior module is used to determine the sub-ability level in an abnormal state based on the detection of each sub-ability level if the firefighter's ability level is lower than a preset ability level threshold, so as to determine the corresponding abnormal firefighting behavior; The abnormal fire factor module is used to determine multiple abnormal fire action features based on the identification of abnormal fire behavior in each fire drill node, determine abnormal fire factors based on multiple abnormal fire action features and corresponding body data sets, and match corresponding fire optimization actions.

Images