Emergency drill evaluation and analysis system and method

Abstract

The invention provides an emergency drill evaluation and analysis system and method, and relates to the field of emergency drill management, and the method comprises the steps: carrying out the key frame sampling of an emergency drill action video of a target person object through employing a video data analysis technology based on deep learning, carrying out the action semantic analysis of each sampled emergency drill action key frame, and carrying out the analysis of the action semantic. And performing context semantic association representation of the feature jump degree according to each action semantic analysis feature so as to intelligently generate a drill evaluation analysis report. Therefore, the action of each target person can be carefully identified and tracked, and the dependence on manual observation and subjective judgment is reduced, so that the problem of inconsistent evaluation results caused by personal differences is reduced. And meanwhile, the logic relation among the actions and the effect of the actions in the whole drilling process can be deeply understood, so that a more accurate drilling evaluation analysis report is generated.

Description

Technical Field

[0001] The present application relates to the field of emergency drill management, and more specifically, to an emergency drill evaluation and analysis system and method. Background Art

[0002] In today's society, various emergencies occur frequently. Emergency drills are simulated training activities conducted by organizations (such as enterprises, departments, medical institutions, etc.) to respond to emergencies (such as natural disasters, terrorist attacks, fires, etc.). By conducting emergency drills regularly, the effectiveness of emergency plans can be tested, the emergency response capabilities and coordination levels of personnel can be improved, and actions can be taken quickly and effectively when real events occur to minimize losses and injuries.

[0003] However, traditional emergency drill evaluation methods have many limitations. In the past, they relied on manual observation and subjective judgment, which was not only time-consuming and laborious, but also due to differences in the professional level and personal cognition of the evaluators, the evaluation results were often highly subjective and inaccurate. For example, in complex emergency drill scenarios, it is difficult for humans to fully and accurately capture every detail of the action and the relationship between the various links, resulting in deviations in the evaluation of the drill effect.

[0004] Therefore, an optimized emergency drill evaluation and analysis scheme is needed. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide an emergency drill evaluation and analysis system and method.

[0006] According to one aspect of the present application, an emergency drill evaluation and analysis system is provided, which includes: an emergency drill action video acquisition module, which is used to acquire an emergency drill action video of a target person object captured by a camera; a key frame sampling semantic analysis module, which is used to perform key frame sampling and action semantic analysis on the emergency drill action video to obtain a time sequence of emergency drill action semantic analysis features; a drill action timing context encoding module, which is used to perform drill action context encoding on the time sequence of the emergency drill action semantic analysis features to obtain drill action timing context semantic association features, wherein the drill action timing context encoding module includes: an emergency drill action semantic jump degree space span calculation unit, which is used to calculate the emergency drill action semantic jump degree space span ... The feature jump degree and message transmission space span of each emergency drill action semantic analysis feature in the time series of action semantic analysis features are used to obtain the time series of emergency drill action semantic feature jump degrees and the time series of emergency drill action semantic feature message transmission space spans; a drill action timing weighted aggregation unit is used to perform significant weighted aggregation on the time series of emergency drill action semantic analysis features based on the time series of emergency drill action semantic feature jump degrees and the time series of emergency drill action semantic feature message transmission space spans to obtain the drill action timing context semantic association features; a drill evaluation analysis report generation module is used to obtain a drill evaluation analysis report based on the drill action timing context semantic association features.

[0007] According to another aspect of the present application, an emergency drill evaluation and analysis method is provided, which includes: obtaining an emergency drill action video of a target person object captured by a camera; performing key frame sampling and action semantic analysis on the emergency drill action video to obtain a time series of emergency drill action semantic analysis features; performing drill action context encoding on the time series of emergency drill action semantic analysis features to obtain drill action temporal context semantic association features, including: calculating the feature jump degree and message transmission space span of each emergency drill action semantic analysis feature in the time series of emergency drill action semantic analysis features to obtain a time series of emergency drill action semantic feature jump degrees and a time series of emergency drill action semantic feature message transmission space spans; based on the time series of emergency drill action semantic feature jump degrees and the time series of emergency drill action semantic feature message transmission space spans, performing significant weighted aggregation on the time series of emergency drill action semantic analysis features to obtain the drill action temporal context semantic association features; based on the drill action temporal context semantic association features, obtaining a drill evaluation and analysis report.

[0008] Compared with the prior art, the emergency drill evaluation and analysis system and method provided by the present application uses deep learning-based video data analysis technology to perform key frame sampling on the emergency drill action video of the target personnel object, and then performs action semantic analysis on each sampled emergency drill action key frame, thereby intelligently generating a drill evaluation and analysis report based on the contextual semantic association representation of the feature jump degree of each action semantic analysis feature. In this way, the actions of each target person can be carefully identified and tracked, reducing the reliance on manual observation and subjective judgment, thereby reducing the problem of inconsistent evaluation results caused by individual differences. At the same time, it is possible to deeply understand the logical connection between each action and its role in the entire drill process, thereby generating a more accurate drill evaluation and analysis report. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] In the attached picture: Figure 1 4 is a system block diagram of an emergency drill evaluation and analysis system according to an embodiment of the present application.

[0011] Figure 2 It is a block diagram of a key frame sampling semantic parsing module in an emergency drill evaluation and analysis system according to an embodiment of the present application.

[0012] Figure 3 It is a block diagram of a drill action timing context encoding module in an emergency drill evaluation and analysis system according to an embodiment of the present application.

[0013] Figure 4 Flow chart of an emergency drill evaluation and analysis method according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0015] In today's society, various emergencies are occurring frequently. Emergency drills, as simulation training activities conducted by enterprises, departments, medical institutions and other organizations for natural disasters, terrorist attacks, fires and other emergencies, are of great importance. By organizing emergency drills regularly, the effectiveness of emergency plans can be tested and the emergency response capabilities and coordination and cooperation levels of personnel can be effectively improved. In this way, when a real emergency occurs, relevant personnel can take action quickly and efficiently to minimize losses and damage.

[0016] However, traditional emergency drill evaluation methods have many limitations. Past evaluation methods mainly rely on manual observation and subjective judgment. This model not only consumes a lot of time and manpower, but also, due to differences in professional quality and personal cognition among evaluators, the evaluation results are often highly subjective and inaccurate. For example, in complex and ever-changing emergency drill scenarios, it is extremely difficult for manual means to fully and accurately capture every subtle movement and the internal connection between each link, which ultimately leads to deviations in the evaluation of the drill effect.

[0017] In response to the above technical problems, the technical concept of the present application is to obtain an emergency drill action video of the target person object captured by a camera, use deep learning-based video sampling and processing technology to perform key frame sampling on the emergency drill action video, and then perform action semantic analysis on each sampled emergency drill action key frame, so as to intelligently generate a drill evaluation and analysis report based on the contextual semantic association representation of the feature jump degree of each action semantic analysis feature. The present application can meticulously identify and track the actions of each target person, reducing reliance on manual observation and subjective judgment, thereby reducing the problem of inconsistent evaluation results caused by individual differences. At the same time, it can deeply understand the logical connection between each action and its role in the entire drill process, thereby generating a more accurate drill evaluation and analysis report.

[0018] Figure 1 FIG. 1 is a system block diagram of an emergency drill evaluation and analysis system according to an embodiment of the present application. Figure 1 As shown, in the emergency drill evaluation and analysis system 100, it includes: an emergency drill action video acquisition module 110, which is used to obtain the emergency drill action video of the target personnel object captured by the camera; a key frame sampling semantic analysis module 120, which is used to perform key frame sampling and action semantic analysis on the emergency drill action video to obtain a time series of emergency drill action semantic analysis features; a drill action timing context encoding module 130, which is used to perform drill action context encoding on the time series of the emergency drill action semantic analysis features to obtain drill action timing context semantic association features; and a drill evaluation and analysis report generation module 140, which is used to obtain a drill evaluation and analysis report based on the drill action timing context semantic association features.

[0019] In an embodiment of the present application, the emergency drill action video acquisition module 110 is used to acquire an emergency drill action video of a target person object captured by a camera. It should be understood that the emergency drill action video records the action performance information of the target person object, the operation information of the equipment and tools used, and the environmental information, etc. For example, during a fire emergency drill, the walking route information, body posture information, and action information on how to operate fire extinguishers, fire hydrants and other fire-fighting equipment of the target person object at the fire drill site can be acquired. By analyzing the collected emergency drill action video, the performance of the target person object in the drill can be understood, and potential problems can be discovered based on this performance, and directions for improvement can be proposed, which will help organizations better optimize emergency drill management.

[0020] The following is a detailed description of the specific implementation process of "obtaining emergency drill action videos of target personnel objects captured by cameras": First, the selection and layout of cameras are key steps. For different scales and types of drill scenes, it is necessary to select suitable cameras. In small indoor drill environments, such as school classroom fire emergency drills, high-definition network cameras are the first choice because of their excellent ability to capture details in limited spaces. It can clearly present the subtle movements of students moving between tables and chairs, operating small fire extinguishers, and evacuating in an orderly manner, ensuring that every key behavior in the drill is accurately recorded. In large outdoor drill scenes, such as urban comprehensive emergency rescue drills covering disaster simulations such as earthquakes and floods, professional surveillance cameras with telephoto lenses and wide dynamic range are indispensable. Such cameras can effectively cover a wide area, from the large-scale assembly and dispatch of rescue teams to the actions of different rescue teams in complex terrain, such as searching for survivors in the ruins after a simulated earthquake, building temporary bridges and dams in flood-stricken areas, etc., all of which can provide clear imaging, and the picture quality can be guaranteed even in outdoor environments with complex and changing light. In terms of camera position layout, the principle of comprehensive no-dead-angle should be followed. In the drill scenes inside buildings, such as fire drills in office buildings, cameras should be installed at key nodes such as the end of corridors, stairwells, elevator entrances, and fire equipment rooms to fully record the movement trajectories and equipment operation conditions of personnel in different functional areas. For outdoor drill sites, the terrain should be comprehensively considered, and cameras should be installed at high places such as the tops of surrounding high-rise buildings, mountain high points, or with the help of specially built towers. At the same time, a multi-camera and multi-angle layout should be used to conduct all-round and three-dimensional monitoring of the drill area to ensure that the activities of the target personnel and objects during the entire drill process can be fully captured by the camera.

[0021] Secondly, the reasonable setting of shooting parameters plays a decisive role in video quality. The choice of resolution and frame rate needs to seek a balance between video clarity and storage cost. For conventional emergency drills, such as general natural disaster response drills organized by communities, 1080P resolution and 25fps frame rate can usually meet basic analysis needs. Such settings can not only ensure the recognizability of personnel actions, but also effectively control the amount of data storage, avoiding premature saturation of storage devices due to excessive resolution and frame rate. However, in some professional drill scenarios with extremely high requirements for action details, such as emergency disposal drills for hazardous chemical leaks in chemical companies, 4K or even higher resolution and frame rates above 50fps are necessary. In chemical drills, the operator's delicate operation of complex valves and pipeline connections provides rich and accurate image information for subsequent precise action semantic analysis. Lighting and contrast adjustment should also not be ignored. In low-light or unevenly lit drill environments, such as emergency lighting equipment failure simulation drills at night or outdoor drills with alternating strong direct sunlight and shadows, the camera needs to be set up in advance. Turn on the infrared function of the camera or add a fill light to ensure that the movements of people are clearly visible in dark environments; at the same time, through manual fine-tuning or using the camera's automatic optimization function, reasonably adjust the contrast to make clear the boundaries between people and background, and different action subjects, avoid blurring or loss of action information due to light problems, and ensure that the video images can truly reflect the actual situation at the drill site.

[0022] Finally, data storage and transmission guarantee is the key support for the entire video acquisition process. In terms of storage, reliable storage devices and methods should be selected. Local hard disk arrays are common basic storage solutions, and their large capacity and high-speed reading and writing capabilities can meet the storage needs of general exercise projects. For large-scale and long-term emergency drills, such as large-scale cross-regional earthquake rescue joint drills, network storage devices (NAS) or cloud storage services are more advantageous. NAS can provide efficient network shared storage functions, which facilitates assessors from multiple departments and multiple locations to access and analyze data at the same time; cloud storage services not only have powerful data redundancy backup capabilities to ensure data security and reliability, but also can achieve remote and convenient access, effectively improving work efficiency. In the data transmission link, it is crucial to build a stable and high-speed transmission network. Inside the exercise site, wired networks are preferred to connect cameras and storage or processing equipment to ensure the stability and high speed of data transmission and avoid data loss due to signal interference or transmission interruption. For outdoor exercise scenarios where wiring is inconvenient, such as geological disaster rescue drills in remote mountainous areas, advanced wireless network technologies such as 5G have become the first choice. The high speed and low latency characteristics of 5G networks ensure that video data can be transmitted to the back-end system quickly and stably. At the same time, it is essential to establish a complete data transmission monitoring mechanism. By monitoring the transmission status in real time, once a transmission failure or abnormality occurs, an alarm can be triggered immediately to repair the problem in a timely manner, thereby ensuring the integrity and continuity of the video data, and providing a solid data foundation for subsequent emergency drill evaluation and analysis, thereby effectively improving the reliability and practicality of the entire emergency drill evaluation system.

[0023] In the embodiment of the present application, the key frame sampling semantic analysis module 120 is used to perform key frame sampling and action semantic analysis on the emergency drill action video to obtain a time series of emergency drill action semantic analysis features. Specifically, Figure 2 FIG. 1 is a block diagram of a key frame sampling semantic parsing module in an emergency drill evaluation and analysis system according to an embodiment of the present application. Figure 2 As shown, the key frame sampling semantic analysis module 120 includes: an emergency drill action video sampling unit 121, which is used to perform key frame sampling on the emergency drill action video to obtain a time sequence of emergency drill action key frames; an emergency drill action semantic analysis unit 122, which is used to input each emergency drill action key frame in the time sequence of the emergency drill action key frames into the action semantic analysis module to obtain a time sequence of emergency drill action semantic analysis coding vectors as a time sequence of the emergency drill action semantic analysis features.

[0024] In the embodiment of the present application, the emergency drill action video sampling unit 121 is used to perform key frame sampling on the emergency drill action video to obtain a time sequence of key frames of the emergency drill action. Specifically, in the embodiment of the present application, the emergency drill action video sampling unit is used to perform key frame sampling on the emergency drill action video at a predetermined sampling frequency to obtain a time sequence of key frames of the emergency drill action. It should be understood that the emergency drill action video often contains a large number of continuous frame images, and the amount of data is huge. However, the changes between many frames are not significant, and there is a large amount of redundant information. In addition, the entire drill action is composed of a series of key action nodes, and not every frame can reflect the important progress of the drill. Therefore, the most representative frames containing key action information can be screened out. The present application performs key frame sampling on the emergency drill action video to focus on those moments when the actions undergo obvious changes and can reflect the core steps and key behaviors of the drill, and obtain a time sequence of key frames of the emergency drill action. In this way, it is helpful to more accurately capture the key action information in the drill, filter out irrelevant transition images, and make the subsequent analysis and evaluation of the action more targeted.

[0025] In an embodiment of the present application, the emergency drill action semantic parsing unit 122 is used to input each emergency drill action key frame in the time sequence of the emergency drill action key frame into the action semantic parsing module to obtain the time sequence of the emergency drill action semantic parsing coding vector as the time sequence of the emergency drill action semantic parsing feature. Specifically, in an embodiment of the present application, the action semantic parsing module is an action semantic parsing module comprising a convolution layer and an RNN layer. It should be understood that each emergency drill action key frame contains the key action features of the current time point, and there is a semantic association dependency between each local key action feature. Based on this, the present application processes each emergency drill action key frame in the time sequence of the emergency drill action key frame through an action semantic parsing module comprising a convolution layer and an RNN layer to capture and mine the action space features of each key frame and characterize the semantic information between each local action, and obtain the time sequence of the emergency drill action semantic parsing coding vector. Specifically, first, the convolution kernel slides on the emergency drill action key frame, and extracts the local spatial features in the key frame through the convolution operation, such as the body position, action posture, object placement in the scene, and other information of the drill personnel. These spatial features are crucial for understanding the semantics of actions. For example, identifying from an image whether a trainee is holding a fire extinguisher and the position of the fire extinguisher is key information for judging the fire-fighting action, and obtaining a sequence of local convolutional features corresponding to the key frame. Then, at each time step (corresponding to each local feature in the sequence of local convolutional features), the RNN combines the current local feature with the hidden state of the previous time step. For example, when processing the action of a trainee from picking up a fire extinguisher to aiming at the fire source, the local features of the fire extinguisher position extracted earlier will be passed through the hidden state to the subsequent local feature stage for processing the action of aiming at the fire source, thereby generating an accurate semantic parsing encoding vector for each key frame, which can well represent the semantic information of the action in the key frame.

[0026] In the embodiment of the present application, the drill action temporal context encoding module 130 is used to perform drill action context encoding on the time series of the semantic parsing features of the emergency drill action to obtain the drill action temporal context semantic association features. Specifically, Figure 3 FIG. 1 is a block diagram of a drill action timing context encoding module in an emergency drill evaluation and analysis system according to an embodiment of the present application. Figure 3As shown, the drill action timing context encoding module 130 includes: an emergency drill action semantic jump degree space span calculation unit 131, which is used to calculate the feature jump degree and message transmission space span of each emergency drill action semantic analysis feature in the time series of the emergency drill action semantic analysis feature to obtain the time series of emergency drill action semantic feature jump degree and the time series of emergency drill action semantic feature message transmission space span; a drill action timing weighted aggregation unit 132, which is used to perform significant weighted aggregation on the time series of the emergency drill action semantic feature jump degree based on the time series of the emergency drill action semantic feature jump degree and the time series of the emergency drill action semantic feature message transmission space span to obtain the drill action timing context semantic association feature.

[0027] It should be understood that the emergency drill action is a process with a time sequence, that is, there is a contextual association between the semantic analysis features of each emergency drill action, that is, they are interdependent in time sequence and logic, and the semantic associations between different features are also different. Relying solely on a single action semantic analysis coding vector can only understand the isolated semantics of each action, and cannot grasp the coherence and mutual influence between actions during the entire drill process. Based on this, the present application performs drill action context encoding on the time series of the semantic analysis features of the emergency drill action to obtain the drill action temporal context semantic association features. Specifically, this method utilizes a unique information transmission mechanism and takes into account the feature jump degree, which can effectively fuse the semantic information of each action, and at the same time dynamically adjust the importance of feature transmission through a gating mechanism, so as to fully reveal how different actions influence each other, thereby characterizing the contextual semantic associations of actions in different situations.

[0028] Specifically, in the embodiment of the present application, the emergency drill action semantic jump degree space span calculation unit is used to: based on the weighted average feature value of each emergency drill action semantic parsing coding vector in the time series of the emergency drill action semantic parsing coding vector and the next emergency drill action semantic parsing coding vector, calculate the feature jump degree of each emergency drill action semantic parsing coding vector to obtain the time series of the emergency drill action semantic feature jump degree. This process can be expressed as: ;in, is the time series of the semantic parsing encoding vectors of the emergency drill actions, and are the first, second, and third time series of the semantic parsing encoding vector of the emergency drill action. and emergency drill action semantic parsing encoding vector, yes Middle The eigenvalues ​​at the positions, yes The number of eigenvalues ​​in , and They are the time series of the semantic weighted average feature values ​​of emergency drill actions. and The semantic weighted average feature value of emergency drill actions, It is the first in the time series of the semantic feature jump degree of emergency drill actions. The jump degree of semantic features of emergency drill actions.

[0029] Based on the number of message transmissions between each emergency drill action semantic parsing coding vector and the last emergency drill action semantic parsing coding vector in the time series of the emergency drill action semantic parsing coding vector, the message transmission space span of each emergency drill action semantic parsing coding vector is calculated to obtain the time series of the emergency drill action semantic feature message transmission space span. This process can be expressed as: ;in, and They are the first and emergency drill action semantic parsing encoding vector, express arrive The number of message delivery times, It is the first in the time series of the spatial span of the semantic feature message transmission of the emergency drill action. The semantic feature message transmission space span of emergency drill actions.

[0030] It should be understood that in the analysis of emergency drill actions, the time series of the semantic analysis coding vectors of emergency drill actions contains the semantic information of the actions at different times in the corresponding emergency drills. This information may show complex changes at different time points, and it is difficult to clearly grasp the changes between actions from only a single vector. In this application, by calculating the weighted average eigenvalue of each emergency drill action semantic analysis coding vector and the next emergency drill action semantic analysis coding vector, and then obtaining the feature jump degree corresponding to the vector, this change can be quantified. That is, by calculating the feature jump degree, the difference information of actions at different time points can be reflected, which helps the model capture the turning point change actions in the action sequence.

[0031] Accordingly, considering that emergency drills are a complex process, the connections between actions may span multiple steps. The traditional analysis method based on adjacency relationships can only capture the connections between directly adjacent actions. In order to help the model understand the association of actions at a longer distance, it is necessary in this application to calculate the message transmission space span of each emergency drill action semantic analysis coding vector based on the number of message transmissions between each emergency drill action semantic analysis coding vector and the last emergency drill action semantic analysis coding vector in the time series of the emergency drill action semantic analysis coding vector to obtain the time series of the emergency drill action semantic feature message transmission space span. Moreover, by calculating the message transmission space span, the critical path of the action semantic information in the entire drill process can be clearly identified.

[0032] Specifically, in an embodiment of the present application, the drill action timing weighted aggregation unit includes: a message transmission significance weight calculation subunit, which is used to calculate the message transmission significance weights of each emergency drill action semantic parsing coding vector in the time series of the emergency drill action semantic feature jump degree and the time series of the emergency drill action semantic message transmission space span to obtain the time series of emergency drill action semantic feature message transmission significance weights; a drill action timing context semantic association feature generation subunit, which is used to calculate the weighted sum of the time series of the emergency drill action semantic parsing coding vector based on the time series of the emergency drill action semantic feature message transmission significance weights to obtain the drill action timing context semantic association coding vector as the drill action timing context semantic association feature.

[0033] More specifically, in the present application, the message transmission significant weight calculation subunit is used to: based on the feature jump degree and message transmission space span of each emergency drill action semantic parsing coding vector in the time series of the emergency drill action semantic parsing coding vector, calculate the message transmission significant factor of each emergency drill action semantic parsing coding vector to obtain the time series of the emergency drill action semantic feature message transmission significant factor. This process can be expressed as: ;in, It is the first in the time series of the semantic feature jump degree of emergency drill actions. The semantic feature jump degree of emergency drill actions, It is the first in the time series of the spatial span of the semantic feature message transmission of the emergency drill action. The spatial span of the semantic feature message transmission of emergency drill actions, and is the modulation parameter, It is the time series of the significant factor of the semantic feature message transmission of emergency drill actions. The semantic feature message transmission saliency factor of emergency drill actions.

[0034] The time series of the significant factors of the semantic feature message of the emergency drill action is gated and screened to obtain the time series of the significant weights of the semantic feature message of the emergency drill action. The process can be expressed as: ;in, It is the time series of the significant factor of the semantic feature message transmission of emergency drill actions. The semantic feature message transmission salient factor of emergency drill actions, is the normalization function, is a masking operation, is a predetermined threshold, is the time series of the significant weight of the semantic feature message of the emergency drill action. The semantic feature message of emergency drill actions conveys significant weights.

[0035] It should be understandable that the characteristic jump degree of each emergency drill action semantic analysis coding vector reflects the drastic degree of change of the action semantics at adjacent time points, which can reflect the turning point of the action; and the characteristic message transmission space span of each emergency drill action semantic analysis coding vector measures the influence range of a certain action semantic information in the entire drill process action semantic information propagation, indicating the degree of temporal and logical connection between the action and other actions. The importance of an action semantic information in the drill process action semantic information propagation cannot be fully evaluated based on the characteristic jump degree or the message transmission space span alone. Based on this, in this application, it is necessary to calculate the message transmission significance factor of each emergency drill action semantic analysis coding vector based on the characteristic jump degree and message transmission space span of each emergency drill action semantic analysis coding vector in the time series of the emergency drill action semantic analysis coding vector to obtain the time series of the emergency drill action semantic feature message transmission significance factor. By calculating the message transmission significance factor, an importance score is assigned to the action semantic information at each time point, so that the truly critical action semantics in the emergency drill process can be highlighted, which helps the model to quickly locate the key actions that have a significant impact on the drill results, concentrate on analysis, and make targeted evaluations in subsequent reports.

[0036] Accordingly, considering that emergency drills involve many complex actions and links, the information carried by the significant factor of each action semantic feature message transmission plays a different role in the entire drill evaluation. Taking fire emergency drills as an example, there may be multiple actions such as personnel evacuation, fire extinguishing operations, and rescue of the wounded that occur in parallel or successively. The information generated by these actions is rich and diverse. If not screened, too much redundant or secondary information may interfere with the analysis of key information. In order to make the model focus on key action semantic information and strengthen the role of important action semantic information in the analysis, it is necessary to perform gated screening on the time series of the significant factors of the semantic feature message transmission of the emergency drill action in this application to obtain the time series of the significant weights of the semantic feature message transmission of the emergency drill action. Through gated screening, the message transmission significant weights related to key actions will increase, so that the semantic feature information representing the key actions can be more prominent in the time series, while the relevant semantic features of the less important actions in the drill will be appropriately weakened.

[0037] Finally, the weighted sum of the time series of the semantic parsing coding vector of the emergency drill action is calculated based on the time series of the significant weight of the semantic feature message transmission of the emergency drill action to obtain the drill action temporal context semantic association coding vector as the drill action temporal context semantic association feature. The above process can be expressed as: ;in, is the time series of the significant weight of the semantic feature message of the emergency drill action. The semantic feature message of emergency drill action transfers significant weights, is the first emergency drill action semantic parsing encoding vector, is the number of vectors in the time series of the semantic parsing encoding vectors of the emergency drill action, It is the semantic association encoding vector of the temporal context of the rehearsal action.

[0038] It should be understandable that in emergency drills, the importance of each action is not consistent. The time series of the significant weights of the semantic feature message transmission of emergency drill actions represents the importance of each action semantic information in the information dissemination process. By calculating the weighted sum of the time series of the semantic parsing coding vectors of emergency drill actions, it is possible to integrate information in a targeted manner according to the importance of the action. In other words, the generated drill action temporal context semantic association coding vector can well reflect the overall coherence of the emergency drill while highlighting key actions, thereby providing a more accurate basis for emergency drill evaluation.

[0039] In an embodiment of the present application, the drill evaluation analysis report generation module 140 is used to obtain a drill evaluation analysis report based on the drill action temporal context semantic association features. Specifically, in an embodiment of the present application, the drill evaluation analysis report generation module is used to: after adding a prompt word at the end of the drill action temporal context semantic association coding vector, input it into the drill evaluation analysis module based on the large language model to obtain the drill evaluation analysis report. That is, the drill action temporal context semantic association features obtained by context semantic association using the time series of the emergency drill action semantic parsing coding vector are generated and processed to utilize the powerful language understanding and generation capabilities of the large language model, and according to the prompt word, to clearly inform the model that the task to be completed is to generate a drill evaluation analysis report. For example, the prompt word can be "generate a detailed drill evaluation analysis report based on the action information of the above emergency drill", so that the large language model will focus on the specific task of emergency drill evaluation and avoid generating content unrelated to the drill.

[0040] The following is a detailed description of a specific implementation process of "after adding a prompt word to the tail of the drill action temporal context semantic association encoding vector, input it into the drill evaluation analysis module based on the large language model to obtain the drill evaluation analysis report": First, with regard to adding a prompt word to the tail of the drill action temporal context semantic association encoding vector, this is a process that requires comprehensive consideration of multiple factors. On the one hand, the design of the prompt word is crucial, which directly guides the focus and direction of the large language model on the drill evaluation. When designing the prompt word, it needs to be customized according to different emergency drill types and specific evaluation purposes. For common emergency drills, such as fire emergency drills, prompt words can be designed from multiple dimensions. For example, for personnel evacuation, it can be "please evaluate in detail the overall speed and order of personnel evacuation in this fire emergency drill, and the rationality of the evacuation route, and analyze whether there is congestion, chaos, etc."; for fire fighting operations, it may be "evaluate the fire fighting process, the fire fighting personnel's proficiency in the operation of fire fighting equipment, including but not limited to the use of fire extinguishers, whether the operation steps of fire hydrants comply with standard procedures, and the timeliness and effectiveness of the operation." The design of these prompts should not only cover all important aspects of the drill, but also be targeted and instructive so that the large language model can better focus on key issues.

[0041] When adding the prompt word to the tail of the drill action temporal context semantic association encoding vector, it should be noted that since the encoding vector is an abstract representation generated based on a deep learning algorithm, and the prompt word is in natural language form, it needs to be converted. Generally speaking, word embedding technology in natural language processing is used, such as Word2Vec, GloVe, etc. These technologies can convert the prompt word of natural language into a corresponding digital code or vector representation, so that it is in the same dimensional space as the original drill action temporal context semantic association encoding vector. Assuming that the original drill action temporal context semantic association encoding vector is a vector with a specific length (assuming it is N), the prompt word is converted into a vector of length M through word embedding technology, then this vector of length M will be seamlessly added to the tail of the original vector to form a new vector of length N+M. This process needs to ensure that the converted vector representation can accurately convey the semantic information of the prompt word and merge with the semantic information of the original encoding vector to avoid information loss or confusion.

[0042] Next, the encoding vector after adding the prompt word is input into the drill evaluation and analysis module based on the large language model. In this process, the first thing to do is to select a suitable large language model according to the specific situation. There are currently a variety of large language models available on the market, such as powerful commercial models such as GPT-3 and GPT-4, which have extensive knowledge reserves and excellent language understanding capabilities, but may require the use of corresponding API interfaces and pay a certain fee; you can also choose open source large language models, such as BERT, etc. For organizations or teams with certain technical capabilities, they can be deployed locally to meet specific customized needs. Taking GPT-4 as an example, it is necessary to use the OpenAI API interface to use the prepared encoding vector containing the prompt word as input data, strictly follow the API call specifications, and send it to the large language model service through HTTP, request or corresponding software development kit (SDK). When calling the large language model, its internal complex deep learning architecture will process the input vector. It will conduct a comprehensive analysis of the drill action information represented by the vector and the added prompt word requirements based on its own language knowledge and semantic understanding capabilities trained on massive data. The large language model will deeply explore the detailed information of the person's actions contained in the encoding vector, including the sequence of actions, duration, and logical relationship between actions, and conduct a comprehensive evaluation based on the evaluation focus indicated by the prompt words.

[0043] Finally, the generated drill evaluation and analysis report will cover many important information. At the beginning of the report, an overall evaluation will be given, and the effect of the entire emergency drill will be summarized. For example, for a fire emergency drill, it may be stated as "this fire emergency drill has shown a certain degree of organization and execution as a whole, but there are many deficiencies in the details, which need to be further improved and perfected". Subsequently, the report will go deep into the action analysis level and conduct a detailed evaluation of various actions of personnel. For example, in the fire emergency drill, it will be pointed out in detail that "some personnel appeared in the evacuation process. The phenomenon of serious crowding, did not follow the predetermined evacuation route, pushed each other, resulting in slow evacuation speed and prolonged overall evacuation time, and some personnel were not familiar with the evacuation signs and failed to make full use of the safety passage"; in terms of fire extinguishing operations, it will be mentioned that "the fire extinguishers failed to correctly master the use of fire extinguishers, such as not pulling out the safety pin, not aiming at the root of the flame, and unfamiliar operation techniques, resulting in low efficiency in the initial fire extinguishing process and missed the best time to extinguish the fire". In the process evaluation section, the entire emergency drill process will be analyzed. For example, "In the emergency response process, there is a certain delay from the discovery of the fire to the initiation of the alarm and notification procedures, which is about [X] minutes. This seriously affects the orderly development of subsequent rescue operations. There are also problems with communication between rescue teams in information transmission and task allocation, which makes some links not tightly connected, affecting the overall rescue efficiency." Finally, the report will give improvement suggestions. For fire emergency drills, it may be recommended to "strengthen evacuation drill training for participants, not only to clarify the evacuation route, but also to set up more eye-catching evacuation signs at the drill site, and organize personnel to conduct multiple familiarization drills on the evacuation route to ensure that everyone can evacuate quickly and orderly; carry out intensive training on the operation of fire-fighting equipment for fire-fighting personnel, so that they can master the use steps of various types of fire-fighting equipment and improve their operational proficiency; optimize the emergency response process to ensure the timeliness and accuracy of information transmission and reduce delays in intermediate links."

[0044] Through the above implementation steps, a comprehensive and detailed exercise evaluation and analysis report will eventually be generated, which can provide targeted feedback and guidance to the organizers and participants of the emergency drills, help them continuously optimize the process and operation of emergency drills, improve the emergency response capabilities of personnel and the comprehensive level of coping with emergencies, and maximize the protection of people’s lives and property and social stability, so as to truly implement emergency drills and realize their important functions of preventing and responding to disasters.

[0045] Here, when each emergency drill action semantic parsing coding vector in the time series of the emergency drill action semantic parsing coding vector respectively represents the local associated image semantic features of the corresponding emergency drill action key frame, when performing feature sequence message transmission based on feature jump degree, the local image semantic space distribution jump degree in the global time domain will have insufficient associated correspondence, thereby causing sparse sequence message association transmission of the drill action temporal context semantic association coding vector. Due to the lack of generated reasoning degree, the semantic quality of the drill evaluation and analysis report obtained by inputting the drill evaluation and analysis module based on the large language model is reduced.

[0046] Therefore, after adding a prompt word to the end of the rehearsal action temporal context semantic association encoding vector, the present application first optimizes the rehearsal action temporal context semantic association encoding vector when inputting it into the rehearsal evaluation analysis module based on the large language model. The optimization process includes the following steps: Based on the relationship between the feature values ​​of the semantic association encoding vector of the rehearsal action temporal context Distance and The distance matrix of the time sequence context semantic association of the exercise action is obtained and the distance matrix of the semantic association of the temporal context of the exercise action .

[0047] The weighted sum of the first distance matrix of the drill action temporal context semantic association and the second distance matrix of the drill action temporal context semantic association is calculated to obtain the drill action temporal context semantic association joint distance matrix. The process can be expressed as: ;in, represents the temporal context semantic association of the exercise action - a distance matrix, represents the temporal context semantic association of the exercise action, It means point multiplication by position. Indicates adding by position point. and represents the weighted hyperparameter, Represents the joint distance matrix of the temporal context semantic association of the drill action. ,

[0048] Determine the eigenvalues ​​of the joint distance matrix of the temporal context semantic association of the exercise action , and compose the joint distance eigenvector of the training action temporal context semantic association .

[0049] The rehearsal action temporal context semantic association encoding vector as a row vector is matrix-multiplied with the rehearsal action temporal context semantic association-distance matrix to obtain the rehearsal action temporal context semantic association-distance query vector. This process can be expressed as: ;in, represents the temporal context semantic association of the exercise action - a distance matrix, represents matrix multiplication, represents the semantic association encoding vector of the temporal context of the exercise action, and is a row vector, A distance query vector representing the temporal context semantic association of the drill action.

[0050] The two distance matrices of the drill action temporal context semantic association are matrix-multiplied with the self-association matrix of the drill action temporal context semantic association encoding vector to obtain the two distance association matrices of the drill action temporal context semantic association. The process can be expressed as: ;in, represents the temporal context semantic association of the exercise action, represents matrix multiplication, represents the semantic association encoding vector of the temporal context of the exercise action, and is a row vector, express The transposed vector of A distance association matrix representing the temporal context semantic association of the drill action.

[0051] After matrix multiplication of the exercise action temporal context semantic association one distance query vector and the exercise action temporal context semantic association two distance association matrix, further dot multiplication with the exercise action temporal context semantic association joint distance eigenvector is performed to obtain an optimized exercise action temporal context semantic association encoding vector. This process can be expressed as: ;in, represents the temporal context semantic association of the exercise action-distance query vector, represents the temporal context semantic association and distance association matrix of the exercise action, It means point multiplication by position. represents matrix multiplication, represents the joint distance eigenvector of the temporal context semantic association of the exercise action, A semantic association encoding vector representing the optimized rehearsal action temporal context.

[0052] That is, the first distance matrix and the second distance matrix of the rehearsal action temporal context semantic association coding vector are used as the fine-grained metric association cluster representation of the rehearsal action temporal context semantic association coding vector, and the dynamic programming of the relationship between association clusters of different association clusters is performed on the self-association representation of the rehearsal action temporal context semantic association coding vector and the rehearsal action temporal context semantic association coding vector respectively to simulate the sparse activation of the association system based on neuron clusters, and the fine-grained predictable sparsity of the rehearsal action temporal context semantic association coding vector is coordinated with the eigenvalue representation of the metric association cluster of the first distance matrix and the second distance matrix of the rehearsal action temporal context semantic association coding vector, so as to avoid the lack of association caused by sparsity affecting the generation of reasoning degree loss, and improve the semantic quality of the rehearsal evaluation analysis report obtained by inputting the rehearsal evaluation analysis module based on the large language model into the rehearsal action temporal context semantic association coding vector.

[0053] In summary, the emergency drill evaluation and analysis system 100 based on the embodiment of the present application is explained, which uses deep learning-based video data analysis technology to perform key frame sampling on the emergency drill action video of the target personnel object, and then performs action semantic analysis on each sampled emergency drill action key frame, so as to intelligently generate a drill evaluation and analysis report based on the contextual semantic association representation of the feature jump degree of each action semantic analysis feature. In this way, the actions of each target person can be carefully identified and tracked, reducing the reliance on manual observation and subjective judgment, thereby reducing the problem of inconsistent evaluation results caused by individual differences. At the same time, it is possible to deeply understand the logical connection between each action and its role in the entire drill process, thereby generating a more accurate drill evaluation and analysis report.

[0054] Figure 4 FIG. 1 is a flow chart of an emergency drill evaluation and analysis method according to an embodiment of the present application. Figure 4 As shown, in the emergency drill evaluation and analysis method, it includes: S1, obtaining an emergency drill action video of a target person object captured by a camera; S2, performing key frame sampling and action semantic analysis on the emergency drill action video to obtain a time series of emergency drill action semantic analysis features; S3, performing drill action context encoding on the time series of emergency drill action semantic analysis features to obtain drill action temporal context semantic association features; S4, obtaining a drill evaluation and analysis report based on the drill action temporal context semantic association features.

[0055] Here, those skilled in the art will appreciate that the specific operations of each step in the above emergency drill evaluation and analysis method have been described in detail above. Figures 1 to 3 The description of the emergency drill evaluation and analysis system has been introduced in detail, and therefore, its repeated description will be omitted.

[0056] In summary, the emergency drill evaluation and analysis method based on the embodiment of the present application is explained, which uses deep learning-based video data analysis technology to perform key frame sampling on the emergency drill action video of the target personnel object, and then performs action semantic analysis on each emergency drill action key frame after sampling, so as to intelligently generate a drill evaluation and analysis report based on the contextual semantic association representation of the feature jump degree of each action semantic analysis feature. In this way, the actions of each target person can be carefully identified and tracked, reducing the reliance on manual observation and subjective judgment, thereby reducing the problem of inconsistent evaluation results caused by individual differences. At the same time, it is possible to deeply understand the logical connection between each action and its role in the entire drill process, thereby generating a more accurate drill evaluation and analysis report.

Claims

1. An emergency drill evaluation and analysis system, characterized in that: include: The emergency drill action video acquisition module is used to acquire the emergency drill action video of the target personnel object captured by the camera; the key frame sampling semantic analysis module is used to perform key frame sampling and action semantic analysis on the emergency drill action video to obtain the time sequence of the emergency drill action semantic analysis features; the drill action temporal context encoding module is used to perform drill action context encoding on the time sequence of the emergency drill action semantic analysis features to obtain the drill action temporal context semantic association features, wherein the drill action temporal context encoding module includes: an emergency drill action semantic jump degree space span calculation unit, which is used to calculate the time sequence of the emergency drill action semantic analysis features in the time sequence The feature jump degree and message transmission space span of each emergency drill action semantic analysis feature are used to obtain the time series of the jump degree of the emergency drill action semantic feature and the time series of the message transmission space span of the emergency drill action semantic feature; a drill action timing weighted aggregation unit is used to perform significant weighted aggregation on the time series of the emergency drill action semantic analysis feature based on the time series of the jump degree of the emergency drill action semantic feature and the time series of the message transmission space span of the emergency drill action semantic feature to obtain the drill action timing context semantic association feature; a drill evaluation analysis report generation module is used to obtain the drill evaluation analysis report based on the drill action timing context semantic association feature.

2. The emergency drill evaluation and analysis system according to claim 1, characterized in that: The key frame sampling semantic analysis module includes: an emergency drill action video sampling unit, which is used to perform key frame sampling on the emergency drill action video to obtain a time sequence of emergency drill action key frames; an emergency drill action semantic analysis unit, which is used to input each emergency drill action key frame in the time sequence of the emergency drill action key frames into the action semantic analysis module to obtain a time sequence of emergency drill action semantic analysis coding vectors as a time sequence of the emergency drill action semantic analysis features.

3. The emergency drill evaluation and analysis system according to claim 2, characterized in that: The emergency drill action video sampling unit is used to perform key frame sampling on the emergency drill action video at a predetermined sampling frequency to obtain a time sequence of the emergency drill action key frames.

4. The emergency drill evaluation and analysis system according to claim 3, characterized in that: The action semantics parsing module is an action semantics parsing module that includes a convolutional layer and a RNN layer.

5. The emergency drill evaluation and analysis system according to claim 4, characterized in that: The emergency drill action semantic jump degree space span calculation unit is used to: calculate the feature jump degree of each emergency drill action semantic parsing coding vector based on the weighted average feature value of each emergency drill action semantic parsing coding vector and the next emergency drill action semantic parsing coding vector in the time sequence of the emergency drill action semantic parsing coding vector to obtain the time sequence of the emergency drill action semantic feature jump degree; calculate the message transmission space span of each emergency drill action semantic parsing coding vector based on the number of message transmissions between each emergency drill action semantic parsing coding vector and the last emergency drill action semantic parsing coding vector in the time sequence of the emergency drill action semantic parsing coding vector to obtain the time sequence of the emergency drill action semantic feature message transmission space span.

6. The emergency drill evaluation and analysis system according to claim 5, characterized in that: The drill action timing weighted aggregation unit includes: a message transmission significant weight calculation subunit, which is used to calculate the message transmission significant weight of each emergency drill action semantic analysis coding vector in the time series of the emergency drill action semantic analysis coding vector based on the time series of the emergency drill action semantic feature jump degree and the time series of the emergency drill action semantic message transmission space span to obtain the time series of emergency drill action semantic feature message transmission significant weights; a drill action timing context semantic association feature generation subunit, which is used to calculate the weighted sum of the time series of the emergency drill action semantic analysis coding vector based on the time series of the emergency drill action semantic feature message transmission significant weights to obtain the drill action timing context semantic association coding vector as the drill action timing context semantic association feature.

7. The emergency drill evaluation and analysis system according to claim 6, characterized in that: The message transmission significance weight calculation subunit is used to: calculate the message transmission significance factor of each emergency drill action semantic analysis coding vector based on the feature jump degree and message transmission space span of each emergency drill action semantic analysis coding vector in the time series of the emergency drill action semantic analysis coding vector to obtain the time series of emergency drill action semantic feature message transmission significance factor; perform gated screening on the time series of the emergency drill action semantic feature message transmission significance factor to obtain the time series of the emergency drill action semantic feature message transmission significance weight.

8. The emergency drill evaluation and analysis system according to claim 7, characterized in that: The drill evaluation analysis report generation module is used to: after adding a prompt word at the end of the drill action temporal context semantic association encoding vector, input it into the drill evaluation analysis module based on the large language model to obtain the drill evaluation analysis report.

9. An emergency drill evaluation and analysis method, characterized in that: include: Acquire an emergency drill action video of a target person object captured by a camera; perform key frame sampling and action semantic analysis on the emergency drill action video to obtain a time series of emergency drill action semantic analysis features; Performing drill action context encoding on the time series of the emergency drill action semantic parsing features to obtain drill action temporal context semantic association features, including: calculating the feature jump degree and message transmission space span of each emergency drill action semantic parsing feature in the time series of the emergency drill action semantic parsing features to obtain the time series of the emergency drill action semantic feature jump degree and the time series of the emergency drill action semantic feature message transmission space span; based on the time series of the emergency drill action semantic feature jump degree and the time series of the emergency drill action semantic feature message transmission space span, performing significant weighted aggregation on the time series of the emergency drill action semantic parsing features to obtain the drill action temporal context semantic association features; Based on the semantic association features of the drill action temporal context, a drill evaluation and analysis report is obtained.

10. The emergency drill evaluation and analysis method according to claim 9, characterized in that: Key frame sampling and action semantic analysis are performed on the emergency drill action video to obtain a time sequence of emergency drill action semantic analysis features, including: key frame sampling is performed on the emergency drill action video to obtain a time sequence of emergency drill action key frames; each emergency drill action key frame in the time sequence of the emergency drill action key frames is input into an action semantic analysis module to obtain a time sequence of emergency drill action semantic analysis coding vectors as the time sequence of the emergency drill action semantic analysis features.

Images