Video event alarm method and system based on hierarchical knowledge graph, and medium

Abstract

The invention discloses a video event alarm method and system based on a hierarchical knowledge graph, and a medium, and belongs to the technical field of video surveillance, and the method comprises the steps: constructing a data set, and obtaining a video sequence; acquiring an image frame based on the video sequence; marking a key frame based on each image frame; constructing a hierarchical knowledge graph based on the key frame; based on the hierarchical knowledge graph, training an alarm model to extract global features; based on the hierarchical knowledge graph, selecting a sub-graph training alarm model to extract key features; detecting an occurrence probability of a specific event in the video data based on a trained alarm model; and determining alarm information based on the occurrence probability of the specific event and the dynamic threshold set. The technical effects of reducing the false alarm rate and the missing report rate and improving the stability and the reliability of the video monitoring system are achieved.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a video event alarm method, system and medium based on a hierarchical knowledge graph. Background Art

[0002] In recent years, with the rapid development of computer vision and machine learning technologies, traditional video surveillance systems have transformed from simple recording and playback functions to intelligent systems capable of complex scene analysis and real-time decision-making.

[0003] In the existing technology, video surveillance systems usually use high-definition cameras with edge computing devices to achieve high-frame-rate video acquisition and preliminary data processing. These systems can support functions such as face recognition and license plate recognition, and are widely used in traffic management, commercial retail, home security and other fields. However, with the integration of artificial intelligence technology, modern video surveillance systems have begun to have behavioral analysis capabilities, such as fall detection and abnormal behavior recognition. These functions have shown high application value in scenarios such as elderly care and school safety.

[0004] However, the existing video surveillance systems have high false alarm and missed alarm rates, especially in environments with large lighting changes and severe occlusions. The stability and reliability of the video surveillance systems need to be improved. Summary of the invention

[0005] To solve the above problems, the first aspect of the present invention provides a video event alarm method based on a hierarchical knowledge graph, comprising:

[0006] Build a data set and obtain video sequences;

[0007] Based on the video sequence, image frames are acquired;

[0008] Based on each image frame, key frames are marked;

[0009] Based on key frames, construct a hierarchical knowledge graph;

[0010] Based on the hierarchical knowledge graph, the alarm model is trained to extract global features;

[0011] Based on the hierarchical knowledge graph, sub-graphs are selected to train the alarm model to extract key features;

[0012] Detect the probability of occurrence of specific events in video data based on the trained alarm model;

[0013] Based on the probability of occurrence of a specific event and a dynamic threshold set, the alarm information is determined.

[0014] In some embodiments, the data set is a video data set including fire and / or person fall events;

[0015] The video sequence is a sequence of image frames in the video data obtained in time order.

[0016] In some embodiments, the image frame is acquired by:

[0017] I(t)=extract_frame(Video,t);

[0018] Where I(t) is the image frame at time point t, Video is the entire video sequence, and t is the time index;

[0019] The keyframes are marked as follows:

[0020] K(t)=is_key_frame(I(t));

[0021] Among them, K(t) is a binary value indicating whether frame I(t) is a key frame.

[0022] In some embodiments, the entity definitions in the hierarchical knowledge graph include: flame, personnel, and environmental objects;

[0023] The environmental objects include furniture, doors and windows;

[0024] The relationship definition between entities in the hierarchical knowledge graph includes:

[0025] Adjacent means that two entities are close in space;

[0026] Overlay: One entity visually covers another entity;

[0027] In line of sight: One entity is visible from the line of sight of another entity;

[0028] The entity is defined as follows:

[0029] v_i = create_node(o_i);

[0030] Among them, o_i is the object identified from the video frame, and vi_i is the node representing the object in the graph;

[0031] The relationship is defined as follows:

[0032] e_ij=(v_i,v_j,type);

[0033] Among them, e_ij is the edge connecting nodes v_i and v_j, and type is the type of edge determined according to their spatial relationship.

[0034] In some embodiments, the alarm model is a graph convolutional network (GCN) model;

[0035] The global feature is a comprehensive feature that can represent all nodes in the graph and their relationships;

[0036] The method of training the alarm model to extract global features is:

[0037] H^(l+1)=sigma(sum(A_ij*W^(l)*H^(l)(j)+B^(l)*H^(l)(i),j belongs toNeighbors(i)));

[0038] in:

[0039] H^(l+1): represents the node features of the l+1th layer;

[0040] H^(l)(j): represents the characteristics of neighbor node j in layer l;

[0041] A_ij: the adjacency matrix coefficient between nodes i and j, reflecting the importance of the connection strength or relationship type between the two nodes;

[0042] W^(l): weight matrix of the lth layer, used to update node features;

[0043] B^(l): bias term of the lth layer, used to adjust the output of each layer;

[0044] Sigma: Activation function, used to introduce nonlinearity so that the model can capture complex patterns.

[0045] In some embodiments, the subgraph is a local graph structure extracted for a specific event or object in the video content, and the subgraph includes a portion of nodes and edges in the original graph;

[0046] The key features are comprehensive features of nodes related to a specific event and their interrelationships;

[0047] The method of selecting sub-graphs to train the alarm model and extract key features is:

[0048] H_sub^(l+1)=sigma(sum(A_sub,ij*W_sub^(l)*H_sub^(l)(j)+

[0049] B_sub^(l)*H_sub^(l)(i),j belongs to Neighbors_sub(i)))

[0050] in:

[0051] H_sub^(l+1): represents the node features of the l+1th layer in the subgraph;

[0052] H_sub^(l)(j): represents the characteristics of the neighbor node j in the lth layer of the subgraph;

[0053] A_sub,ij: the adjacency matrix coefficient between nodes i and j in the subgraph, reflecting the connection strength or the importance of the relationship type between the two nodes;

[0054] W_sub^(l): weight matrix of the subgraph-specific layer l, used to update node features;

[0055] B_sub^(l): bias term for the subgraph specific layer l, used to adjust the output of each layer;

[0056] Sigma: activation function, used to introduce nonlinearity so that the model can capture complex patterns;

[0057] The specific events include: fire events and fall events.

[0058] In some embodiments, the method of detecting the occurrence probability of a specific event in the video data is:

[0059] P(event|F)=softmax(sum(W_d*F_k+b,k in Features))

[0060] in:

[0061] P(event|F): the probability of an event occurring given a feature set F;

[0062] F_k: the kth feature in the feature set extracted from the graph;

[0063] W_d: weight matrix for event classification;

[0064] b: bias term, used to adjust the model output;

[0065] Softmax: A function that converts linear output into a probability distribution.

[0066] In some embodiments, the dynamic threshold set is a set of thresholds used to determine whether a specific event occurs;

[0067] The alarm information is confirmed by comparing the probability of occurrence of a specific event with a dynamic threshold concentration threshold;

[0068] The dynamic threshold is automatically adjusted based on real-time environmental data;

[0069] The dynamic threshold is set as follows:

[0070] threshold=base_threshold-(environment_factor*adjustment_rate)

[0071] Among them, threshold is the preset probability threshold, base_threshold is the basic threshold, environment_factor represents the value obtained from the environmental sensor, and adjustment_rate is the adjustment coefficient.

[0072] The second aspect of this specification provides a video event alarm system based on a hierarchical knowledge graph, which is used to execute the aforementioned video event alarm method based on a hierarchical knowledge graph.

[0073] A third aspect of the present specification provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the aforementioned video event alarm method based on a hierarchical knowledge graph.

[0074] By adopting the above technical solution, the present invention mainly has the following technical effects:

[0075] By training the alarm model to extract key features of global features, the trained alarm model is used to detect the probability of occurrence of specific events in video data; by comparing the probability of occurrence of specific events with the dynamic threshold set, the alarm information is determined, thereby greatly reducing the false alarms caused by the mismatch between the threshold and the scene, achieving the technical effect of reducing the false alarm rate and the missed alarm rate, and improving the stability and reliability of the video surveillance system. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a schematic diagram of an application scenario of a video event alarm system based on a hierarchical knowledge graph according to some embodiments of this specification;

[0077] Figure 2 It is an exemplary flow chart of a video event alarm method based on a hierarchical knowledge graph according to some embodiments of this specification. DETAILED DESCRIPTION

[0078] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0079] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0080] As shown in this specification, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0081] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.

[0082] Figure 1 It is a schematic diagram of an application scenario of a video event alarm system based on a hierarchical knowledge graph according to some embodiments of this specification;

[0083] like Figure 1 As shown, the application scenario may include a server 110, a network 120, a terminal device 130, a monitoring device 140, a storage device 150, and a target scene 160. In some embodiments, the video event alarm system 100 based on the hierarchical knowledge graph can monitor the scene information in the target scene 160 by implementing the video event alarm method based on the hierarchical knowledge graph disclosed in this specification, and issue an alarm message in time when an event occurs in the target scene 160. For example, in a typical application scenario, the video event alarm system 100 based on the hierarchical knowledge graph can obtain the scene information of the target scene 160 through the monitoring device 140, determine the alarm event in the target scene 160 through the server 110 based on the scene information, and issue an alarm message in time. In some embodiments, the alarm events include: fire and people falling.

[0084] The server 110 and the terminal device 130 can be connected through the network 120, and the server 110 can be connected to the storage device 150 through the network 120. The server 110 may include a processing device 112, and the processing device 112 may be used to execute the video event alarm method based on the hierarchical knowledge graph described in some embodiments of this specification. The network 120 can connect the various components of the application scene 100 and / or connect the system with the external resource part. The storage device 150 can be used to store data and / or instructions. The storage device 150 can be directly connected to the server 110 or inside the server 110. The terminal device 130 refers to one or more terminal devices or software. In some embodiments, the terminal device 130 can receive the alarm information of the target scene 160 sent by the processing device and display it to the user. Exemplarily, the terminal device 130 may include one or any combination of a mobile device 130-1, a tablet computer 130-2, a laptop computer 130-3, etc. or other devices with input and / or output functions. The monitoring device 140 can be used to obtain geological information of the target scene 160. An exemplary monitoring device 140 may include a monitoring camera. An exemplary target scene 160 may be a public place such as a shopping mall or a hospital, or a home.

[0085] The video event alarm system 100 based on the hierarchical knowledge graph is provided only for the purpose of illustration and is not intended to limit the scope of this specification. For those of ordinary skill in the art, various modifications or variations can be made according to the description of this specification. For example, the video event alarm system 100 based on the hierarchical knowledge graph can be implemented on other devices to achieve similar or different functions. However, changes and modifications will not deviate from the scope of this specification.

[0086] Figure 2 is an exemplary flow chart of a video event alarm method based on a hierarchical knowledge graph according to some embodiments of this specification;

[0087] like Figure 2 As shown, the process 200 includes the following steps. In some embodiments, the process 200 can be executed by the processing device 112.

[0088] Step 210, constructing a data set and acquiring a video sequence;

[0089] In some embodiments, the data set is a video data set including fire and / or person fall events. In some embodiments, the video data set includes various environmental conditions, including: different lighting conditions, different backgrounds, and different camera angles.

[0090] In some embodiments, the flame video may come from a controlled flame test in a laboratory setting or from surveillance footage of an actual fire scene; the fall event video may come from security cameras in a medical institution, public place, or home environment.

[0091] In some embodiments, the video sequence is a sequence of image frames in the video data obtained in time order; wherein the time order may be in forward time order or in reverse time order.

[0092] Step 220, acquiring image frames based on the video sequence;

[0093] In some embodiments, by acquiring image frames from a video sequence, the content of each image frame may be analyzed individually.

[0094] In some embodiments, the image frame is acquired by:

[0095] I(t)=extract_frame(Video,t);

[0096] Among them, I(t) is the image frame at time point t, Video is the entire video sequence, and t is the time index.

[0097] Step 230, labeling key frames based on each image frame;

[0098] In some embodiments, the key frame is an image frame containing useful information. In some embodiments, the key frame may be an image frame where an event occurs.

[0099] In some embodiments, key frames are marked as follows:

[0100] K(t)=is_key_frame(I(t));

[0101] Among them, K(t) is a binary value indicating whether frame I(t) is a key frame.

[0102] In some embodiments, labeling the key frame further includes: labeling the event category and the coordinates of the event occurrence of the key frame; in some embodiments, the event category of the key frame includes: flame and / or fall. In some embodiments, the event category and the coordinates of the event occurrence of the key frame can be labeled manually or automatically by a pre-trained target detection algorithm.

[0103] Step 240, constructing a hierarchical knowledge graph based on the key frames;

[0104] In some embodiments, a hierarchical knowledge graph is a special knowledge graph that not only contains entities (nodes) and the relationships (edges) between them, but also emphasizes the hierarchical structure of these entities and relationships. This structure can capture the classification information of entities and the hierarchical relationships between them, so that the knowledge graph can more richly and accurately represent complex knowledge systems.

[0105] In some embodiments, the entity definitions in the hierarchical knowledge graph include: flames, personnel, and environmental objects; in some embodiments, the environmental objects may be furniture, doors, windows, and the like.

[0106] In some embodiments, the relationship definitions between entities in the hierarchical knowledge graph include: adjacent, indicating that two entities are close in space; covering: one entity visually covers another entity; within line of sight: one entity is visible from the line of sight of another entity.

[0107] In some embodiments, the entity is defined as follows:

[0108] v_i = create_node(o_i);

[0109] Among them, o_i is the object recognized from the video frame, and vi_i is the node in the graph representing the object.

[0110] In some embodiments, objects can be identified in key frames by a target detection algorithm. An exemplary target detection algorithm can be a YOLO algorithm. For each object o_i detected in the key frame, a node v_i can be created for it in the graph.

[0111] In some embodiments, the relationship is defined as follows:

[0112] e_ij=(v_i,v_j,type);

[0113] Among them, e_ij is the edge connecting nodes v_i and v_j, and type is the type of edge determined according to their spatial relationship (such as adjacent, overlapping, or in line of sight).

[0114] In some embodiments, if two objects o_i and o_j are spatially intersecting or close in a keyframe, an edge is defined between them, and the type of the edge may be determined based on the spatial relationship.

[0115] By building a hierarchical knowledge graph, complex video content can be converted into a structured graphical representation, which helps to understand the interaction of various elements in the scene on the one hand, and can also use graph algorithms to perform advanced analysis, such as path search, pattern recognition, etc., which is very useful for event prediction and response. In addition, this structured representation can also support complex queries, such as "identify all people who can directly look at the flame", which plays a very critical role in emergency evacuation and danger warning.

[0116] Step 250, based on the hierarchical knowledge graph, training the alarm model to extract global features;

[0117] In some embodiments, the alarm model can be any deep learning model that can be used to process graph structured data.

[0118] In some embodiments, the alarm model may be a graph convolutional network (GCN) model. In some embodiments, a graph convolutional network (GCN) is suitable for capturing and utilizing topological information in graph data, such as social networks, molecular structures, and traffic networks. GCN learns the representation of nodes by performing convolution operations on the graph, thereby being able to capture dependencies between nodes.

[0119] In some embodiments, the global feature is a comprehensive feature that can represent all nodes in the graph and their relationships.

[0120] In some embodiments, the training alarm model extracts global features in the following manner:

[0121] H^(l+1)=sigma(sum(A_ij*W^(l)*H^(l)(j)+B^(l)*H^(l)(i),j belongs toNeighbors(i)));

[0122] in:

[0123] H^(l+1): represents the node features of the l+1th layer;

[0124] H^(l)(j): represents the characteristics of neighbor node j in layer l;

[0125] A_ij: the adjacency matrix coefficient between nodes i and j, reflecting the importance of the connection strength or relationship type between the two nodes;

[0126] W^(l): weight matrix of the lth layer, used to update node features;

[0127] B^(l): The bias term of the lth layer, used to adjust the output of each layer.

[0128] Sigma: Activation function, used to introduce nonlinearity so that the model can capture complex patterns.

[0129] For example, in a surveillance video, which includes multiple persons, environmental objects, and potential safety events (such as fire), the steps of extracting global features through the learning model include:

[0130] Build VideoKG (knowledge graph), identify all relevant entities (people, flames, etc.) in the video and their relationships (adjacent, overlapping, etc.), and build a knowledge graph;

[0131] Apply GCN and use graph convolutional networks to process knowledge graphs. For each node, GCN considers the information of all its neighbors, uses the adjacency matrix to determine the contribution weight of the neighboring nodes, adjusts the features through the weight matrix and bias terms, and applies activation functions to generate new node features.

[0132] In some embodiments, for each node, GCN can aggregate the information of neighbors based on the weights in the adjacency matrix (e.g., directly connected nodes may have a greater influence than indirectly connected nodes), and this process is repeated multiple times (through multiple layers of GCN) so that the features of each node can capture information from a wide range of neighborhoods.

[0133] Feature extraction,After multiple layers of graph convolution, the features of each node are updated,,containing information of the global view.

[0134] In some embodiments, by training the alarm model to extract global features, we can extract useful information from the complex scenes of the entire video, and improve the accuracy and efficiency of event detection by detecting and analyzing global features. For example, in security monitoring scenarios, it can help quickly identify potential security threats, such as quickly and accurately identifying and tracking the source of a fire and its scope of influence when a fire occurs, so as to respond in time and take measures to avoid or reduce losses.

[0135] Step 260, based on the hierarchical knowledge graph, select a subgraph to train the alarm model to extract key features;

[0136] In some embodiments, the subgraph is a local graph structure extracted for a specific event or object in the video content, and the subgraph includes a portion of nodes and edges in the hierarchical knowledge graph, and is usually used to represent local features or structures in the video, for example, a character relationship network in the video or an object layout in a scene.

[0137] In some embodiments, the key features are comprehensive features of nodes related to a specific event and their relationships.

[0138] In some embodiments, the specific events include: fire events and fall events; in some embodiments, the subgraph may be a subgraph containing all entities and relationships related to fire, and / or, the subgraph may be a subgraph containing all entities and relationships related to human falls.

[0139] In some embodiments, the training alarm model extracts key features in the following manner:

[0140] H_sub^(l+1)=sigma(sum(A_sub,ij*W_sub^(l)*H_sub^(l)(j)+

[0141] B_sub^(l)*H_sub^(l)(i),j belongs to Neighbors_sub(i)))

[0142] in:

[0143] H_sub^(l+1): represents the node features of the l+1th layer in the subgraph.

[0144] H_sub^(l)(j): represents the features of the neighbor node j in the lth layer of the subgraph.

[0145] A_sub,ij: The adjacency matrix coefficient between nodes i and j in the subgraph, reflecting the importance of the connection strength or relationship type between the two nodes.

[0146] W_sub^(l): The weight matrix of the subgraph-specific layer l, used to update node features.

[0147] B_sub^(l): bias term for the subgraph specific layer l, used to adjust the output of each layer;

[0148] Sigma: Activation function, used to introduce nonlinearity so that the model can capture complex patterns.

[0149] Step 270, detecting the occurrence probability of a specific event in the video data based on the trained alarm model;

[0150] In some embodiments, the video data may be real-time surveillance video of a shopping mall, hospital, or the like.

[0151] In some embodiments, the method of detecting the occurrence probability of a specific event in the video data is:

[0152] P(event|F)=softmax(sum(W_d*F_k+b,k in Features))

[0153] in:

[0154] P(event|F): the probability of an event occurring given a feature set F;

[0155] F_k: the kth feature in the feature set extracted from the graph;

[0156] W_d: weight matrix for event classification;

[0157] b: bias term, used to adjust the model output;

[0158] Softmax: A function that converts linear output into a probability distribution.

[0159] In some embodiments, the feature set F can be extracted in the process of extracting global features in step 250 and / or extracting key features in step 260, and the weight matrix W_d and the bias term b are learned during the training process to optimally map the features to event probabilities.

[0160] Step 280, determining alarm information based on the probability of occurrence of a specific event and a dynamic threshold set;

[0161] In some embodiments, the dynamic threshold set refers to a set of thresholds used to determine whether a specific event occurs.

[0162] In some embodiments, the alarm information can be confirmed by comparing the probability of occurrence of a specific event with a threshold in a dynamic threshold set. For example, the probability of occurrence of a specific event is compared with a threshold in a dynamic threshold set, and an alarm information is generated when one or more data exceeds a threshold in the dynamic threshold set.

[0163] In some embodiments, the dynamic threshold can be automatically adjusted based on real-time environmental data (such as light intensity, scene changes, etc.).

[0164] In some embodiments, the dynamic threshold is set as follows:

[0165] threshold=base_threshold-(environment_factor*adjustment_rate)

[0166] Among them, threshold is a preset probability threshold, base_threshold is a basic threshold, environment_factor represents a value obtained from an environment sensor (such as light level), and adjustment_rate is an adjustment coefficient.

[0167] The reason is that at night or in bad weather conditions, the threshold may need to be lowered to increase sensitivity to events. Different environments have different corresponding thresholds, which greatly reduces false alarms caused by mismatches between thresholds and scenarios.

[0168] In some embodiments, after the alarm information is determined, the alarm information can be sent to relevant departments and / or the fire extinguishing system can be activated, thereby greatly improving the speed and efficiency of responding to incidents, reducing reliance on manual monitoring, and enhancing safety performance and disaster prevention capabilities.

[0169] This specification provides a video event alarm system based on a hierarchical knowledge graph, which is used to execute the aforementioned video event alarm method based on a hierarchical knowledge graph.

[0170] This specification provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the aforementioned video event alarm method based on a hierarchical knowledge graph.

[0171] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0172] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0173] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A video event alarm method based on hierarchical knowledge graph, characterized in that: include: Build a data set and obtain video sequences; Based on the video sequence, image frames are acquired; Based on each image frame, key frames are marked; Based on key frames, construct a hierarchical knowledge graph; Based on the hierarchical knowledge graph, the alarm model is trained to extract global features; Based on the hierarchical knowledge graph, sub-graphs are selected to train the alarm model to extract key features; Detect the probability of occurrence of specific events in video data based on the trained alarm model; Based on the probability of occurrence of a specific event and a dynamic threshold set, the alarm information is determined.

2. A video event alarm method based on hierarchical knowledge graph according to claim 1, characterized in that: The data set is a video data set including fire and / or personnel falling events; The video sequence is a sequence of image frames in the video data obtained in time order.

3. A video event alarm method based on hierarchical knowledge graph according to claim 2, characterized in that: The way to get the image frame is: I(t)=extract_frame(Video,t); Where I(t) is the image frame at time point t, Video is the entire video sequence, and t is the time index; The keyframes are marked as follows: K(t)=is_key_frame(I(t)); Among them, K(t) is a binary value indicating whether frame I(t) is a key frame.

4. The video event alarm method based on hierarchical knowledge graph according to claim 1 is characterized in that: The entity definitions in the hierarchical knowledge graph include: flame, personnel and environment objects; The environmental objects include furniture, doors and windows; The relationship definition between entities in the hierarchical knowledge graph includes: Adjacent means that two entities are close in space; Overlay: One entity visually covers another entity; In line of sight: One entity is visible from the line of sight of another entity; The entity is defined as follows: v_i = create_node(o_i); Among them, o_i is the object identified from the video frame, and vi_i is the node representing the object in the graph; The relationship is defined as follows: e_ij=(v_i,v_j,type); Among them, e_ij is the edge connecting nodes v_i and v_j, and type is the type of edge determined according to their spatial relationship.

5. The video event alarm method based on hierarchical knowledge graph according to claim 1 is characterized in that: The alarm model is a graph convolutional network GCN model; The global feature is a comprehensive feature that can represent all nodes in the graph and their relationships; The method of extracting global features from the training alarm model is as follows: H^(l+1)=sigma(sum(A_ij*W^(l)*H^(l)(j)+B^(l)*H^(l)(i),j belongs toNeighbors(i))); in: H^(l+1): represents the node features of the l+1th layer; H^(l)(j): represents the characteristics of neighbor node j in layer l; A_ij: the adjacency matrix coefficient between nodes i and j, reflecting the importance of the connection strength or relationship type between the two nodes; W^(l): weight matrix of the lth layer, used to update node features; B^(l): bias term of the lth layer, used to adjust the output of each layer; Sigma: Activation function, used to introduce nonlinearity so that the model can capture complex patterns.

6. A video event alarm method based on hierarchical knowledge graph according to claim 5, characterized in that: The subgraph is a local graph structure extracted for a specific event or object in the video content, and the subgraph includes a part of nodes and edges in the original graph; The key features are comprehensive features of nodes related to a specific event and their interrelationships; The method of selecting sub-graphs to train the alarm model and extract key features is: H_sub^(l+1)=sigma(sum(A_sub,ij*W_sub^(l)*H_sub^(l)(j)+B_sub^(l)*H_sub^(l)(i),j belongs to Neighbors_sub(i))) in: H_sub^(l+1): represents the node features of the l+1th layer in the subgraph; H_sub^(l)(j): represents the characteristics of the neighbor node j in the lth layer of the subgraph; A_sub,ij: the adjacency matrix coefficient between nodes i and j in the subgraph, reflecting the connection strength or the importance of the relationship type between the two nodes; W_sub^(l): weight matrix of the subgraph-specific layer l, used to update node features; B_sub^(l): bias term for the subgraph specific layer l, used to adjust the output of each layer; Sigma: activation function, used to introduce nonlinearity so that the model can capture complex patterns; The specific events include: fire events and fall events.

7. The video event alarm method based on hierarchical knowledge graph according to claim 1 is characterized in that: The method of detecting the occurrence probability of a specific event in the video data is: P(event|F)=softmax(sum(W_d*F_k+b,k in Features)) in: P(event|F): the probability of an event occurring given a feature set F; F_k: the kth feature in the feature set extracted from the graph; W_d: weight matrix for event classification; b: bias term, used to adjust the model output; Softmax: A function that converts linear output into a probability distribution.

8. The video event alarm method based on hierarchical knowledge graph according to claim 1 is characterized in that: The dynamic threshold set is a threshold set used to determine whether a specific event occurs; The alarm information is confirmed by comparing the probability of occurrence of a specific event with a dynamic threshold concentration threshold; The dynamic threshold is automatically adjusted based on real-time environmental data; The dynamic threshold is set as follows: threshold=base_threshold-(environment_factor*adjustment_rate), wherein threshold is a preset probability threshold, base_threshold is a basic threshold, environment_factor represents a value obtained from an environmental sensor, and adjustment_rate is an adjustment coefficient.

9. A video event warning system based on hierarchical knowledge graph, characterized in that: Used to execute the video event alarm method based on hierarchical knowledge graph as described in any one of claims 1-8.

10. A computer-readable storage medium storing computer instructions, characterized in that: After the computer reads the computer instructions in the storage medium, the computer executes the video event alarm method based on the hierarchical knowledge graph as described in any one of claims 1 to 8.

Images