Edge early warning method based on visual identification and semantic fusion

Abstract

The invention relates to the field of production safety management and artificial intelligence, and discloses an edge early warning method based on visual identification and semantic fusion, which is composed of the following modules: a data acquisition module, which is used for collecting field real-time videos and images, the visual identification module is constructed based on a PyTorch deep learning framework and is used for receiving the video and image information transmitted by the data acquisition module; and the semantic fusion module is used for receiving the target boundary position and the category label list output by the visual identification module and carrying out cross-model data fusion and semantic understanding. According to the method, two mechanisms of rule-based deterministic analysis and model-based anomaly detection are fused, the rule base can accurately identify typical violation and dangerous behaviors, the anomaly detection can capture unexpected anomaly modes, the limitation of a single method is effectively solved, missing detection and misjudgment are reduced, and the false alarm rate of the system is greatly reduced.

Description

Technical Field

[0001] This invention relates to the fields of production safety management and artificial intelligence technology, specifically to an edge warning method based on visual recognition and semantic fusion. Background Technology

[0002] In high-risk industries such as mining, power, construction, and petrochemicals, video surveillance systems have become the primary means of supervision. However, due to the large number of monitoring points and limited display resources on the central control screen, relying solely on manual observation is inefficient and prone to overlooking critical risk behaviors. Therefore, the industry has widely begun to introduce artificial intelligence recognition algorithms based on computer vision to perform target detection and behavior analysis on personnel, equipment, and key areas. The "Edge Early Warning Method Based on Decoupled Visual Recognition and Semantic Fusion" proposed in this invention addresses these shortcomings of existing visual recognition systems in terms of behavioral understanding depth, semantic output structuring capabilities, and edge deployment adaptability. It systematically improves and integrates these technologies to achieve a risk identification and early warning mechanism with low resource consumption, high semantic expression capabilities, strong interpretability, and high scene adaptability.

[0003] Currently, object detection models, represented by the YOLO series, are widely used in industrial video analytics due to their lightweight structure and high static recognition accuracy. These models are suitable for identifying static objects such as people, vehicles, and equipment, but their core is based on single-frame image inference, lacking the ability to model behavioral sequences. In complex scenarios, such as unauthorized entry, occlusion, touching equipment beyond boundaries, and falls, traditional object detection models often fail to capture the complete behavioral chain, resulting in high false positive and false negative rates. Meanwhile, in recent years, large-scale models such as VisionTransformer (ViT), SegmentAnythingModel (SAM), and large-parameter pose recognition models (such as OpenPose) have demonstrated stronger semantic understanding capabilities, enabling fine-grained region analysis and contextual modeling. However, these models typically require significant computing power, have long inference times, and are difficult to deploy directly on edge computing devices used in high-risk environments, especially in harsh industrial environments such as network outages, dust, and high temperatures, failing to meet the requirements for real-time performance, stability, and response speed. Furthermore, existing systems generally output borders and category labels, lacking a unified structured semantic representation, making it difficult to form a complete expression of "events" and lacking the ability to logically link with backend knowledge graphs, rule engines, or early warning systems. Current algorithms cannot effectively answer the question "who, when, where, and what did they do?", nor can they explain early warning behavior from multiple perspectives (such as people, equipment, environment, and management), resulting in poor system interpretability, a single response strategy, and insufficient false alarm suppression capabilities. Therefore, there is an urgent need for a new method that can achieve: ① decoupled replacement and modular management of recognition models; ② unified semantic fusion and structural expression of multi-model recognition results; ③ lightweight deployment and real-time response under edge computing conditions; ④ output standardized "time-location-event" triples and "human-machine-object-law-environment-management" six elements to support downstream linkage of structured semantics. This invention proposes, based on this, to use a lightweight, decoupled visual recognition module for edge reasoning, quickly completing the recognition of behavior and entity elements in images or videos, and outputting standard structured data through a semantic fusion module. The backend system can use this structured semantics as input and hand it over to a large model or rule engine for complex logical judgment, thereby significantly improving recognition accuracy, enhancing interpretability, and effectively reducing the computing consumption and deployment cost of front-end hardware, realizing an efficient collaborative mechanism of "edge reasoning + cloud reasoning". Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an edge warning method based on visual recognition and semantic fusion, which solves the problem that traditional target detection models often fail to capture the complete behavioral chain, resulting in high false positive and false negative rates.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an edge warning system based on visual recognition and semantic fusion, comprising the following modules:

[0006] Data acquisition module: The data acquisition module collects real-time video and images of the site by setting up and installing cameras in high-risk work environments by staff, and transmits the collected video and images to the visual recognition module after preprocessing;

[0007] Visual recognition module: The visual recognition module is built on the PyTorch deep learning framework and consists of a variety of pluggable AI sub-models. It is used to receive video and image information transmitted from the data acquisition module, and to identify and analyze different elements in the video based on the AI ​​sub-models. At the same time, it uses mainstream detection networks to locate the target position, and uses a classification network to determine the target category. The identified target boundary position and category label list are then transmitted to the semantic fusion module.

[0008] Semantic fusion module: The semantic fusion module receives the target boundary location and category label list output from the visual recognition module, performs cross-model data fusion and semantic understanding, generates event description data for external output, and transmits the final generated event description to the risk reasoning and early warning module;

[0009] Risk Reasoning and Early Warning Module: The risk reasoning and early warning module receives event description data from the semantic fusion module, and uses fusion decision combined with rule reasoning and anomaly detection mechanisms for analysis and processing to arrive at the final early warning decision;

[0010] Edge Deployment and Interaction Module: The edge deployment and interaction module is connected to the risk reasoning and early warning module and is integrated and deployed on an edge computing device, including an industrial gateway or an embedded AI box. The edge deployment and interaction module works in conjunction with the risk reasoning and early warning module. When the risk reasoning and early warning module issues an early warning, the edge deployment and interaction module immediately triggers an on-site audible and visual alarm and sends the alarm information to the cloud management platform or the manager's mobile APP via 5G / Ethernet.

[0011] Preferably, in the data acquisition module, the preprocessing specifically involves: using edge technology devices to cache, adjust the resolution, and encode and decode the image and video information acquired by the data acquisition module, thereby providing a foundation for the visual recognition module.

[0012] Preferably, in the visual recognition module, the recognition and analysis of different elements in the video frame based on the AI ​​sub-model specifically includes: a target detection and classification sub-model, used to identify objects or targets in the scene and classify their attributes. The objects or targets include: personnel, protective equipment, mechanical equipment, vehicles, and fire sources.

[0013] Preferably, the mainstream detection networks in the visual recognition module include: YOLO series and SSD.

[0014] Preferably, in the visual recognition module, the classification network determines the target category, including: personnel category and equipment type.

[0015] Preferably, the visual recognition module includes multiple pluggable AI sub-models, including a face recognition sub-model and a pose recognition sub-model.

[0016] The face recognition sub-model is used to detect and recognize faces from videos, match personnel identities, locate face positions using face detection algorithms, call a pre-trained face feature extraction network, map the face into an identity vector, compare it with the enterprise personnel database to identify identity information, and finally output personnel ID or name, as well as recognition confidence.

[0017] The posture recognition sub-model is used to analyze the skeletal key points and movement postures of on-site personnel. It extracts the skeletal key points of personnel based on the human posture estimation algorithm and further judges the person's movement state, including standing, walking, climbing, and falling. Finally, it outputs the person's posture description or movement category and corresponding confidence level.

[0018] Preferably, in the visual recognition module, the various pluggable AI sub-models are all designed in a decoupled manner and communicate through a predefined data interface.

[0019] Preferably, in the semantic fusion module, cross-model data fusion and semantic understanding are performed to generate an externally output event description, specifically as follows:

[0020] First, perform temporal and spatial labeling: attach a timestamp and a pre-configured location identifier to each frame or detected event to form a basic "time-location" index;

[0021] Next, event element extraction is performed: Based on the visual recognition results, the events occurring in the current scene are analyzed and defined as behaviors or states with safety significance, including: "personnel entering a restricted area", "not wearing a safety helmet", "person falling", and "abnormal equipment movement".

[0022] Finally, the semantic fusion module associates the above-mentioned elements with the event and organizes them into a structured semantic record. When needed, this semantic record can be connected to the industry knowledge graph: by querying the relevant background knowledge in the knowledge graph through the personnel ID and equipment ID in the event, the background knowledge includes personnel training records, equipment maintenance history, and relevant safety procedures, further enriching the meaning of the event.

[0023] Preferably, in the risk reasoning and early warning module, the rule-compliant reasoning includes:

[0024] A series of predefined safety rules and risk criteria are established. These rules are based on expert experience and safety regulations and are applicable to common hazardous scenarios. The rule engine matches event semantic data with the rule base, comparing whether the six elements contained in the event meet the conditions of any risk rule. If a match is successful, a corresponding risk alert is generated, indicating the risk type and recommended measures, including:

[0025] If personnel approach a running crane without wearing reflective clothing, it is considered a high-risk event.

[0026] If unauthorized personnel enter the danger zone, a safety alarm will be triggered;

[0027] Based on the knowledge graph, if a device is found to be overdue for maintenance, a maintenance alert will be triggered once the device is detected to be running.

[0028] The anomaly detection includes: for novel anomalies that cannot be covered by fixed rules, a machine learning model is introduced to detect abnormal behavior. This model can use unsupervised or semi-supervised learning to train on historical data on site, establish normal patterns of personnel activities and equipment operation. Once the data distribution of the current scene deviates from the normal pattern, including the appearance of never-before-seen action combinations or equipment states, the model outputs an anomaly score. When the anomaly score exceeds the threshold and the event is not covered by any existing rules, it is regarded as an unknown anomaly that triggers an alarm, thereby discovering some potential risks that the rule base did not foresee.

[0029] The fusion decision includes: combining rule reasoning results with anomaly detection results through system setting strategies to improve alarm accuracy. Specifically, the system setting strategies are as follows:

[0030] When both rules and anomaly models point to risk, increase the alert level and reduce the possibility of false alarms.

[0031] When the rule determines a risk but the anomaly model determines it is normal, it may be a false alarm. The alarm priority can be reduced or manual confirmation can be required.

[0032] Conversely, when the anomaly model detects an anomaly but there is no corresponding rule, it is marked as a new type of anomaly to alert operations personnel.

[0033] Through this fusion strategy, the system effectively reduces the false alarm rate, avoids frequent disturbances caused by misjudgments from a single algorithm, and ensures that truly urgent risks receive a rapid response.

[0034] A method for using an edge warning system based on visual recognition and semantic fusion includes the following steps:

[0035] S1. Deploy the data acquisition module to collect and preprocess data: Deploy cameras at high-risk work sites to collect real-time video and images, which are then cached, have their resolution adjusted, and are encoded and decoded by edge devices before being transmitted to the visual recognition module.

[0036] S2. Visual Recognition Module Analyzes Elements: The visual recognition module receives video and image data transmitted by the data acquisition module in step S1. Based on the PyTorch framework, it identifies people, equipment, and action elements in the video through pluggable AI sub-models for target detection and classification, face recognition, and posture recognition, and outputs target location, category, identity, and posture information.

[0037] S3. Semantic fusion generates structured events: The semantic fusion module receives the parsed elements generated by the visual recognition module in step S2, and marks the time and location of the parsed elements. Then, it extracts the "unauthorized entry" security event, associates it with the six elements of "person, machine, object, law, environment, and management", and forms a semi-structured event record.

[0038] S4. Risk reasoning leads to early warning decision: The risk reasoning and early warning module receives the semi-structured event records given by the semantic fusion module in step S3, and then analyzes the event data by combining rule reasoning and anomaly detection through fusion strategy to generate the final early warning decision.

[0039] S5. Edge module executes early warning response: The edge deployment and interaction module triggers on-site audible and visual alarms based on the final early warning decision information obtained in step S4, and sends the early warning information to the cloud platform or the administrator's mobile APP via 5G / Ethernet.

[0040] This invention provides an edge warning method based on visual recognition and semantic fusion. It possesses the following features:

[0041] Beneficial effects:

[0042] 1. This invention integrates rule-based deterministic analysis and model-based anomaly detection. The rule base can accurately identify typical violations and dangerous behaviors, while anomaly detection can capture unexpected abnormal patterns. The two complement each other, effectively solving the limitations of single methods, reducing missed detections and false alarms, significantly reducing the false alarm rate of the system, and making alarms more reliable. Compared with systems that rely solely on traditional video surveillance or simple AI detection, the accuracy and robustness of early warnings are significantly improved.

[0043] 2. This invention adopts a decoupled modular architecture to design a visual perception system, making each sub-model independent and replaceable. Compared with the existing approach of coupling all tasks to a single model, it is convenient to add or remove recognition functions or upgrade models according to the needs of the scenario without reconstructing the entire system. This gives it good flexibility and maintainability, and it can continuously adapt to new requirements and the development of new technologies.

[0044] 3. This invention integrates multi-source visual information such as target detection, classification, face and posture recognition into a unified semantic expression, achieving a comprehensive understanding of the situation on site. It goes beyond single-point detection and constructs the concept of "event" by associating with the identity of people, behavior, and equipment status, making risk assessment more accurate. It also creates conditions for the system to connect with industry knowledge graphs and perform more advanced reasoning and analysis.

[0045] 4. This invention proposes a semi-structured event representation method in the form of "triples + six elements", which converts complex video detection results into standardized event records, including time, location, basic event information, and details of six aspects: people, machines, objects, methods, environment, and management. This representation is not only easy for computer processing, but also carries rich semantic context. It can be directly used to construct event nodes of a safety production knowledge graph or to interface with existing knowledge bases, supporting more interpretable and traceable safety management decisions.

[0046] 5. This invention is designed specifically for edge devices. Through model lightweighting and optimized scheduling, it can run stably on devices with limited computing power and power. Compared with cloud solutions, it can complete analysis and decision-making in real time on-site, with no network latency, strong timeliness, and a large amount of video data does not need to be uploaded to the cloud, saving bandwidth and protecting data privacy. It can be deployed in environments with poor network conditions to achieve local intelligence. The instant early warning capability helps to intervene in dangerous situations at the first time and significantly improves emergency response efficiency. Attached Figure Description

[0047] Figure 1 This is a system flowchart of the present invention;

[0048] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0049] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Please see the appendix Figure 1 - Appendix Figure 2 This invention provides an edge warning system based on visual recognition and semantic fusion, which consists of the following modules:

[0051] Data acquisition module: The data acquisition module collects real-time video and images of the site by setting up and installing cameras at the site of high-risk work environment by staff. The collected video and images are pre-processed and then transmitted to the visual recognition module. It should be noted that the installation position and angle of the cameras are optimized to ensure full coverage of key work areas in the high-risk work environment.

[0052] The preprocessing specifically involves using edge technology devices to cache, adjust the resolution of, and encode and decode the image and video information acquired by the data acquisition module, thereby providing a foundation for the visual recognition module.

[0053] Visual recognition module: The visual recognition module is built on the PyTorch deep learning framework and consists of a variety of pluggable AI sub-models. It is used to receive video and image information transmitted from the data acquisition module, and to identify and analyze different elements in the video based on the AI ​​sub-models. At the same time, it uses mainstream detection networks to locate the target position, and uses a classification network to determine the target category. The identified target boundary position and category label list are then transmitted to the semantic fusion module.

[0054] The specific steps of identifying and analyzing different elements in the video frame based on the AI ​​sub-model are as follows:

[0055] The target detection and classification sub-model is used to identify objects or targets in a scene and classify their attributes. The objects or targets include: personnel, protective equipment, mechanical equipment, vehicles, and fire sources.

[0056] The mainstream detection networks include: YOLO series, SSD;

[0057] The classification network determines the target category, including: personnel category and equipment type;

[0058] The various pluggable AI sub-models include: a face recognition sub-model and a pose recognition sub-model.

[0059] The face recognition sub-model is used to detect and identify faces in videos and match personnel identities. It employs a face detection algorithm to locate the face, calls a pre-trained face feature extraction network to map the face into an identity vector, compares it with the company's personnel database to identify identity information, and finally outputs the personnel ID or name, along with the recognition confidence score.

[0060] The posture recognition sub-model is used to analyze the skeletal key points and movement postures of on-site personnel. Based on the human posture estimation algorithm, it extracts the skeletal key points of personnel and further judges the person's movement state, including standing, walking, climbing, and falling. Finally, it outputs the person's posture description or movement category and corresponding confidence level.

[0061] Furthermore, all of the various pluggable AI sub-models adopt a decoupled design and communicate through predefined data interfaces. In this way, when it is necessary to replace one of the AI ​​sub-models, it is only necessary to ensure that the new model follows the same interface specification, without changing the entire system architecture. This modular design makes it easy to customize the optimal model combination according to the application scenario, while adapting to the computing power limitations of edge devices.

[0062] Semantic fusion module: This module receives the target boundary location and category label list output from the visual recognition module, performs cross-model data fusion and semantic understanding, generates event description data for external output, and transmits the final generated event description to the risk reasoning and early warning module.

[0063] The specific steps for performing cross-model data fusion and semantic understanding to generate event descriptions for external output are as follows:

[0064] First, perform temporal and spatial labeling: add a timestamp and a pre-configured location identifier (camera ID corresponds to a specific area) to each frame or detected event to form a basic "time-location" index;

[0065] Next, event element extraction is performed: Based on the visual recognition results, the events occurring in the current scene are analyzed and defined as behaviors or states with safety significance, including: "personnel entering a restricted area", "not wearing a safety helmet", "person falling", and "abnormal equipment movement".

[0066] Finally, the semantic fusion module associates the above-mentioned elements with the event and organizes them into a structured semantic record. When needed, this semantic record can be connected to the industry knowledge graph: by querying the relevant background knowledge in the knowledge graph through the personnel ID and equipment ID in the event, the background knowledge includes personnel training records, equipment maintenance history, and relevant safety procedures, further enriching the meaning of the event.

[0067] Example:

[0068] An event record can be represented as: Time = T1, Location = L1, Event = "Personnel entered the construction area without wearing a safety helmet", Related elements: Person = Zhang San (worker, not wearing a safety helmet), Machine = Excavator (operating), Material = Several building materials, Law = Violation of Article X of the Water Safety Regulations, Environment = Dim site lighting, Management = Area supervisor not on duty. After the above processing, this module outputs semi-structured event semantic data, which contains machine-readable structured fields while retaining key on-site information, facilitating subsequent risk assessment and storage.

[0069] Risk Reasoning and Early Warning Module: This module receives event description data from the semantic fusion module and analyzes it using a combination of fusion decision-making, rule-based reasoning, and anomaly detection mechanisms to arrive at a final early warning decision.

[0070] The rule-compliant reasoning includes:

[0071] A series of predefined safety rules and risk criteria are established. These rules are based on expert experience and safety regulations and are applicable to common hazardous scenarios. The rule engine matches event semantic data with the rule base, comparing whether the six elements contained in the event meet the conditions of any risk rule. If a match is successful, a corresponding risk alert is generated, indicating the risk type and recommended measures, including:

[0072] 1. If personnel approach a running crane without wearing reflective clothing, it is considered a high-risk event;

[0073] 2. If unauthorized personnel enter the danger zone, a safety alarm will be triggered;

[0074] 3. Based on the knowledge graph, if a certain piece of equipment is found to be overdue for maintenance, a maintenance warning will be triggered once the equipment is detected to be running.

[0075] The anomaly detection includes: for novel anomalies that cannot be covered by fixed rules, a machine learning model is introduced to detect abnormal behavior. This model can use unsupervised or semi-supervised learning to train on historical data on site and establish normal patterns of personnel activities and equipment operation. Once the data distribution of the current scene deviates from the normal pattern, including the appearance of never-before-seen action combinations or equipment states, the model outputs an anomaly score. When the anomaly score exceeds the threshold and the event is not covered by any existing rules, it is regarded as an unknown anomaly and an alarm is triggered. This can discover some potential risks that the rule base has not foreseen.

[0076] Example:

[0077] The system can also issue a warning if a person's behavior is abnormal (possibly due to physical discomfort or an accident);

[0078] The fusion decision includes: combining rule reasoning results with anomaly detection results through system setting strategies to improve alarm accuracy. Specifically, the system setting strategies are as follows:

[0079] When both rules and anomaly models point to risk, increase the alert level and reduce the possibility of false alarms.

[0080] When the rule determines a risk but the anomaly model determines it is normal, it may be a false alarm. The alarm priority can be reduced or manual confirmation can be required.

[0081] Conversely, when the anomaly model detects an anomaly but there is no corresponding rule, it is marked as a new type of anomaly to alert operations personnel.

[0082] Through this fusion strategy, the system effectively reduces the false alarm rate, avoids frequent disturbances caused by misjudgments by a single algorithm, and ensures that truly urgent risks receive a rapid response.

[0083] Edge Deployment and Interaction Module: The edge deployment and interaction module is connected to the risk reasoning and early warning module and is integrated and deployed on an edge computing device, including an industrial gateway or an embedded AI box. The edge deployment and interaction module works with the risk reasoning and early warning module to immediately trigger an on-site audible and visual alarm when the risk reasoning and early warning module issues an early warning, and sends the alarm information to the cloud management platform or the manager's mobile APP via 5G / Ethernet.

[0084] A method for using an edge warning system based on visual recognition and semantic fusion includes the following steps:

[0085] S1. Deploy the data acquisition module to collect and preprocess data: Deploy cameras at high-risk work sites to collect real-time video and images, which are then cached, have their resolution adjusted, and are encoded and decoded by edge devices before being transmitted to the visual recognition module.

[0086] S2. Visual Recognition Module Analyzes Elements: The visual recognition module receives video and image data transmitted by the data acquisition module in step S1. Based on the PyTorch framework, it uses pluggable AI sub-models such as target detection and classification, face recognition, and posture recognition to identify elements such as people, equipment, and actions in the video, and outputs target location, category, identity, and posture information.

[0087] S3. Semantic fusion generates structured events: The semantic fusion module receives the parsed elements generated by the visual recognition module in step S2, and labels the parsed elements with time and location. It then extracts security events such as "unauthorized entry" and associates them with the six elements of "people, machines, objects, laws, environment, and management" to form a semi-structured event record.

[0088] S4. Risk reasoning leads to early warning decision: The risk reasoning and early warning module receives the semi-structured event records given by the semantic fusion module in step S3, and then analyzes the event data by combining rule reasoning and anomaly detection through fusion strategy to generate the final early warning decision.

[0089] S5. Edge module executes early warning response: The edge deployment and interaction module triggers on-site audible and visual alarms based on the final early warning decision information obtained in step S4, and sends the early warning information (including event details) to the cloud platform or the administrator's mobile APP via 5G / Ethernet.

[0090] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An edge warning system based on visual recognition and semantic fusion, characterized in that, It consists of the following modules: Data acquisition module: The data acquisition module collects real-time video and images of the site by setting up and installing cameras in high-risk work environments by staff, and transmits the collected video and images to the visual recognition module after preprocessing; Visual recognition module: The visual recognition module is built on the PyTorch deep learning framework and consists of a variety of pluggable AI sub-models. It is used to receive video and image information transmitted from the data acquisition module, and to identify and analyze different elements in the video based on the AI ​​sub-models. At the same time, it uses mainstream detection networks to locate the target position, and uses a classification network to determine the target category. The identified target boundary position and category label list are then transmitted to the semantic fusion module. Semantic fusion module: The semantic fusion module receives the target boundary location and category label list output from the visual recognition module, performs cross-model data fusion and semantic understanding, generates event description data for external output, and transmits the final generated event description to the risk reasoning and early warning module; Risk Reasoning and Early Warning Module: The risk reasoning and early warning module receives event description data from the semantic fusion module, and uses fusion decision combined with rule reasoning and anomaly detection mechanisms for analysis and processing to arrive at the final early warning decision; Edge Deployment and Interaction Module: The edge deployment and interaction module is connected to the risk reasoning and early warning module and is integrated and deployed on an edge computing device, including an industrial gateway or an embedded AI box. The edge deployment and interaction module works in conjunction with the risk reasoning and early warning module. When the risk reasoning and early warning module issues an early warning, the edge deployment and interaction module immediately triggers an on-site audible and visual alarm and sends the alarm information to the cloud management platform or the manager's mobile APP via 5G / Ethernet.

2. The edge warning system based on visual recognition and semantic fusion according to claim 1, characterized in that, In the data acquisition module, the preprocessing specifically involves using edge technology devices to cache, adjust the resolution, and encode and decode the image and video information acquired by the data acquisition module, thereby providing a foundation for the visual recognition module.

3. The edge warning system based on visual recognition and semantic fusion according to claim 1, characterized in that, In the visual recognition module, the identification and analysis of different elements in the video frame based on the AI ​​sub-model is specifically as follows: target detection and classification sub-model, which is used to identify objects or targets in the scene and classify their attributes. The objects or targets include: personnel, protective equipment, mechanical equipment, vehicles, and fire sources.

4. The edge warning system based on visual recognition and semantic fusion according to claim 1, characterized in that, The main detection networks used in the visual recognition module include the YOLO series and SSD.

5. The edge warning system based on visual recognition and semantic fusion according to claim 1, characterized in that, In the visual recognition module, the classification network determines the target category, including: personnel category and equipment type.

6. The edge warning system based on visual recognition and semantic fusion according to claim 1, characterized in that, The visual recognition module includes several pluggable AI sub-models, including a face recognition sub-model and a pose recognition sub-model. The face recognition sub-model is used to detect and recognize faces from videos, match personnel identities, locate face positions using face detection algorithms, call a pre-trained face feature extraction network, map the face into an identity vector, compare it with the enterprise personnel database to identify identity information, and finally output the personnel ID or name, as well as the recognition confidence level. The posture recognition sub-model is used to analyze the skeletal key points and movement postures of on-site personnel. It extracts the skeletal key points of personnel based on the human posture estimation algorithm and further judges the person's movement state, including standing, walking, climbing, and falling. Finally, it outputs the person's posture description or movement category and corresponding confidence level.

7. An edge warning system based on visual recognition and semantic fusion according to claim 1, characterized in that, In the visual recognition module, the various pluggable AI sub-models all adopt a decoupled design and communicate through predefined data interfaces.

8. An edge warning system based on visual recognition and semantic fusion according to claim 1, characterized in that, The semantic fusion module performs cross-model data fusion and semantic understanding to generate event descriptions for external output, specifically as follows: First, perform temporal and spatial labeling: attach a timestamp and a pre-configured location identifier to each frame or detected event to form a basic "time-location" index; Next, extract event elements: Analyze the events occurring in the current scene based on the visual recognition results, and define the events as behaviors or states with safety significance, including: "personnel entering a restricted area", "not wearing a safety helmet", "person falling", and "abnormal equipment movement"; Finally, the semantic fusion module associates the above-mentioned elements with the event and organizes them into a structured semantic record. When needed, this semantic record can be connected to the industry knowledge graph: by querying the relevant background knowledge in the knowledge graph through the personnel ID and equipment ID in the event, the background knowledge includes personnel training records, equipment maintenance history, and relevant safety procedures, further enriching the meaning of the event.

9. An edge warning system based on visual recognition and semantic fusion according to claim 1, characterized in that, The risk reasoning and early warning module includes rule-compliant reasoning: A series of predefined safety rules and risk criteria are established. These rules are based on expert experience and safety regulations and are applicable to common hazardous scenarios. The rule engine matches event semantic data with the rule base, comparing whether the six elements contained in the event meet the conditions of any risk rule. If a match is successful, a corresponding risk alert is generated, indicating the risk type and recommended measures, including: If personnel approach a running crane without wearing reflective clothing, it is considered a high-risk event. If unauthorized personnel enter the danger zone, a safety alarm will be triggered. Based on the knowledge graph, if a device is found to be overdue for maintenance, a maintenance alert will be triggered once the device is detected to be running. The anomaly detection includes: for novel anomalies that cannot be covered by fixed rules, a machine learning model is introduced to detect abnormal behavior. This model can use unsupervised or semi-supervised learning to train on historical data on site, establish normal patterns of personnel activities and equipment operation. Once the data distribution of the current scene deviates from the normal pattern, including the appearance of never-before-seen action combinations or equipment states, the model outputs an anomaly score. When the anomaly score exceeds the threshold and the event is not covered by any existing rules, it is regarded as an unknown anomaly that triggers an alarm, thereby discovering some potential risks that the rule base did not foresee. The fusion decision includes: combining rule-based reasoning results with anomaly detection results through system setting strategies to improve alarm accuracy. Specifically, the system setting strategies are as follows: When both rules and anomaly models point to risk, increase the alert level and reduce the possibility of false alarms. When the rule determines a risk but the anomaly model determines it is normal, it may be a false alarm. The alarm priority can be reduced or manual confirmation can be required. Conversely, when the anomaly model detects an anomaly but there is no corresponding rule, it is marked as a new type of anomaly to alert operations personnel. Through this fusion strategy, the system effectively reduces the false alarm rate, avoids frequent disturbances caused by misjudgments from a single algorithm, and ensures that truly urgent risks receive a rapid response.

10. A method for using an edge warning system based on visual recognition and semantic fusion according to any one of claims 1-9, characterized in that, Includes the following steps: S1. Deploy the data acquisition module to collect and preprocess data: Deploy cameras at high-risk work sites to collect real-time video and images, which are then cached, have their resolution adjusted, and are encoded and decoded by edge devices before being transmitted to the visual recognition module. S2. Visual Recognition Module Analyzes Elements: The visual recognition module receives video and image data transmitted by the data acquisition module in step S1. Based on the PyTorch framework, it identifies people, equipment, and action elements in the video through pluggable AI sub-models for target detection and classification, face recognition, and posture recognition, and outputs target location, category, identity, and posture information. S3. Semantic fusion generates structured events: The semantic fusion module receives the parsed elements generated by the visual recognition module in step S2, and labels the parsed elements with time and location. Then, it extracts the "unauthorized entry" security event and associates it with the six elements of "person, machine, object, law, environment, and management" to form a semi-structured event record. S4. Risk reasoning leads to early warning decision: The risk reasoning and early warning module receives the semi-structured event records given by the semantic fusion module in step S3, and then analyzes the event data by combining rule reasoning and anomaly detection through fusion strategy to generate the final early warning decision. S5. Edge module executes early warning response: The edge deployment and interaction module triggers on-site audible and visual alarms based on the final early warning decision information obtained in step S4, and sends the early warning information to the cloud platform or the administrator's mobile APP via 5G / Ethernet.

Images