An intelligent identification system and method for workers falling off edges based on dangerous sign reasoning

Abstract

The present invention discloses an intelligent identification system and method for workers falling near edges oriented to dangerous sign reasoning. The system extracts the features of key objects in the edge falling scene to clarify the spatial scope of each dangerous edge scene; performs structured knowledge combination according to subject class, attribute class, object class, position class, and behavior state class, and constructs a knowledge graph of dangerous signs in the edge area through a graph database; visually identifies the features of the dangerous area of ​​falling near the edge and the behavior state of the workers; and uses graph database reasoning to evaluate the safety status of the workers' edge operations based on the visual identification results. The present invention combines the workers' behavior state and utilizes the constructed knowledge rules of dangerous signs in the edge area to identify dangerous edge events based on intelligent visual detection. The invention has a high level of intelligence and is highly extensible and practical.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision semantic understanding, including deep learning technology and semantic reasoning technology, and in particular to an intelligent identification system and method for workers falling off edges for dangerous sign reasoning. Background Art

[0002] The construction industry is an important industry supporting social and economic development, and it is also a typical high-risk industry. Among the many types of safety accidents, falling from height is the most important type of accident, accounting for more than half of the accidents. In various high-altitude work scenarios, climbing, hanging, platform, and cross-operation are difficult, and experienced workers are usually selected and equipped with reliable safety equipment. The probability of a high-altitude fall accident is minimal. However, falling from edge operations, which are not difficult, has become the most common accident in all high-altitude operations. Construction workers generally do not wear safety belts during edge operations, but mainly rely on the construction workers' active awareness to control the risk of falling. It is precisely because of the over-reliance on the construction workers' active awareness that their unsafe behavior in the edge environment becomes the driving factor of safety accidents.

[0003] There are always signs of accidents. Heinrich's safety pyramid law shows that before a serious injury or death accident occurs, there may have been 29 minor injury accidents and 300 near misses. These 300 near misses do not mean that the vehicle is in a safe state, but that it is a lucky escape with a certain probability. This theory fully demonstrates the inseparable relationship between near misses and accidents. Therefore, it is urgent to identify and control near misses at the level of near misses, to warn of a large number of near misses, and to prevent near misses from evolving into safety accidents.

[0004] In recent years, computer vision has begun to be applied to construction safety management. It can automatically extract various types of target information from images, and recognize and predict behavioral actions based on temporal relationships and target semantic information. However, if we only rely on the superb ability of computer vision in the field of image perception and ignore the semantic relationship between perceived objects, a "semantic gap" will appear between the underlying features and high-level semantics, that is, the perception results will have a high false alarm rate. How to integrate semantic information under the premise of visual perception can enable machines to recognize the world in a regular way. Integrating computer vision technology with semantic reasoning for construction safety behavior perception and modeling the potential associations between multiple semantics will provide new means and perspectives for intelligent management of construction safety.

[0005] In summary, based on the above problems, it is urgent to design an intelligent edge fall danger reasoning method to improve the visualization basis for safety management and performance evaluation of construction sites. Summary of the invention

[0006] In view of the frequent occurrence of accidents of falling from construction edges, the purpose of the present invention is to provide an intelligent identification system and method for workers falling from construction edges oriented to near-miss reasoning, so as to prevent the evolution of near-misses into accidents.

[0007] An intelligent identification system for workers falling off edge oriented to dangerous sign reasoning, comprising a construction edge operation image acquisition unit, a semantic database model, an edge dangerous area extraction model, a worker behavior state extraction model and a semantic reasoning model cascaded;

[0008] The construction edge operation image acquisition unit is used to acquire construction edge operation images; and input the edge danger area extraction model and the worker behavior state extraction model respectively;

[0009] The edge danger zone extraction model includes an edge zone space range division module and an edge danger zone feature extraction module, wherein the edge zone space range division module is used to divide the edge zone space range of the input construction edge operation image, and the edge danger zone feature extraction module is used to extract the danger zone and the object in the danger zone;

[0010] The worker behavior state extraction model includes a construction worker attribute extraction module and a worker behavior feature extraction module; wherein the construction worker attribute extraction module extracts the attributes of the worker in the input construction edge work image; the worker behavior feature extraction module extracts the features of the input construction edge work image, and the features are the worker behavior states;

[0011] The semantic reasoning model includes a visual coordinate relationship mining module, a visual recognition result semanticization module and a database semantic reasoning module; the visual coordinate relationship mining module receives the results output by the worker behavior state extraction model and the edge danger zone extraction model, and outputs the visual bounding box coordinates of the danger zone, object, worker attribute and worker behavior state respectively. The intersection-and-union ratio between the visual bounding box coordinates is calculated, and the spatial position relationship between the danger zone, object, worker attribute and worker behavior state is obtained based on the intersection-and-union ratio; the visual recognition result semanticization module converts the image into a semantic expression based on the intersection-and-union ratio; and inputs the semantic expression into the database semantic reasoning module; the database semantic reasoning module performs danger sign reasoning on the semantic expression output by the visual recognition result semanticization module based on the knowledge rules of danger signs of edge falling in the semantic database model, and then determines whether a danger sign event occurs in the construction edge operation image;

[0012] The semantic database model is a knowledge graph composed of knowledge rule elements of dangerous signs of falling from an edge. The knowledge rules of dangerous signs of falling from an edge are based on the theory of cause of edge accidents and the theory of dangerous signs. The knowledge graph will serve as the basis for distinguishing dangerous sign events.

[0013] Furthermore, the knowledge graph is composed of subject class, attribute class, object class, location class and behavior status class, and semantic information elements are obtained by arranging and combining the subject class, attribute class, object class, location class and behavior status class.

[0014] Furthermore, the object extracted by the edge danger zone feature extraction module is a protective measure.

[0015] Furthermore, the worker attributes extracted by the construction worker attribute extraction module include no safety helmet and no safety belt, no safety helmet and safety belt, safety helmet and no safety belt, and safety helmet and safety belt;

[0016] Furthermore, the near-miss events include the following three categories:

[0017] The first category: “Workers” without safety helmets and safety belts / with safety helmets and safety belts / with safety helmets and safety belts / with safety helmets but without safety belts “stand / walk / lean / climb / squat” on “the edge of stairs / the side of stairs / the edge of grooves / the edge of roofs / the edge of balconies” without any protective measures;

[0018] The second category: “Workers” without safety helmets and safety belts / without safety helmets but with safety belts “standing / walking / leaning / climbing / sitting” on “the edge of stairs / stair sides / grooves / roof edges / balcony edges” with “protective measures”;

[0019] The third category: "Workers with helmets but no safety belts / with helmets and safety belts" "climbing" on "stair edges / stair sides / grooves / roof edges / balcony edges" with "protective measures";

[0020] The above three categories are all set as dangerous omen events in the knowledge rules of dangerous omen of falling from edge.

[0021] An intelligent identification method for workers falling near edges based on near-miss reasoning includes the following steps:

[0022] Step 1, collecting construction edge operation drawings;

[0023] Step 2: construct an edge danger zone extraction model, and use the constructed edge danger zone extraction model to extract the edge danger zone and objects in the danger zone from the construction edge operation image;

[0024] Step 3: Build a worker behavior state extraction model, and use the built worker behavior state extraction model to extract worker attributes and behavior states from the construction edge operation diagram;

[0025] Step 4: Based on the output results of the worker behavior state extraction model and the edge danger zone extraction model, the visual bounding box coordinates of the danger zone, object, worker attributes and worker behavior state are output, and the intersection-and-union ratio between the visual bounding box coordinates is calculated; based on the intersection-and-union ratio, the spatial position relationship between the danger zone, object, worker attributes and worker behavior state is obtained, the visual coordinate relationship mining is realized, and the visual recognition results are semanticized; the database semantic reasoning uses the Cypher language to reason about the semanticized recognition results, and takes the edge danger sign event knowledge graph as the standard to query whether the edge danger sign event appears in the result.

[0026] Further, the process of step 2 is as follows:

[0027] S1: Augment the construction edge area work images to construct the edge work original image dataset; use the labelme tool to label the edge danger area and protective measures, and generate a json file to complete the construction of the edge danger area labeling dataset. The edge work original image dataset and the edge danger area labeling dataset are collectively referred to as the edge danger area image dataset;

[0028] S2: Use Mask-RCNN to load the constructed edge danger zone image dataset to train the network model and complete the construction of the edge danger zone extraction model;

[0029] S3: Input the collected construction edge operation map into the trained model, and use the model to extract the edge danger area;

[0030] Further, the process of step 3 is as follows:

[0031] S1: Augment the edge work images to construct the edge work original image dataset; use the labelme tool to annotate the worker attributes and three-point skeleton formation, generate a json file to complete the construction of the worker attribute and behavior status annotation dataset; the edge work original image dataset and the worker attribute and behavior status annotation dataset are collectively referred to as the worker attribute and behavior status image dataset;

[0032] S2: Load the constructed worker attribute and behavior status image dataset to train the network model, add skeleton key point recognition before the original target detection Fast-RCNN framework, use Openpose's skeleton key point recognition network, use MobileNetV2 lightweight convolutional neural network to replace the original VGG-19 network, for the recognition of left and right ankles and sacrum, connect the key points to form a triangle; complete the construction of the worker behavior status extraction model;

[0033] S3: Input the collected construction edge work images into the trained model, and use the model to extract worker attributes and behavior status.

[0034] Furthermore, the workers' behavioral state extraction and classification adopts three-point skeleton connection shape extraction. The three-point skeleton includes left and right ankle bones and sacrum. The behavioral characteristics are extracted according to the triangle characteristics formed by the three-point skeleton connection line; the workers' behavioral state classification includes: standing, walking, leaning, climbing, and squatting. When the left and right ankle points almost coincide and are far away from the sacrum, they are extracted as standing state; when the two sides of the sacrum connection are nearly equal and much larger than the third side, and the plane where the triangle is located is perpendicular to the reference ground, it is extracted as walking state; when the two sides of the sacrum connection are nearly equal and much larger than the third side, and the plane where the triangle is located is not perpendicular to the reference ground, it is extracted as leaning state; when the two sides of the sacrum connection are greatly different, it is extracted as climbing state, and when the plane where the triangle is located is almost parallel to the reference ground, it is extracted as squatting state.

[0035] Beneficial effects of the present invention:

[0036] 1. The present invention can realize the recognition and reasoning of dangerous signs of construction edge operations. Compared with the existing technology, the recognition object is clearer, that is, perception is performed at the dangerous stage of the accident; the detection accuracy is higher, and at the visual human-like analysis level, the visual content is automatically parsed into a natural language description that conforms to human cognition, and the problem of high false alarm rate of visual technology alone is alleviated by distinguishing natural semantics. The detection range is wider, and the fusion perception of various elements in the edge scene is realized, which promotes the concept recognition of complex semantics at the construction site.

[0037] 2. The present invention combines the worker's behavior state to perform dangerous sign reasoning, perceives the behavior state through three-point skeleton formation, and includes the common behavior state of workers in edge work into the recognition model, thereby improving the detection accuracy and enriching the detection means.

[0038] 3. The present invention can be combined with the monitoring system of the construction site to realize the hardware development of the warning of the danger of falling from the edge. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flow chart of a method for intelligently identifying worker falls near edges based on near-miss reasoning provided by an embodiment of the present invention;

[0040] Figure 2 A knowledge graph composed of knowledge rule elements of danger signs of falling from an edge provided in an embodiment of the present invention;

[0041] Figure 3 A flow chart of edge danger zone extraction training provided by an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of edge danger zone extraction results provided by an embodiment of the present invention;

[0043] Figure 5A flowchart of worker behavior state extraction training provided by an embodiment of the present invention;

[0044] Figure 6 A schematic diagram of worker behavior status extraction results provided by an embodiment of the present invention;

[0045] Figure 7 7a and 7b are schematic diagrams of semantic reasoning results provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are only used to explain the present invention and are not intended to limit the present invention.

[0047] Figure 1 An intelligent identification system for workers falling off an edge and oriented to dangerous sign reasoning is provided in an embodiment of the present invention. The identification system is cascaded by a construction edge operation image acquisition unit, a semantic database model, an edge dangerous area extraction model, a worker behavior state extraction model and a semantic reasoning model.

[0048] The construction edge operation image acquisition unit is used to acquire construction edge operation images; and input them into the edge danger area extraction model and the worker behavior state extraction model respectively.

[0049] The edge danger zone extraction model and the worker behavior state extraction model respectively use feature extraction networks to extract features from the input construction edge operation images. More specifically, the edge danger zone extraction model includes an edge zone space range division module and an edge danger zone feature extraction module. The edge zone space range division module is used to divide the edge zone space range of the input construction edge operation image, that is, to divide the dangerous area and the non-dangerous area; the edge danger zone feature extraction module is used to extract the dangerous area and the object in the dangerous area, which is the protective measure.

[0050] The worker behavior state extraction model includes a construction worker attribute extraction module and a worker behavior feature extraction module; the construction worker attribute extraction module extracts the worker attributes from the input construction edge work image, and sets the worker attributes to include no safety helmet and no safety belt, no safety helmet with safety belt, with safety helmet but no safety belt, and with safety helmet and safety belt; the worker behavior feature extraction module extracts features from the input construction edge work image, and the features are the worker behavior states.

[0051] The semantic reasoning model includes a visual coordinate relationship mining module, a visual recognition result semanticization module, and a database semantic reasoning module. The visual coordinate relationship mining module receives the output results of the worker behavior state extraction model and the edge danger zone extraction model, and outputs the visual bounding box coordinates of the danger zone, object, worker attribute, and worker behavior state respectively. The intersection over union (IOU) between the visual bounding box coordinates is calculated; based on the IOU, the spatial position relationship between the danger zone, object, worker attribute, and worker behavior state is obtained.

[0052] The semanticization module of visual recognition results converts it into semantic expression according to the intersection-union ratio; and inputs the semantic expression into the database semantic reasoning module;

[0053] The database semantic reasoning module performs dangerous sign reasoning on the semantic expression output by the visual recognition result semanticization module according to the knowledge rules of dangerous sign of edge falling in the semantic database model, and then determines whether a dangerous sign event occurs in the construction edge operation image.

[0054] The semantic database model is a knowledge graph composed of knowledge rule elements of near-miss hazards. The knowledge rules of near-miss hazards are based on the near-miss accident causation theory and the near-miss theory. The knowledge graph will serve as the basis for distinguishing near-miss events.

[0055] The knowledge graph composed of the knowledge rule elements of edge fall danger signs is as follows: Figure 2 As shown in the figure, the knowledge graph is composed of subject class (i.e. worker), attribute class, object class, location class and behavior state class, and semantic information elements are obtained by permuting and combining subject class, attribute class, object class, location class and behavior state class. The constructed knowledge graph can be expressed through the neo4j graph database.

[0056] The subject class: Worker,

[0057] The attribute categories are: no safety helmet and no safety belt, no safety helmet with safety belt, with safety helmet but no safety belt, with safety helmet and safety belt.

[0058] Object type: With protective measures, without protective measures

[0059] The behavior status categories include: standing, walking, leaning, climbing, and squatting.

[0060] The location categories include: non-adjacent edges, floor adjacent edges, stair side edges, gutter adjacent edges, roof facing edges and balcony adjacent edges.

[0061] Based on the above information, the neo4j graph database can express a variety of semantic information, for example, a worker without a helmet or a safety belt is walking on the edge of a floor without a guardrail.

[0062] In this embodiment, near-miss events include the following three categories:

[0063] The first category: "(Workers) without safety helmets and safety belts / with safety helmets and safety belts / with safety helmets and safety belts / with safety helmets but without safety belts) on (stairs edge / stairs side / groove edge / roof edge / balcony edge) (standing / walking / leaning / climbing / squatting) without protective measures";

[0064] The second category: "(Workers) without safety helmets and safety belts (without safety helmets and safety belts) on (the edge of stairs / the side of stairs / the edge of grooves / the edge of roofs / the edge of balconies) (standing / walking / leaning / climbing / squatting)";

[0065] The third category: "(Workers) with (safety helmets but no safety belts / safety helmets and safety belts) (climbing) on ​​(the edge of stairs / the side of stairs / the edge of the groove / the edge of the roof / the edge of the balcony) (with protective measures)"

[0066] The above three categories are all set as dangerous omen events in the knowledge rules of dangerous omen of falling from edge.

[0067] Based on the above-mentioned intelligent identification system for workers falling near edges based on near-miss reasoning, the present application also proposes an intelligent identification method for workers falling near edges based on near-miss reasoning, including the following steps:

[0068] Step 1: Collect construction edge operation images. In the embodiment of the present invention, construction edge operation images can be obtained through various methods such as network retrieval, monitoring collection and field collection, and the data quality of the obtained construction edge operation images can be enhanced by using uniform light processing and denoising processing.

[0069] Step 2, Figure 3 The flowchart of building a dangerous edge region extraction model and performing dangerous edge region extraction training provided in the embodiment of the present invention, the detailed steps are as follows:

[0070] S1: Constructing a dataset of edge hazard areas

[0071] The edge work images are augmented to construct the edge work original image dataset; the labelme tool is used to annotate the edge danger area and protective measures, and a json file is generated to complete the construction of the edge danger area annotated dataset. The edge work original image dataset and the edge danger area annotated dataset are collectively referred to as the edge danger area image dataset.

[0072] S2: Selection and training of edge danger zone extraction model

[0073] Use Mask-RCNN to load the constructed edge danger zone image dataset to train the network model. Divide the edge danger zone image dataset into a test set and a validation set at a ratio of 9:1. After completing a batch of training, verify the training effect of the validation set. If the effect is not good, adjust the model parameters for retraining. Before training, set the batch size (Batch-size) to 32 and the training epoch (Epoch) to 150. Transfer learning training method (Transfer Learning) is used during training. The network weights of the preliminary training model are used as the initial weights of the classification network to reduce training time and memory consumption and improve the classification accuracy of the algorithm on small data sets; set early stopping (Early Stopping) for training. When the loss value converges, it means that the training is complete, that is, stop training to avoid overfitting. Save the best training weights.

[0074] S3: Input the collected construction edge operation map into the trained model and use the model to extract the edge danger area, such as Figure 4 Dangerous areas of falling edges and protective measures are shown.

[0075] Step 3, Figure 5 The flowchart of building a worker behavior state extraction model and worker behavior state extraction training provided by the embodiment of the present invention; the detailed steps are as follows:

[0076] S1: Augment the edge work images to construct the edge work original image dataset; use the labelme tool to annotate the worker attributes and three-point skeleton, generate a json file to complete the construction of the worker attribute and behavior state annotation dataset. The edge work original image dataset and the worker attribute and behavior state annotation dataset are collectively referred to as the worker attribute and behavior state image dataset;

[0077] The worker attribute extraction and classification targets workers equipped with different safety equipment, and the framing results include the following categories: workers without safety helmets and safety belts, workers without safety helmets but with safety belts, workers with safety helmets but without safety belts, and workers with safety helmets and safety belts.

[0078] The worker behavior feature extraction and classification adopts three-point skeleton connection shape extraction, the three-point skeleton includes left and right ankle bones and sacrum, and the behavior feature extraction is performed according to the triangle feature formed by the three-point skeleton connection line; the worker behavior state classification includes: standing, walking, leaning, climbing, squatting, when the left and right ankle bone points are almost coincident and are far away from the sacrum, it is extracted as standing state; when the two sides of the sacrum connection are nearly equal and much larger than the third side, and the plane where the triangle is located is perpendicular to the reference ground, it is extracted as walking state; when the two sides of the sacrum connection are nearly equal and much larger than the third side, and the plane where the triangle is located is not perpendicular to the reference ground, it is extracted as leaning state; when the two sides of the sacrum connection are greatly different, it is extracted as climbing state, and when the plane where the triangle is located is almost parallel to the reference ground, it is extracted as squatting state.

[0079] S2: Worker behavior state extraction model selection and training

[0080] Load the constructed worker attribute and behavior status image dataset to train the network model. Add skeleton key point recognition before the original target detection Fast-RCNN framework. Use Openpose's skeleton key point recognition network. Use MobileNetV2 lightweight convolutional neural network to replace the original VGG-19 network for the recognition of left and right ankles and sacrum, and connect the key points to form a triangle. Divide the worker attribute and behavior status image dataset into a test set and a validation set at a ratio of 9:1. After completing a batch of training, verify the training effect on the validation set. If the effect is not good, adjust the model parameters and retrain. Before training, set the batch size (Batch-size) to 32 and the training epoch (Epoch) to 200. Set early stopping for training. When the loss value converges, it means that the training is complete, that is, stop training to avoid overfitting. Save the best training weights.

[0081] S3: Input the collected construction edge operation map into the trained model, and use the model to extract the worker attributes and behavior status, such as Figure 6 The worker attributes and the worker behavior status formed by the three-point skeleton connection are shown.

[0082] Step 4, Figure 7 The following is a schematic diagram of the semantic reasoning results provided by the embodiment of the present invention, and the detailed steps are as follows:

[0083] S1: Visual coordinate relationship mining

[0084] The visual coordinate relationship mining module receives the output results of the worker behavior state extraction model and the edge danger zone extraction model, and outputs the visual bounding box coordinates of the danger zone, object, worker attribute and worker behavior state respectively. The intersection over union (IoU) between the visual bounding box coordinates is calculated; the spatial position relationship between the danger zone, object, worker attribute and worker behavior state is obtained based on the IoU; the relationship between the visual bounding boxes of each object includes inclusion, separation and intersection.

[0085] Step S2: Semantization of visual recognition results

[0086] The semanticization of visual recognition results converts the relationship between the pixel coordinates of multiple objects into semantic relationships based on nodes and edges. The recognized objects are represented by nodes, the coordinate relationship is represented by edges, and the relationship between each object corresponds to internal, external, and edge. The specific process includes converting the label data of the visual object into CSV format and using the LoadCSV command of the neo4j graph database to import the CSV data.

[0087] Step S3: Database semantic reasoning

[0088] Database semantic reasoning uses Cypher language to reason about semantic recognition results, and uses the knowledge graph of dangerous edge events as the standard to query whether dangerous edge events appear in the results. Events that meet the dangerous edge knowledge rules are identified in advance and marked with different colors in the reasoning model to facilitate distinction from events that do not meet the dangerous edge knowledge rules.

[0089] In this embodiment, step 2 and step 3 can be performed simultaneously, and the order of the steps is only for the convenience of description.

[0090] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. An intelligent identification system for workers falling off edges based on dangerous signs reasoning. It is characterized in that It includes a construction edge operation image acquisition unit, a semantic database model, an edge danger area extraction model, a worker behavior state extraction model and a semantic reasoning model cascaded; The construction edge operation image acquisition unit is used to acquire construction edge operation images; and input the edge danger area extraction model and the worker behavior state extraction model respectively; The edge danger zone extraction model includes an edge zone space range division module and an edge danger zone feature extraction module, wherein the edge zone space range division module is used to divide the edge zone space range of the input construction edge operation image, and the edge danger zone feature extraction module is used to extract the danger zone and the object in the danger zone; The worker behavior state extraction model includes a construction worker attribute extraction module and a worker behavior feature extraction module; wherein the construction worker attribute extraction module extracts the attributes of the worker in the input construction edge work image; the worker behavior feature extraction module extracts the features of the input construction edge work image, and the features are the worker behavior states; The semantic reasoning model includes a visual coordinate relationship mining module, a visual recognition result semanticization module and a database semantic reasoning module; the visual coordinate relationship mining module receives the results output by the worker behavior state extraction model and the edge danger zone extraction model, and outputs the visual frame coordinates of the danger zone, object, worker attribute and worker behavior state respectively; the intersection and union ratio between the visual frame coordinates is calculated, and the spatial position relationship between the danger zone, object, worker attribute and worker behavior state is obtained based on the intersection and union ratio; the visual recognition result semanticization module converts it into a semantic expression according to the intersection and union ratio; and inputs the semantic expression into the database semantic reasoning module; the database semantic reasoning module performs danger sign reasoning on the semantic expression output by the visual recognition result semanticization module according to the edge fall danger sign knowledge rules in the semantic database model, and then determines whether a danger sign event occurs in the construction edge operation image; The semantic database model is a knowledge graph composed of knowledge rule elements of dangerous signs of falling from an edge. The knowledge rules of dangerous signs of falling from an edge are based on the theory of cause of edge accidents and the theory of dangerous signs. The knowledge graph will serve as the basis for distinguishing dangerous sign events.

2. According to the intelligent identification system for workers falling near edges for near-miss reasoning according to claim 1, It is characterized in that The knowledge graph is composed of subject class, attribute class, object class, location class and behavior state class, and semantic information elements are obtained by arranging and combining the subject class, attribute class, object class, location class and behavior state class.

3. According to the intelligent identification system for workers falling near edges for near-miss reasoning according to claim 1, It is characterized in that The objects extracted by the edge danger zone feature extraction module are protection measures.

4. According to the intelligent identification system for workers falling near edges for near-miss reasoning according to claim 1, It is characterized in that The worker attributes extracted by the construction worker attribute extraction module include no safety helmet and no safety belt, no safety helmet but with safety belt, with safety helmet but no safety belt, and with safety helmet and safety belt.

5. According to the intelligent identification system for workers falling near edges for near-miss reasoning according to claim 1, It is characterized in that The near miss events include the following three categories: The first category: "Workers" without safety helmets and safety belts / with safety helmets and safety belts / with safety helmets and safety belts / with safety helmets but without safety belts "standing / walking / leaning / climbing / sitting on the edge of stairs / stair sides / grooves / roof edges / balcony edges" without "protective measures"; The second category: "Workers" without safety helmets and safety belts / without safety helmets but with safety belts "standing / walking / leaning / climbing / sitting" on "the edge of stairs / stair sides / groove edges / roof edges / balcony edges" with "protective measures"; The third category: "Workers with helmets but no safety belts / workers with helmets and safety belts" "climbing" on "stair edges / stair sides / groove edges / roof edges / balcony edges" with "protective measures"; The above three categories are all set as dangerous omen events in the knowledge rules of dangerous omen of falling from edge.

6. An intelligent identification method for workers falling off edges based on dangerous signs reasoning, It is characterized in that The steps include: Step 1, collecting construction edge operation drawings; Step 2: construct an edge danger zone extraction model, and use the constructed edge danger zone extraction model to extract the edge danger zone and objects in the danger zone from the construction edge operation image; Step 3: Build a worker behavior state extraction model, and use the built worker behavior state extraction model to extract worker attributes and behavior states from the construction edge operation diagram; Step 4, based on the output results of the worker behavior state extraction model and the edge danger zone extraction model, output the visual bounding box coordinates of the danger zone, the object, the worker attributes and the worker behavior state, and calculate the intersection-over-union ratio between the visual bounding box coordinates; Based on the intersection-and-union ratio, the spatial position relationship between the dangerous area, the object, the worker's attributes and the worker's behavior status is obtained to realize the visual coordinate relationship mining and semanticize the visual recognition results; Database semantic reasoning uses Cypher language to reason about semantic recognition results, and uses the knowledge graph of dangerous edge events as a standard to query whether dangerous edge events appear in the results; The process for step 2 is as follows: S2.1: Augment the construction edge area work images to construct the edge work original image dataset; use the labelme tool to label the edge danger area and protective measures, generate a json file to complete the construction of the edge danger area labeling dataset, and the edge work original image dataset and the edge danger area labeling dataset are collectively referred to as the edge danger area image dataset; S2.2: Use Mask-RCNN to load the constructed edge danger zone image dataset to train the network model and complete the construction of the edge danger zone extraction model; S2.3: Input the collected construction edge operation map into the trained model, and use the model to extract the edge danger area; The process for step 3 is as follows: S3.1: Augment the images of construction edge work to construct the edge work original image dataset; use the labelme tool to annotate the worker attributes and three-point skeleton formation, generate a json file to complete the construction of the worker attribute and behavior status annotation dataset; the edge work original image dataset and the worker attribute and behavior status annotation dataset are collectively referred to as the worker attribute and behavior status image dataset; S3.2: Load the constructed worker attribute and behavior status image dataset to train the network model, add skeleton key point recognition before the original target detection Fast-RCNN framework, use Openpose's skeleton key point recognition network, use MobileNetV2 lightweight convolutional neural network to replace the original VGG-19 network, for the recognition of left and right ankles and sacrum, connect the key points to form a triangle; complete the construction of the worker behavior status extraction model; S3.3: Input the collected construction edge work images into the trained model, and use the model to extract worker attributes and behavior status.

7. The method for intelligently identifying worker falls on edges based on near-miss reasoning according to claim 6, It is characterized in that The extraction and classification of workers' behavioral states adopts three-point skeleton connection shape extraction. The three-point skeleton includes left and right ankle bones and sacrum. The behavioral features are extracted according to the triangle features formed by the three-point skeleton connection line. The classification of workers' behavioral states includes: standing, walking, leaning, climbing, and squatting. When the left and right ankle points almost coincide and are far away from the sacrum, they are extracted as standing state; when the two sides of the sacrum connection are nearly equal and much larger than the third side, and the plane where the triangle is located is perpendicular to the reference ground, it is extracted as walking state; when the two sides of the sacrum connection are nearly equal and much larger than the third side, and the plane where the triangle is located is not perpendicular to the reference ground, it is extracted as leaning state; when the two sides of the sacrum connection are greatly different, it is extracted as climbing state, and when the plane where the triangle is located is almost parallel to the reference ground, it is extracted as squatting state.

Images