Worker behavior detection method and system based on computer vision and knowledge graph

Abstract

The application relates to a worker behavior detection method based on computer vision and a knowledge graph, which comprises the following steps: acquiring a construction site image, and identifying a target entity in the construction site according to a target detection model; performing panoramic segmentation on the construction site image by using a panoramic segmentation model to acquire scene information where workers are located; performing motion trajectory tracking on the workers in the construction site image by using a trajectory tracking identification model to acquire motion state information of the workers; performing secondary data mining according to the information acquired from the construction site image to acquire image feature information corresponding to an ontology model, so as to create a graphic data instance; and judging the type of unsafe behavior, the consequence of the unsafe behavior and giving intervention or management measures for the unsafe behavior by performing conditional matching with rules in a rule base according to the graphic data instance. Compared with the prior art, the application has the advantages that the automatic judgment of the unsafe behavior of the workers can be realized, the application is easy to expand, and the application can flexibly adapt to the needs of different construction scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of construction site worker behavior detection, and in particular to a worker behavior detection method and system based on computer vision and a knowledge graph. BACKGROUND

[0002] In recent years, construction safety problems have attracted widespread attention. According to accident cause statistics, 80-90% of construction safety accidents are caused by unsafe behavior of people, so unsafe behavior on the construction site is considered to be the main cause of accidents. Behavior safety management is an effective method to regulate worker behavior and avoid accidents.

[0003] With the continuous development of technology, construction safety intelligent supervision is the trend of future development. Computer vision, deep learning and other technologies are gradually applied to the identification of unsafe behavior on the construction site. In the identification of unsafe behavior in scenarios such as worker intrusion into dangerous construction areas, worker safety equipment wearing, and worker high-altitude operation, computer vision technology has outstanding performance. Compared with manual supervision, automatic supervision relying on computer vision technology has significant advantages: efficient, accurate, labor-saving, and does not interfere with the construction site frontline construction operation.

[0004] However, the current construction site worker unsafe behavior identification technology based on computer vision still has some common limitations, including: limited application scenario range, often only judging a single unsafe behavior; poor scalability, unable to flexibly meet the identification needs of new unsafe behaviors due to changes in regulatory requirements; unable to further extract semantic relationships between targets, making it difficult to meet the needs of knowledge-intensive safety supervision. SUMMARY

[0005] The purpose of the present application is to overcome the above-mentioned limitations of the prior art, such as limited application scenarios, poor scalability, and inability to further extract semantic relationships between targets, and to provide a worker behavior detection method and system based on computer vision and a knowledge graph.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] A worker behavior detection method based on computer vision and a knowledge graph, comprising the following steps:

[0008] Obtain a construction site image, and identify the target entities in the construction site according to a pre-established and trained target detection model; perform panoramic segmentation on the construction site image using a pre-established and trained panoramic segmentation model to obtain scene information of the workers; track the motion trajectory of the workers in the construction site image using a pre-established and trained trajectory tracking identification model to obtain the motion state information of the workers;

[0009] According to the information obtained from the construction site image, secondary data mining is performed to obtain image feature information corresponding to the pre-constructed ontology model, thereby creating a graph data instance;

[0010] According to the graph data instance, conditional matching is performed with the rules in the pre-established rule base, thereby judging the type of unsafe behavior, the consequences of unsafe behavior, and giving intervention or management measures for unsafe behavior.

[0011] Further, the semantic concepts in the ontology model include:

[0012] behavior subject, the main initiator of unsafe behavior, including workers;

[0013] object, an entity existing in the construction site image other than the behavior subject;

[0014] activity, a change caused by the behavior subject;

[0015] place, the location and interface involved in the unsafe behavior;

[0016] time, the time involved in the unsafe behavior;

[0017] attribute, a specific description of the nature of the entity;

[0018] behavior consequence, the result caused by the subject behavior;

[0019] feedback, intervention or management measures acting on the behavior subject based on the behavior consequence.

[0020] Further, the judgment process of the type of unsafe behavior includes: at a specific time and place, the behavior subject with specific attributes engages in a specific activity, and a specific interaction and association occurs between the behavior subject and the object with specific attributes, causing a tendency of behavior consequence, and giving corresponding feedback.

[0021] Further, the rules include result items and condition items, and the result items and condition items are in the form of triples;

[0022] The process of judging the type of unsafe behavior according to the conditional matching of the rules includes: according to the condition items in the rules, the information in the graph data instance is matched, thereby obtaining the corresponding result items, and outputting the type of unsafe behavior that occurs.

[0023] Further, the identified target entities of the construction site include behavior subjects and objects, and the interaction and association between the subjects and objects are judged according to the overlap ratio of the identified subjects and objects;

[0024] According to the scene information of the identified worker, the attribute information of the position where the behavior subject is located is obtained.

[0025] According to the result of tracking the motion trajectory of the worker, attribute information of the motion state corresponding to the behavior subject is acquired;The place and time are pre-input information.

[0026] Further, the acquisition process of the position includes: taking a point in the detection frame of the detected target entity as a key point, using a panoramic segmentation model to acquire the category of the image pixel point corresponding to the key point, so as to judge the position.

[0027] Further, the acquisition process of the motion state includes: using the coordinate change information of the detection frame of the target entity in different image frames, calculating the average pixel speed of the target entity, so as to be mapped into real speed, and obtain the corresponding motion state.

[0028] Further, the rules in the rule base include:

[0029] Internal logic rules for judging unsafe behavior types;

[0030] Unsafe behavior consequence judgment rules for judging the consequences of unsafe behavior;

[0031] Unsafe behavior feedback judgment rules for feedback intervention or management measures of unsafe behavior.

[0032] Further, the unsafe behavior consequence judgment rules and the unsafe behavior feedback judgment rules are obtained by encoding transformation based on the corresponding clauses in the construction safety specification.

[0033] The application also provides a worker behavior detection system based on computer vision and knowledge graph, comprising:

[0034] A camera is arranged at the construction site for acquiring construction site images;

[0035] An image feature extraction module is used to identify construction site target entities in the construction site images by using a pre-established and trained target detection model;The panoramic segmentation model is used to perform panoramic segmentation on the construction site images to acquire the scene information of the workers;The trajectory tracking identification model is used to track the motion trajectory of the workers in the construction site images to acquire the motion state information of the workers;

[0036] An ontology model matching module is used to perform secondary data mining according to the information acquired from the construction site images, acquire image feature information corresponding to the pre-constructed ontology model, and create a graphical data instance;

[0037] A knowledge reasoning module is configured to determine the type of unsafe behavior, the consequence of the unsafe behavior and the intervention or management measure for the unsafe behavior by matching the condition of the rule in the pre-established rule base according to the graphic data instance.

[0038] Compared with the prior art, the present application provides a construction site worker unsafe behavior detection system based on computer vision technology and a knowledge graph. First, the classification of unsafe behavior entities and the relationship between unsafe behavior entities are clearly defined, and on this basis, a construction site worker unsafe behavior ontology model is constructed. Second, computer vision algorithms are used to extract feature information of different dimensions of construction site worker unsafe behavior from images. Finally, the above ontology model is used as the skeleton, and the above image feature information is used as the filling to construct a construction site worker unsafe behavior graph database, and a construction site worker unsafe behavior rule reasoning module adapted to the knowledge graph is developed, which together constitute the construction site worker unsafe behavior knowledge graph. Using the knowledge graph, the worker unsafe behavior in the image captured by the construction site monitoring equipment can be automatically detected, and the following advantages are provided:

[0039] (1) The present application combines computer vision technology and ontology to construct a construction site worker unsafe behavior knowledge graph and develop a matching knowledge reasoning module, thereby establishing a construction site worker unsafe behavior detection system based on the knowledge graph. The system introduces diversified computer vision information for unsafe behavior judgment, is easy to expand, has the ability to handle complex and variable unsafe behavior rules, and effectively solves the problem of automatic recognition of complex construction site worker unsafe behavior rules.

[0040] (2) The construction site worker unsafe behavior knowledge graph constructed by the present application provides a way to integrate and structurally store safety information and knowledge related to unsafe behavior in images. Through automatic accumulation of safety data, it serves the needs of data backtracking, querying and analysis mining in construction safety management. Second, the automatic reasoning function of the knowledge graph in the present application realizes the automatic recognition of construction site worker unsafe behavior from the semantic level. Through the definition of rules, it can flexibly adapt to the needs of different construction scenes and give corresponding management suggestions (such as intervention measure recommendation), thereby reducing the hidden dangers of construction sites and preventing accidents, and providing technical support for the optimization and efficiency improvement of construction site safety supervision methods. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A flowchart of a worker behavior detection method based on computer vision and a knowledge graph provided in an embodiment of the present application is shown in the figure;

[0042] Figure 2A schematic diagram of the architecture of a construction site worker unsafe behavior ontology model provided in an embodiment of the present application is shown in Figure 1.

[0043] Figure 3 A specific construction flow diagram of a reasoning-capable knowledge graph provided in an embodiment of the present application is shown in Figure 2.

[0044] Figure 4 A schematic diagram of an example of a computer vision algorithm detection result of an image sample and an instance creation of a graph database provided in an embodiment of the present application is shown in Figure 3.

[0045] Figure 5 A specific component diagram of a rule base provided in an embodiment of the present application is shown in Figure 4.

[0046] Figure 6 An integration diagram of outputting a reasoning result to a graph database provided in an embodiment of the present application is shown in Figure 5. DETAILED DESCRIPTION

[0047] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0048] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0049] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0050] Embodiment 1

[0051] As shown in Figure 1, the present embodiment provides a worker behavior detection method based on computer vision and knowledge graph, including the following steps: Figure 1

[0052] ​S1: Obtain construction site images, and identify target entities in the construction site according to a pre-established and trained target detection model; perform panoramic segmentation on the construction site images using a pre-established and trained panoramic segmentation model to obtain scene information of workers; track the motion trajectories of workers in the construction site images using a pre-established and trained trajectory tracking identification model to obtain motion state information of the workers;

[0053] S2: Perform secondary data mining based on the information obtained from the construction site images to obtain image feature information corresponding to the pre-established ontology model, thereby creating a graph data instance;

[0054] S3: According to the graph data instance, conditionally match the rules in the pre-established rule base to determine the type of unsafe behavior, the consequences of the unsafe behavior, and the intervention or management measures for the unsafe behavior.

[0055] Corresponding to include:

[0056] (1) Based on relevant literature, specifications, and expert experience, etc., the specific connotation of unsafe behavior is determined, and the classification of unsafe behavior entities and the relationship between entities are defined, thereby constructing an ontology model of worker unsafe behavior in the construction site.

[0057] (2) Taking pictures taken by a monocular camera as the object, a variety of computer vision technologies are used, including target recognition algorithms, panoramic segmentation algorithms, and trajectory tracking algorithms, to extract image feature information of worker unsafe behavior from multiple dimensions. On the one hand, a target detection model is trained to identify various entities related to worker unsafe behavior in the construction site; on the other hand, existing computer vision algorithms, including panoramic segmentation algorithms under the Detectron2 framework and DeepSORT multi-target trajectory tracking algorithms, are used to obtain semantic information such as attribute information and image background information of the above entities.

[0058] (3) Taking the ontology model of worker unsafe behavior in the construction site as the skeleton and the image feature information of worker unsafe behavior in the construction site obtained based on computer vision technology as the filling, a structured graph database is constructed in the graph database system Neo4j as a basic knowledge graph.

[0059] (4) Develop a knowledge reasoning module based on the worker unsafe behavior knowledge stored in the graph database in the Neo4j system to automatically reason about worker unsafe behavior in images taken by construction site monitoring equipment, detect the type of unsafe behavior, and further reason to obtain corresponding behavior management knowledge (i.e., intervention and management measures).

[0060] Specifically, the semantic concepts in the ontology model include:

[0061] Subject, the main initiator of unsafe behavior, including workers;

[0062] Object, entities existing in the construction site image except for the subject;

[0063] Activity, changes caused by the subject;

[0064] Place, the location and interface involved in the unsafe behavior;

[0065] Time, the time involved in the unsafe behavior;

[0066] Attribute, a specific description of the nature of the entity;

[0067] Behavior consequence, the result caused by the subject's behavior;

[0068] Feedback, intervention or management measures acting on the subject based on the behavior consequence.

[0069] The judgment process of the unsafe behavior type includes that the subject with a specific attribute engages in a specific activity at a specific time and place, and a specific interaction and association occurs between the subject and the object with a specific attribute, causing a tendency of behavior consequence, and giving corresponding feedback.

[0070] The rule includes a result item and a condition item, and both the result item and the condition item are in the form of a triple;

[0071] The process of judging the unsafe behavior type according to the rule includes that according to the condition item in the rule, the information in the graph data instance is matched to obtain the corresponding result item, and the type of the unsafe behavior occurring is output.

[0072] The identified target entities of the construction site include the subject and the object, and the interaction and association between the subject and the object are judged according to the overlap ratio of the identified subject and object;

[0073] According to the scene information of the identified worker, the attribute information of the position where the subject is located is obtained;

[0074] According to the result of tracking the motion trajectory of the worker, the attribute information of the motion state corresponding to the subject is obtained; the place and the time are pre-input information.

[0075] The process of obtaining the position includes that a point in the detection frame of the detected target entity is taken as a key point, the class of the image pixel point corresponding to the key point is obtained by using a panoramic segmentation model, so as to judge the position.

[0076] The acquisition process of the motion state comprises: taking the coordinate change information of the detection frame of the target entity in different image frames, calculating the average pixel speed of the target entity, mapping the average pixel speed into a real speed, and obtaining the corresponding motion state.

[0077] The embodiment also provides a worker behavior detection system based on computer vision and a knowledge graph, comprising:

[0078] A camera is arranged at the construction site and used to acquire construction site images.

[0079] An image feature extraction module is used to identify the construction site target entity of the construction site image by using a pre-established and trained target detection model, to perform panoramic segmentation on the construction site image by using a pre-established and trained panoramic segmentation model, to acquire the scene information of the workers, and to acquire the motion state information of the workers by using a pre-established and trained trajectory tracking identification model to track the motion trajectory of the workers in the construction site image.

[0080] A body model matching module is used to perform secondary data mining according to the information acquired from the construction site image, to acquire image feature information corresponding to the pre-established body model, and to create a graphic data instance.

[0081] A knowledge reasoning module is used to judge the unsafe behavior type, the consequence of the unsafe behavior and the intervention or management measures of the unsafe behavior by performing conditional matching with the rules in the pre-established rule base according to the graphic data instance.

[0082] The specific content and beneficial effects of the system of the present application can be referred to the above-mentioned method embodiment, and will not be repeated here.

[0083] The specific implementation process of the above-mentioned method will be introduced below by taking a specific example.

[0084] The worker behavior detection method based on computer vision and a knowledge graph of the present example comprises the following steps:

[0085] (1) The modeling of the ontology can be divided into two parts: entity definition and relationship definition between entities. First, the unsafe behavior of the workers at the construction site is defined as: at a specific time and place, a behavior subject (i.e. a worker) with specific attributes engages in a specific activity (i.e. construction production operation) and interacts and associates with an object (i.e. machinery, materials, environment, etc.) with specific attributes, (possibly) causes a tendency of consequences (i.e. unsafe behavior), and the manager should give corresponding feedback (i.e. intervention and management measures).

[0086] (2) From the definition of unsafe behavior of construction site workers mentioned above, 8 key words are extracted as semantic concepts in the ontology model, which are: behavior subject, object, activity, location, time, attribute, behavior consequence and feedback. They are the first-level classification of entities in the ontology model, and their connotations and secondary classification are given in Table 1.

[0087] Table 1 Definition of semantic concepts (i.e. entities) of unsafe behavior of construction site workers and their secondary classification

[0088]

[0089]

[0090] (3) According to the characteristics of computer vision technology, the graph database language is used to describe the relationship between the above entities, so as to better meet the needs of intuition and visualization. Figure 2 The architecture of the ontology model of unsafe behavior of construction site workers is shown, which mainly contains 4 parts of information: pre-input information, image subject information, entity attribute information, and safety knowledge.

[0091] (4) In the ontology model of unsafe behavior of construction site workers, the pre-input information provides supplementary information of the construction scene that is difficult to obtain directly in the image, such as time and location, which limits the spatio-temporal attributes of the construction scene where the behavior subject (i.e. worker) is located. The behavior subject (i.e. worker) and the object (i.e. machinery, materials, environment, etc.) together constitute the image subject information part, and their spatial positions are directly related. At the same time, the behavior subject (i.e. worker) and the object (i.e. machinery, materials, environment, etc.) are respectively attached with corresponding attribute information, which is used to record their behavior characteristics. The above entities and the relationship between entities constitute the activity (i.e. construction production operation). Finally, safety knowledge is mined and deduced from the activity (i.e. construction production operation), and feedback is applied to the safety management of the behavior subject (i.e. worker).

[0092] (5) After completing the construction of the above ontology model, in order to realize the accurate identification of unsafe behavior of construction site workers, multiple computer vision technologies are needed, including YOLO V5s target detection algorithm, panoramic segmentation algorithm under Detectron2 framework and DeepSORT multi-target trajectory tracking algorithm, which are used to complete target recognition, panoramic segmentation and trajectory tracking tasks respectively, to identify target entities and obtain multi-dimensional image feature information of unsafe behavior of construction site workers as attribute information of target entities. The specific method is:

[0093] (6) The first step is to identify the entity targets in the construction site. The YOLO V5s target detection algorithm is used to train 100 rounds of detection weights using the SODA open source dataset, achieving identification of 14 types of construction site target entities, including personnel, safety helmets, safety vests, scaffolding, fences, boards, bricks, wood, tower crane hooks, cutting machines, electrical boxes, trolleys, hoppers, and slogans.

[0094] (7) The second step is to perform panoramic segmentation of the construction site image. The panoramic segmentation algorithm under the Detectron2 framework is used to obtain the "context" information of the workers in the image, to establish the connection between the foreground targets and the background, and to obtain the "location" and other subject attributes.

[0095] (8) The third step is to track and identify the unsafe behavior subjects (i.e., workers). The Deep Simple Online and Realtime Tracking (DeepSORT) multi-target tracking algorithm is used to obtain the motion trajectory of the "worker" category target, achieving the acquisition of dynamic attributes such as "motion state" and "displacement".

[0096] (9) The fourth step is to perform secondary data mining on the discrete image feature information, and extract the relationship information between the information corresponding to the ontology model, i.e., the general spatial location relationship and the attribute subordination relationship. The specific approach is as follows:

[0097] (10) On the one hand, the spatial location relationship between two targets is determined based on the calculation result of the Intersection over Union (IoU) of their detection boxes, as shown in equation (1). The spatial location relationship is divided into three categories: within, overlap, and away.

[0098]

[0099] (11) On the other hand, the attribute subordination relationship refers to the relationship between a specific attribute and a certain characteristic entity. The following two typical attributes of workers are used as examples to illustrate their definitions and acquisition methods.

[0100] (12) First, for the static attribute "location" of the worker, the bottom midpoint of the personnel detection box is set as the key point, and the class of the image pixel point corresponding to the key point is obtained using the panoramic segmentation algorithm. The location of the worker is determined based on the pixel class.

[0101] (13) Second, for the dynamic attribute "motion state" of the worker, 15 frames are taken as a unit, and the coordinates of the worker detection box midpoint in the first frame and the 15th frame are assumed to be (x1, y1) and (x15, y15) respectively. The motion state of the worker is determined based on the distance between the two points.15 , y 15 ) — With the DeepSORT algorithm, the coordinate information of the same object in different frames can be matched according to the id of the object detection box, so that the data information of (x1, y1) and (x 15 , y 15 ) can be obtained. Then the average pixel speed of the worker in the 15 frames can be calculated according to formula (2). Then, taking the general height of a person 1.6 m as a reference, the pixel speed is mapped to the real speed, as shown in formula (3).

[0102]

[0103]

[0104] In formula (3), H is the pixel height of the personnel detection box. Considering the possible errors in speed calculation, a threshold is set to simplify the worker's speed attribute to a "motion state" attribute, and the mapping relationship is shown in formula (4).

[0105]

[0106] (14) In order to realize the automatic reasoning of the unsafe behavior of construction site workers in the knowledge graph, a graphical database of unsafe behavior knowledge and a rule reasoning module are constructed, and a data transmission path between the two is established, Figure 3 the specific construction process of the inferable knowledge graph.

[0107] (15) Select Neo4j graph database system as the carrier of the knowledge graph, and perform data modeling to convert the above obtained construction site entity target recognition results, entity attributes and entity relationship information into database instances. First, the image feature information of the construction site worker's unsafe behavior obtained by computer vision technology is uniformly output as a structured text with a suffix of ".json", and each subject or object object is assigned a non-repeating id; Then, write a Python program to realize the automatic reading and input of the specified json file information, and the Neo4j system structures the information according to the input content to complete the creation of graph data instances.

[0108] (16) When creating graph data instances, in order to facilitate the representation of pre-input information in the same image, the "ImageID" node is introduced as the initial node of a single graph data instance, which connects the behavior subject, object and pre-input information. At the same time, each subject or object corresponds to an "Attribute:<entity name>" node, and the attribute information of each subject or object is attributed to the "Attribute:<entity name>" node corresponding to the subject or object, so as to facilitate subsequent safety knowledge reasoning and query. Figure 4An example of the detection result of a computer vision algorithm and the creation of a graph database instance of an image sample is shown.

[0109] (17) The rule inference module includes three parts: rule base, inference mechanism, and reading and outputting of triples.

[0110] (18) The rules in the rule base are stored in the form of text in a designated text file. Each rule includes a result item and a condition item, and there is a derivable relationship between the two. The basic format of each item is the same, which is a triple in the form of relation (X, R, Y), wherein X and Y represent two objects with a front-back order, and R represents the relationship between the two. Items are connected by the “&” symbol. For example, a simple equal substitution rule is expressed as follows:

[0111] When the condition item of the rule is met, the rule is triggered to perform knowledge inference, and the result obtained by inference is stored in the database in the form of the above triple as a new fact.

[0112] (19) Starting from the inference requirement, the rule inference in the present application includes two parts: unsafe behavior type and possible consequences, and intervention or management measure recommendation of unsafe behavior. Figure 5 The specific composition of the rule base is shown, which includes three types of rules: the first type of rule is internal logic, and this type of rule is always true, which simplifies the inference process; the second and third types of rules correspond to the inference contents of the two items respectively, and the two types of rules are obtained by encoding the corresponding clauses in the construction safety specification.

[0113] (20) The inference mechanism in the rule inference module can be simply described as follows: according to the rules in the rule base, the objects meeting the conditions in the fact database are matched, and new triple relationships are generated by inference and stored in the complete database (i.e. fact + inference result). The newly generated triple is the target inference result, and finally these results are output to the graph database, thereby completing the knowledge architecture in the construction personnel unsafe behavior graph.

[0114] (21) After inference, each behavior subject (i.e. worker) will obtain an independent behavior judgment result. After the inference result is output to the graph database, it is integrated into the “Result:<entity name>” node connected to the corresponding behavior subject (i.e. worker), as shown in Figure 6 .

[0115] The preferred embodiments of the present application have been described above in detail. It should be understood that modifications and variations to the preferred embodiments could be made by those skilled in the art in light of the teachings above. It is therefore contemplated that the application can encompass other variations and modifications that fall within the scope of the claims.

Claims

1. A worker behavior detection method based on computer vision and knowledge graph, characterized in that, The method comprises the following steps: obtaining a construction site image, and identifying a target entity in the construction site according to a pre-established and trained target detection model; performing panoramic segmentation on the construction site image using a pre-established and trained panoramic segmentation model to obtain scene information of workers; and tracking a motion trajectory of the workers in the construction site image using a pre-established and trained trajectory tracking and identification model to obtain motion state information of the workers; performing secondary data mining according to information obtained from the construction site image to obtain image feature information corresponding to a pre-established ontology model, thereby creating a graph data instance; judging an unsafe behavior type, a consequence of the unsafe behavior, and giving an intervention or management measure for the unsafe behavior by condition matching with rules in a pre-established rule base according to the graph data instance. 2.The worker behavior detection method based on computer vision and knowledge graph according to claim 1, characterized in that, The semantic concepts in the ontology model include: a behavior subject, a main initiator of the unsafe behavior, including a worker; an object, an entity in the construction site image other than the behavior subject; an activity, a change caused by the behavior subject; a location, a position and interface involved in the unsafe behavior; a time, a time involved in the unsafe behavior; an attribute, a specific description of the nature of an entity; a behavior consequence, a result caused by the behavior of the subject; feedback, an intervention or management measure for the behavior subject based on the behavior consequence. 3.The worker behavior detection method based on computer vision and knowledge graph according to claim 2, characterized in that, The judgment process of the unsafe behavior type includes that the behavior subject with a specific attribute engages in a specific activity at a specific time and location, and a specific interaction and association occurs between the behavior subject and the object with a specific attribute, causing a tendency of a behavior consequence, and giving a corresponding feedback. 4.The worker behavior detection method based on computer vision and knowledge graph according to claim 3, characterized in that, The rules include result items and condition items, and the result items and the condition items are in the form of triples; The process of judging the unsafe behavior type according to the condition matching with the rules includes that information in the graph data instance is matched according to the condition items in the rules, thereby obtaining corresponding result items, and outputting the type of the unsafe behavior occurring. 5.The worker behavior detection method based on computer vision and knowledge graph according to claim 2, characterized in that, The identified target entity in the construction site includes the behavior subject and the object, and the interaction and association between the behavior subject and the object are judged according to an overlap ratio of the identified behavior subject and the object; The scene information of the workers is used as attribute information of a location where the behavior subject is located; Attribute information of a motion state of the behavior subject is obtained according to a result of the motion trajectory tracking of the workers; and the location and the time are pre-input information.

6. The worker behavior detection method based on computer vision and knowledge graph according to claim 5, characterized in that, The process of obtaining the location includes that a point in a detection frame of the detected target entity is used as a key point, a category of an image pixel point corresponding to the key point is obtained using the panoramic segmentation model, and thereby the location is judged.

7. The worker behavior detection method based on computer vision and knowledge graph according to claim 5, characterized in that, The process of obtaining the motion state includes that coordinate change information of the detection frame of the target entity in different image frames is used to calculate an average pixel speed of the target entity, thereby mapping the average pixel speed into a real speed to obtain a corresponding motion state. 8.The worker behavior detection method based on computer vision and knowledge graph according to claim 1, characterized in that, The rules in the rule base include: an internal logic rule for judging the unsafe behavior type; an unsafe behavior consequence judgment rule for judging the consequence of the unsafe behavior. The unsafe behavior feedback judgment rule is used for feeding back intervention or management measures of the unsafe behavior. 9.The worker behavior detection method based on computer vision and knowledge graph according to claim 8, characterized in that, The unsafe behavior consequence judgment rule and the unsafe behavior feedback judgment rule are both obtained through coding transformation based on corresponding clauses in the construction safety specification. 10.A worker behavior detection system based on computer vision and knowledge graph, characterized in that, The method comprises the following steps: a camera is arranged at the construction site to obtain a construction site image; an image feature extraction module is used to identify a construction site target entity of the construction site image by using a pre-established and trained target detection model; a panoramic segmentation model is used to perform panoramic segmentation on the construction site image to obtain scene information of workers; a trajectory tracking identification model is used to track a motion trajectory of the workers in the construction site image to obtain motion state information of the workers; an ontology model matching module is used to perform secondary data mining according to information obtained from the construction site image to obtain image feature information corresponding to a pre-established ontology model, thereby creating a graphic data instance; a knowledge reasoning module is used to judge an unsafe behavior type, a consequence of the unsafe behavior and give intervention or management measures of the unsafe behavior by performing conditional matching with rules in a pre-established rule base according to the graphic data instance.

Images