A risk behavior recognition method based on attention mechanism character interaction perception

Abstract

The present application relates to the technical field of artificial intelligence, in particular to a risk behavior recognition method based on attention mechanism character interaction perception, which comprises the following steps: collecting data for human-object interaction behavior recognition, making a data set, training an existing target detection network using the data set to obtain a character target detection model; detecting actual monitoring scene images using the character target detection model and obtaining corresponding character target detection data; constructing a character feature extraction module to extract the appearance features of people and objects; inputting the character target detection data into a character position relationship module to obtain character spatial position relationship features; establishing a character interaction relationship fusion strategy to output the confidence of each type of character interaction behavior. The present application can focus on the features of the part where the interaction behavior occurs by constructing a character feature extraction module, can better extract the appearance features of people and objects and fuse the upper and lower information, and can improve the accuracy of behavior recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a risk behavior recognition method based on attention mechanism character interaction perception. BACKGROUND

[0002] At present, the risk behavior recognition in the construction site is mainly based on the visual detection of the site safety officer, which depends on the subjective experience and artificial judgment of the safety officer, and is time-consuming and laborious, and the efficiency is not high. It is very critical and necessary to use precise, effective, automatic and intelligent risk behavior recognition and prevention and control technology to ensure the order and safety of the construction site. With the development of computer vision and deep learning technology, it has become a trend to use new methods to realize risk behavior recognition and monitoring in the field of construction intelligence.

[0003] Traditional human-object interaction behavior recognition algorithms mainly use handcrafted local features, which are captured and classified into specific categories. These handcrafted features include color, HOG and SIFT, etc. These features are not obvious, resulting in low recognition rate and poor generalization ability of traditional human-object interaction behavior recognition algorithms. The two-stage three-branch network HO-RCNN based on deep learning can effectively extract the features of "human flow", "object flow" and "human-object flow", and then fuse the three to obtain the final prediction result. To better enrich the feature and semantic information, human state features are integrated into HO-RCNN, graph neural networks are used, and attention modules are added to improve the accuracy of the model for behavior recognition. However, these deep learning-based models cannot well pay attention to the part where the behavior occurs, and the feature extraction capability can be further improved. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art, and provides a risk behavior recognition method based on attention mechanism character interaction perception. By fusing the self-attention mechanism of VIT with channel attention and spatial attention, a character appearance feature extraction network is constructed, and by fusing the interaction relationship features with object appearance features and character appearance features in three channels, an attention mechanism-based character interaction behavior recognition algorithm is constructed, and the human-object interaction behavior recognition technology is applied to the risk prevention and control of the construction scene.

[0005] To solve the above problems, the present application provides a risk behavior recognition method based on attention mechanism character interaction perception, comprising the following steps:

[0006] Collect data for human-object interaction behavior recognition and make a data set. Use the data set to train the existing target detection network to obtain a character target detection model.

[0007] The actual monitoring scene image is detected by using a person target detection model, and corresponding person target detection data is obtained;

[0008] A person feature extraction module is constructed to process the person target detection data and extract appearance features of people and objects;

[0009] According to the person target detection data, corresponding gray image mask images are made, the gray image mask images are input into a person position relationship module, and person spatial position relationship features are obtained;

[0010] A person interaction relationship fusion strategy is established, the appearance features of people and objects are added, and the appearance features are multiplied with the person spatial position relationship features to obtain final prediction features, and each type of person interaction behavior confidence is output according to the final prediction features.

[0011] It should be noted that the data for recognizing the interaction behavior between people and objects is collected, and a data set is made, which includes:

[0012] Based on the camera, pictures related to construction risk prevention behaviors are collected, pictures related to construction risk prevention behaviors in different construction operation scenes are screened, and pictures of this category are retained;

[0013] The retained pictures are labeled by using a labeling tool labelme, the absolute coordinates in the pictures are converted into relative coordinates, and the obtained json file is converted into a YOLO format;

[0014] The interaction action type information of people and objects is added to form a data set.

[0015] Further, the pictures related to construction risk prevention behaviors in different construction operation scenes include:

[0016] Correct / abnormal behavior pictures of wearing safety helmets;

[0017] Correct / abnormal behavior pictures of wearing protective clothing;

[0018] Correct / abnormal behavior pictures of holding tools;

[0019] Pictures of the interaction relationship between workers and objects in other construction scenes.

[0020] Further, the person target detection model selects a YOLOV5s network model.

[0021] Further, the person target detection data includes human body frame data and object frame data.

[0022] It should be noted that the person feature extraction module processes the person target detection data images and extracts appearance features of people, which includes:

[0023] Adjusting the person target detection data to the designed size feature layer by ROI Align;

[0024] The convolutional neural network converts the input feature layer into a set of N Patch sequences;

[0025] The Patch0 sequence is additionally initialized, spliced with the N Patch sequences output by the convolutional neural network, and added to the position encoding vector to obtain a Patch0-PatchN sequence vector carrying position information, which is input into the Vision Transformer model.

[0026] Further, the person feature extraction module processes the person target detection data image to extract the appearance features of the person, which further includes:

[0027] Using a convolutional neural network CNN to adjust the channel of the global feature of the image in the person target detection data to the same dimension as the channel attention weight, and performing point multiplication in the channel dimension to obtain the channel attention feature CA, CA[:,:,i] = Q[i] × F[:,:,i] i = 1, 2, 3, …, d, where d is the dimension of the channel attention, Q represents the channel attention weight, and F is the feature after the main network output feature acts on the CNN;

[0028] The channel attention feature CA is subjected to maximum pooling and average pooling to obtain a double-channel feature layer;

[0029] Using a convolutional neural network to convert the double-channel feature layer into a single-channel feature P, and performing point multiplication between the single-channel feature P as a spatial attention matrix and the channel attention CA to obtain a spatial attention feature SA, SA[:,:j] = P[:,:] × CA[:,:,j] j = 1, 2, 3, …, d;

[0030] After average pooling of the spatial attention feature SA, S is obtained, and W is obtained by splicing S and Q, which completes the extraction of the appearance features of the person, wherein S contains context information, Q refers to the channel attention feature and the feature of the part where the interactive behavior occurs, and W refers to the context information and the interactive behavior feature.

[0031] Further, the corresponding grayscale image mask image is made according to the person target detection data, which is input into the person position relationship module to obtain the person spatial position relationship feature, which includes:

[0032] Initializing two all-zero matrices with the same size as the width W and height H of the target detection data output feature;

[0033] Set the area covered by the human body frame or object frame to 1 to obtain a mask matrix;

[0034] Superimpose the corresponding human body frame gray image mask image and object frame gray image mask image to form a three-dimensional matrix data with a size of 2xWxH;

[0035] Input the three-dimensional matrix data into a convolutional neural network and a fully connected network to obtain spatial position features.

[0036] More, the established character interaction relationship fusion strategy multiplies the appearance features of the characters and the spatial position relationship features of the characters to obtain final prediction features, and according to the classification, outputs the confidence of each type of character interaction behavior, including:

[0037] output=softmax(∑R[k]×(M[l]+O[l])[k])k,l=1,2,3,…,2d, wherein M represents the appearance features of the human body, O represents the appearance features of the object, R represents the relative position features, and output represents the confidence of outputting each type of character interaction behavior.

[0038] The present application can focus on the part of the feature of the interaction behavior by constructing the character feature extraction module, can better extract the appearance features of the human and the object and fuse the upper and lower information, improve the accuracy of behavior recognition, at the same time, the human and object interaction behavior recognition technology is applied to the construction site risk behavior prevention and control, the model can intelligently analyze the risk behavior, saves a lot of manpower, and improves the construction risk behavior prevention and control work efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creating any creative labor.

[0040] Figure 1 It is a risk behavior recognition method flowchart based on attention mechanism character interaction perception in the embodiment of the present application;

[0041] Figure 2 It is a risk behavior recognition algorithm flowchart design schematic diagram based on attention mechanism character interaction perception in the embodiment of the present application;

[0042] Figure 3 It is a flowchart of collecting data of human and object interaction behavior recognition under construction risk prevention and control and making data set in the embodiment of the present application;

[0043] Figure 4 is a structural schematic diagram of a person feature extraction module in an embodiment of the present application;

[0044] Figure 5 is a person appearance feature processing flow schematic diagram in an embodiment of the present application;

[0045] Figure 6 is a structural schematic diagram of the VIT model in an embodiment of the present application;

[0046] Figure 7 is a flow schematic diagram of processing a person target detection data image and extracting a person appearance feature in an embodiment of the present application;

[0047] Figure 8 is a flow schematic diagram of processing a person target detection data image and obtaining a person spatial position relationship feature in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0049] The risk behavior recognition method based on attention mechanism person interaction perception in the embodiments of the present application fuses the self-attention mechanism of VIT with channel attention and spatial attention, constructs a person appearance feature extraction network, and fuses the interaction relationship feature with the object appearance feature and the person appearance feature in three channels to construct an attention mechanism based person interaction behavior recognition algorithm, and applies the person and object interaction behavior recognition technology to construction risk behavior prevention and control.

[0050] The risk behavior recognition method based on the attention mechanism character interaction perception provided by the embodiment of the application comprises the following steps: collecting data of human-object interaction behavior recognition, and making a data set; training an existing target detection network by using the data set to obtain a character target detection model; detecting an actual monitoring scene image by using the character target detection model, and obtaining corresponding character target detection data; constructing a character feature extraction module, performing image processing on the character target detection data, and extracting appearance features of people and objects; making corresponding gray image mask images according to the character target detection data, inputting the gray image mask images into the character feature extraction module, and obtaining character spatial position relationship features; establishing a character interaction relationship fusion strategy, adding the appearance features of people and objects, and multiplying the appearance features of people and objects with the character spatial position relationship features to obtain final prediction features, and classifying the final prediction features to output a confidence degree of each type of character interaction behavior.

[0051] Specifically, Figure 1 A flowchart of the risk behavior recognition method based on the attention mechanism character interaction perception in the embodiment of the application is shown, which comprises the following steps:

[0052] In S101, data of human-object interaction behavior recognition is collected, and a data set is made; an existing target detection network is trained by using the data set to obtain a character target detection model.

[0053] In S102, an actual monitoring scene image is detected by using the character target detection model, and corresponding character target detection data is obtained.

[0054] In S103, a character feature extraction module is constructed, image processing is performed on the character target detection data, and appearance features of people and objects are extracted.

[0055] In S104, corresponding gray image mask images are made according to the character target detection data, the gray image mask images are input into a character position relationship module, and character spatial position relationship features are obtained.

[0056] In S105, a character interaction relationship fusion strategy is established, appearance features of people and objects are added, and the appearance features of people and objects are multiplied with character spatial position relationship features to obtain final prediction features; the final prediction features are classified, and a confidence degree of each type of character interaction behavior is output.

[0057] Specifically, Figure 2 A flowchart of the risk behavior recognition algorithm based on the attention mechanism character interaction perception in the embodiment of the application is shown, and the risk behavior recognition algorithm based on the attention mechanism character interaction comprises three branches: a person-object flow branch, a human body flow branch and an object flow branch, wherein the person-object flow branch is responsible for extracting relative position features of people and objects, the human body flow branch is responsible for extracting appearance features of human bodies, and the object flow branch is responsible for extracting appearance features of objects.

[0058] Further, the appearance features of the human body flow branch extracted from the human body are point-added with the appearance features of the object extracted from the object flow branch, and then are point-multiplied with the human-object relative position features extracted from the human-object flow branch to obtain final prediction features, and the confidence of each type of human-object interaction behavior is output to realize the action behavior prediction.

[0059] Specifically, Figure 3 The data of collecting human-object interaction behavior recognition under construction risk prevention and control in the embodiment of the application is shown, and a flowchart of making a data set is shown, including the following steps:

[0060] S301, collecting pictures related to construction risk prevention and control behaviors based on a camera;

[0061] S302, screening pictures related to construction risk prevention and control behaviors in different scenes, and retaining such category pictures;

[0062] S303, using a labeling tool labelme to label the retained pictures, converting the absolute coordinates in the pictures into relative coordinates, converting the obtained json file into YOLO format, and adding human action type information to form a data set.

[0063] It should be noted that the pictures related to construction risk prevention and control behaviors in different scenes include:

[0064] Correct / abnormal behavior pictures of wearing a safety helmet, further, the interaction relationship between a human and a safety helmet;

[0065] Correct / abnormal behavior pictures of wearing a protective suit, further, the interaction relationship between a human and a protective suit;

[0066] Correct / abnormal behavior pictures of holding a tool, further, the interaction relationship between a human and a construction tool;

[0067] It should be noted that the selection of the human target detection model needs to consider the model performance, inference speed and edge device computing power, and the human target detection model can be selected from YOLOV3, YOLOV3-spp, YOLOV3-tiny, YOLOV4, YOLOV4-tiny, YOLOV5 and YOLOV5s, and the YOLOV5s network model is preferably selected in the embodiment of the application.

[0068] Further, the human target detection data envelope includes human body frame data and object frame data.

[0069] Specifically, Figure 4A structural schematic diagram of a person feature extraction module in an embodiment of the present application is shown, and the person feature extraction module comprises a VIT (Vision Transformer) model, a channel attention model and a spatial attention model.

[0070] Specifically, the human body frame data and the object frame data are converted into N patch sequences through ROI Align and a CNN, patch0 sequence is additionally initialized, is spliced with N patch sequences output by the convolutional neural network CNN, and is added with a position encoding vector to obtain a patch0-patchN sequence vector carrying position information, and the vector is input into the VIT (Vision Transformer) model; the global feature of the feature output by the backbone network is adjusted to a feature F with the same dimension as the channel attention weight by using the convolutional neural network CNN, the F is multiplied by the channel attention weight Q to obtain a channel attention feature CA, CA[:,:,i] = Q[i] * F[:,:,i] i = 1, 2, 3, …, d; the channel attention feature CA is subjected to maximum pooling and average pooling to obtain a double-channel feature layer; the double-channel feature layer is converted into a single-channel feature P by using the convolutional neural network CNN, the single-channel feature P is multiplied by the channel attention feature CA to obtain a spatial attention feature SA, SA[:,:,j] = P[:,:] * CA[:,:,j] j = 1, 2, 3, …, d; the spatial attention feature SA is subjected to average pooling to obtain S containing context information features, S is spliced with the channel attention feature and the feature Q of the interactive behavior occurrence position to obtain W fused with the context information and the interactive behavior feature.

[0071] The module can not only extract the global features of the human body or the object, but also can focus on the positions of the human body and the object where the interactive behavior occurs, that is, the VIT model gives greater weights to the features of these positions.

[0072] Specifically, Figure 5 A person feature processing flowchart in an embodiment of the present application is shown, and the flowchart comprises the following steps:

[0073] S501, adjusting the person target detection data into a designed size feature layer through ROI Align;

[0074] S502, converting the input feature layer into N patch sequences by using a convolutional neural network;

[0075] S503. The Patch0 sequence is additionally initialized, concatenated with the N Patch sequences output by the convolutional neural network, and added to the position encoding vector to obtain the Patch0-PatchN sequence vector carrying position information, and then fed into the VIT (Vision Transformer) model.

[0076] In this embodiment of the invention, a human feature module is constructed to process human target detection data images and extract human appearance features.

[0077] based on Figure 5 The process shown involves adjusting the human target detection data to a pre-designed feature layer using ROI Align and CNN; the convolutional neural network converts the input feature layer into a set N-patch sequence; the Patch0 sequence is additionally initialized, concatenated with the N-patch sequence output by the convolutional neural network, and added to the position encoding vector to obtain the Patch0-PatchN sequence vector carrying position information, which is then fed into the VIT (VisionTransformer) model.

[0078] Specifically, Figure 6 The diagram shows the structure of the VIT model in an embodiment of the present invention. The VIT model is composed of L layers of encoders in series. The input features and output features of the encoders have the same dimension. It is composed of a multi-head self-attention mechanism, a multilayer perceptron, a normalization layer and a shortcut layer.

[0079] based on Figure 6 The structure shown involves passing human bounding box data and object bounding box data through a convolutional neural network (CNN), then adding them to a positional encoder to obtain a Patch0-PatchN sequence vector carrying positional information. This sequence vector is input into an L-layer encoder, which then passes through a normalization layer and a multi-head self-attention mechanism to obtain a first sequence vector. This first sequence vector is then added to the original sequence vector to obtain a second sequence vector. This second sequence vector then passes through a normalization layer and a multilayer perceptron, and is added to the second sequence vector again. This sequence vector operation is repeated L times to obtain the channel attention weights.

[0080] It should be noted that the VIT model operates entirely based on an attention mechanism, and the input it accepts is a sequence vector.

[0081] Furthermore, Figure 7 This illustration shows a flowchart of processing human target detection data images and extracting the appearance features of the human figure according to an embodiment of the present invention, including the following steps:

[0082] S701, adjusting the channel of the global feature of the image in the person target detection data to the dimension of the channel attention weight using a convolutional neural network, and performing point multiplication on the channel dimension to obtain a channel attention feature CA, CA[:,:,i]=Q[i]x F[:,:,i] i=1, 2, 3, …, d, wherein d is the dimension of the channel attention, Q represents the channel attention weight, and F is the feature after the output feature of the backbone network acts on the CNN;

[0083] S702, performing maximum pooling and average pooling on the channel attention feature CA to obtain a double-channel feature layer;

[0084] S703, converting the double-channel feature layer into a single-channel feature P using a convolutional neural network, and performing point multiplication on the channel attention CA by taking the single-channel feature P as a spatial attention matrix to obtain a spatial attention feature SA, SA[:,:,j]=P[:,:]x CA[:,:,j] j=1, 2, 3, …, d;

[0085] S704, obtaining S by performing average pooling on the spatial attention feature SA, and obtaining W by splicing S and Q, wherein S contains context information, Q refers to the channel attention feature and the feature of the part where the interactive behavior occurs, and W refers to the context information and the interactive behavior feature.

[0086] In the embodiment of the application, the image of the person target detection data is processed, the appearance feature of the person is extracted, the semantic feature outside the human body and object region and related to the interactive behavior is extracted by using the channel attention model and the spatial attention model, and the context information is fused.

[0087] Based on Figure 7The flow shown uses a convolutional neural network to adjust the channel of the global feature of the image in the person target detection data to the dimension of the channel attention weight, point multiplies the two in the channel dimension to obtain the channel attention feature CA, CA, :, :, i) = Q[i] * F, :, :, i) i = 1, 2, 3, …, d, wherein d is the dimension of the channel attention, Q represents the channel attention weight, and F is the feature after the output feature of the backbone network acts on the CNN. The channel attention feature CA is subjected to maximum pooling and average pooling to obtain a double-channel feature layer. The double-channel feature layer is converted into a single-channel feature P using a convolutional neural network, the single-channel feature P is point multiplied with the spatial attention matrix to obtain a spatial attention feature SA, SA, :, :, j) = P, :, :)* CA, :, :, j) j = 1, 2, 3, …, d. The spatial attention feature SA is subjected to average pooling to obtain S, S is spliced with Q to obtain W, and the appearance feature extraction of the person is completed, wherein S contains context information, Q refers to the channel attention feature and the feature of the part where the interactive behavior occurs, and W refers to the context information and the interactive behavior feature fused.

[0088] More, Figure 8 A flowchart for processing a person target detection data image to obtain a person spatial position relationship feature in the embodiment of the application is shown, including the following steps:

[0089] S801, two all-zero matrices are initialized, and the size is consistent with the width W and the height H of the target detection data output feature;

[0090] S802, the area covered by the human body frame or the object frame is set to 1 to obtain a mask matrix;

[0091] S803, the corresponding human body frame grayscale image mask image and the object frame grayscale image mask image are superimposed to form a three-dimensional matrix data with a size of 2*W*H;

[0092] S804, the three-dimensional matrix data is input into the spatial position feature in the convolutional neural network and the fully connected network feature.

[0093] In the embodiment of the application, the corresponding grayscale image mask image is made according to the person target detection data, which is input into the person feature extraction module to obtain the person spatial position relationship feature.

[0094] Based on Figure 8The flowchart shown, initializes two all-zero matrices, the size of which is consistent with the target detection data output feature width W and height H. The area covered by the human body frame or object frame is set to 1 to obtain a mask matrix. The corresponding human body frame gray image mask image and object frame gray image mask image are superimposed to form a three-dimensional matrix data with a size of 2xWxH. The three-dimensional matrix data is input into the convolutional neural network and the fully connected network to obtain the spatial position feature.

[0095] It should be noted that the gray image mask image includes a human body frame gray image mask image and an object frame gray image mask image.

[0096] Further, the establishment of the character interaction relationship fusion strategy multiplies the appearance features of the characters and the spatial position relationship feature points of the characters to obtain the final prediction features, and outputs the confidence of each type of character interaction behavior according to the classification, including:

[0097] output=softmax(∑R[k]×(M[l]+O[l])[k])k,l=1,2,3,…,2d, wherein M represents the human appearance feature, O represents the object appearance feature, R represents the relative position feature, and output represents the confidence of outputting each type of character interaction behavior.

[0098] In summary, the embodiment of the present application constructs a character feature extraction module fused with VIT, channel attention and spatial attention, so that the model can focus on the part of the feature that occurs the interaction behavior, and can better extract the appearance features of people and objects and fuse the upper and lower information, thereby improving the accuracy of behavior recognition. And the human and object interaction behavior recognition technology is applied to construction risk prevention and control, so that the model can intelligently analyze the construction risk prevention and control behavior, saving a lot of manpower and improving the construction risk prevention and control work efficiency.

[0099] The above has introduced the embodiment of the present application in detail, and the specific examples have been used to describe the principle and implementation mode of the present application; the above embodiment description is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for identifying risky behaviors based on attention-based human interaction perception, characterized in that, The method includes: Collect data on human-object interaction behavior recognition, create a dataset, and use the dataset to train an existing object detection network to obtain a human object detection model; The human target detection model is used to detect human targets in actual surveillance scene images and obtain the corresponding human target detection data. A human feature extraction module is constructed to perform image processing on human target detection data and extract the appearance features of the human; The character feature extraction module processes the character target detection data image to extract the character's appearance features, including: adjusting the character target detection data to a feature layer of a designed size using ROI Align; converting the input feature layer into a set N Patch sequence using a convolutional neural network; additionally initializing the Patch0 sequence, concatenating it with the N Patch sequences output by the convolutional neural network, and adding it to the position encoding vector to obtain the Patch0-PatchN sequence vector carrying position information, and then feeding it into the Vision Transformer model. The aforementioned character feature extraction module processes the character target detection data image to extract the character's appearance features. This includes: using a convolutional neural network (CNN) to adjust the dimensions of the global features of the image in the character target detection data to be the same as the dimension of the channel attention weights; and then multiplying the two along the channel dimension to obtain the channel attention feature (CA). ; Where d is the dimension of channel attention, Q represents the channel attention weight, and F is the feature of the backbone network output after being applied to the convolutional neural network; max pooling and average pooling are applied to the channel attention feature CA to obtain a dual-channel feature layer; the convolutional neural network is used to transform the dual-channel feature layer into a single-channel feature P, and the single-channel feature P is multiplied by the channel attention feature CA as a spatial attention matrix to obtain the spatial attention feature SA. ; The spatial attention feature SA is averaged and pooled to obtain S. S is concatenated with Q to obtain W, thus completing the extraction of the appearance features of the person. S contains contextual information, Q refers to the channel attention feature and the feature of the location where the interaction occurs, and W refers to the feature that combines contextual information and interaction behavior. Based on the human target detection data, corresponding grayscale image masks are created. The grayscale image masks are then input into the human position relationship module to obtain the human spatial position relationship features. A strategy for fusing human-object interaction relationships is established. The appearance features of people and objects are added together and multiplied by the spatial positional relationship features of the people to obtain the final predicted features. Based on the final predicted features, the system is classified and the confidence score of each type of human interaction behavior is output.

2. The risk behavior identification method as described in claim 1, characterized in that, The process of collecting data on human-object interaction behavior and creating a dataset includes: Based on images collected by cameras related to construction risk prevention and control behaviors, images related to construction risk prevention and control behaviors in different construction operation scenarios are filtered and retained. The retained images were annotated using the labelme annotation tool. The absolute coordinates in the images were converted to relative coordinates, and the resulting JSON file was converted into YOLO format. Add information about the types of interactions between people and objects to create a dataset.

3. The risk behavior identification method as described in claim 2, characterized in that, The images related to construction risk prevention and control behaviors in different construction operation scenarios include: Images of correct / abnormal helmet wearing behaviors; Images showing correct and incorrect ways of wearing protective clothing; Images showing correct / abnormal tool usage behavior; Images depicting the interaction between workers and objects in other construction scenarios.

4. The risk behavior identification method as described in claim 1, characterized in that, The target detection model used is the YOLOv5s network model.

5. The risk behavior identification method as described in claim 1, characterized in that, The human target detection data includes human bounding box data and object bounding box data.

6. The risk behavior identification method as described in claim 1, characterized in that, The step of creating corresponding grayscale image masks based on the target detection data and inputting them into the target position relationship module to obtain the spatial position relationship features of the target includes: Initialize two zero matrices with the same size as the width W and height H of the target detection data output features; Set the area covered by the human body bounding box or object bounding box to 1 to obtain the mask matrix; The corresponding grayscale image mask of the human body bounding box and the grayscale image mask of the object bounding box are superimposed to form a mask of size [size missing]. Three-dimensional matrix data; The spatial location features are obtained by inputting the three-dimensional matrix data into a convolutional neural network and a fully connected network.

7. The risk behavior identification method as described in claim 1, characterized in that, The strategy for establishing character interaction relationships involves multiplying the character's appearance features by their spatial location relationship features to obtain the final predicted features. Based on these features, the confidence score for each type of character interaction behavior is output, including: ; Where M represents human appearance features, O represents object appearance features, R represents relative position features, and output represents the confidence level of each type of human interaction behavior.

Images