Method and system for automatically detecting unsafe behaviors of workers based on machine vision

Abstract

The invention discloses a method and system for automatically detecting unsafe behaviors of workers based on machine vision, and relates to the field of industrial safety, and the method comprises the steps: collecting and preprocessing the behavior data of the workers, the behavior data of the workers comprising position information, equipment state, certificate state and motion trail; performing unsafe behavior detection on the preprocessed operator behavior data to generate a behavior target detection frame; inputting the behavior target detection frame into a target tracking algorithm to perform multi-target tracking, and generating a personnel movement track data set; performing multivariate statistical analysis on the behavior feature vector by using a quantitative theory to generate an unsafe behavior reaction matrix; and evaluating a risk level in the unsafe behavior reaction matrix through a fuzzy analytic hierarchy process, and performing safety early warning notification according to the risk level. According to the method, multivariate statistical analysis is carried out on the behavior feature vectors through a quantification theory, and deep quantification and comprehensive evaluation of the unsafe behavior features of the workers are realized.

Description

Technical Field

[0001] The present invention relates to the field of industrial safety, and in particular to a method and system for automatically detecting unsafe worker behaviors based on machine vision. Background Art

[0002] In recent years, computer vision technology has been widely used in the field of industrial safety for behavior recognition and analysis. With the large-scale application of the Transformer algorithm and machine vision, the accuracy of feature extraction has been greatly improved. Detecting unsafe worker behaviors in industrial scenarios has become more important. Conventional methods are mainly based on a multi-stage detection framework. The YOLO algorithm is used to locate workers in real time under fixed lighting scenes and clear viewing conditions and collect rigid behaviors such as falls and climbs. The LSTM time series algorithm is used to classify the actions of cropped human body areas to effectively identify unsafe worker behaviors.

[0003] However, existing methods still face limitations in complex industrial scenarios. When the working environment has dynamic lighting changes, non-fixed viewing angles, and partial occlusion of the human body, the target detection device is prone to missed detections and mis-segmentation, resulting in feature loss in the behavior classifier input data. In particular, when identifying non-rigid actions such as bending over to pick up objects and operating equipment with one hand, the motion continuity modeling fails due to the loss of key frames, making it difficult to meet the reliability requirements of real-time warnings of unsafe behaviors in high-risk working environments. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method for automatically detecting unsafe behaviors of workers based on machine vision to solve the problem of difficulty in providing real-time early warning of unsafe behaviors in high-risk working environments.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In the first aspect, the present invention provides a method for automatic detection of unsafe behaviors of workers based on machine vision, which includes collecting and preprocessing the behavior data of operating personnel, wherein the behavior data of the operating personnel include location information, equipment status, certificate status and motion trajectory; performing unsafe behavior detection on the preprocessed behavior data of the operating personnel to generate a behavior target detection frame; inputting the behavior target detection frame into a target tracking algorithm to perform multi-target tracking to generate a personnel motion trajectory data set; extracting behavior feature vectors from the personnel motion trajectory data set, performing multivariate statistical analysis on the behavior feature vectors using quantification theory to generate an unsafe behavior response matrix; evaluating the risk level in the unsafe behavior response matrix through the fuzzy hierarchical analysis method, and issuing a safety warning notification based on the risk level.

[0008] As a preferred solution of the method for automatically detecting unsafe behaviors of workers based on machine vision according to the present invention, the unsafe behaviors are detected on the pre-processed operator behavior data to generate a behavior target detection frame. The specific steps are as follows:

[0009] Define unsafe behavior types based on historical operator behavior data;

[0010] The convolutional neural network of the YOLOv5 algorithm is used to extract features from the pre-processed operator behavior data and generate a multi-scale behavior feature map;

[0011] Perform target detection on the multi-scale behavior feature graph and identify the specific type of unsafe behavior of the operator based on the target detection results;

[0012] Based on the bounding box regression mechanism of the YOLOv5 algorithm, bounding box regression is performed on the specific type of unsafe behavior of the operator to generate the behavior target detection frame;

[0013] The behavior target detection frame refers to the coordinate information of the operator and the specific type of unsafe behavior.

[0014] As a preferred solution of the method for automatically detecting unsafe worker behaviors based on machine vision described in the present invention, the behavior target detection frame is input into the target tracking algorithm for multi-target tracking to generate a personnel motion trajectory dataset. The specific steps are as follows:

[0015] Assign a target identifier to each worker based on the behavioral target detection box;

[0016] The behavioral target detection frames are input into the target tracking algorithm, and multiple behavioral target detection frames are grouped into independent target clusters through multi-target clustering;

[0017] Kalman filtering is used to predict the motion trajectory of independent target clusters and generate initial motion trajectory;

[0018] The initial motion trajectory is corrected across modalities using the weighted average method;

[0019] The target identifier is bound to the corrected initial motion trajectory to generate a personnel motion trajectory dataset.

[0020] As a preferred solution of the method for automatic detection of unsafe worker behaviors based on machine vision described in the present invention, the cross-modal trajectory correction refers to the process of smoothing and optimizing the initial motion trajectory by fusing the coordinate information and position information of the behavior target detection frame through the weighted averaging method.

[0021] As a preferred solution of the method for automatically detecting unsafe worker behaviors based on machine vision of the present invention, the specific steps of extracting behavioral feature vectors from the personnel motion trajectory data set are as follows:

[0022] Time slice the personnel motion trajectory dataset to obtain motion trajectory points;

[0023] Calculate the operator's movement speed and the time they stay in the dangerous area based on the movement trajectory points;

[0024] Count the frequency and trajectory movement patterns of unsafe behaviors based on the type of unsafe behaviors and movement speed;

[0025] The operator's movement speed, stay time in the dangerous area, unsafe behavior frequency and trajectory movement pattern are normalized and fused into a behavior feature vector.

[0026] As a preferred solution of the method for automatically detecting unsafe worker behaviors based on machine vision of the present invention, wherein: the method uses quantification theory to perform multivariate statistical analysis on the behavior feature vector to generate an unsafe behavior response matrix, and the specific steps are as follows:

[0027] Use quantification theory to convert behavioral feature vectors into quantitative scores;

[0028] Correlation coefficients between quantitative scores and unsafe behavior types were calculated;

[0029] The variation components of the behavioral feature vector are extracted through principal component analysis;

[0030] According to the variation components and correlation coefficients, the unsafe behavior response scores of each operator are calculated, and the unsafe behavior response scores are integrated into an unsafe behavior response matrix through a matrix structure.

[0031] As a preferred solution of the method for automatically detecting unsafe behaviors of workers based on machine vision according to the present invention, the risk level in the unsafe behavior response matrix is ​​evaluated by the fuzzy analytic hierarchy process, and the specific steps are as follows:

[0032] Construct a safety assessment model based on the fuzzy analytic hierarchy process (FAHP);

[0033] The safety assessment model is divided into the scheme layer, the criterion layer and the target layer;

[0034] The solution layer receives the unsafe behavior response matrix;

[0035] The criterion layer uses the unsafe behavior response matrix to determine the behavior feature vector index, and performs fuzzy processing to generate a fuzzy judgment matrix;

[0036] According to the fuzzy judgment matrix, the fuzzy weight of each behavior feature vector indicator is calculated to obtain the risk level score of the unsafe behavior;

[0037] The target layer divides the risk level of unsafe behaviors according to the risk level score.

[0038] In a second aspect, the present invention provides a system for automatically detecting unsafe worker behaviors based on machine vision, comprising:

[0039] The acquisition module is used to collect and pre-process operator behavior data, including location information, equipment status, certificate status, and movement trajectory;

[0040] The detection module is used to detect unsafe behaviors on the pre-processed operator behavior data and generate behavior target detection frames;

[0041] The tracking module is used to input the behavior target detection frame into the target tracking algorithm for multi-target tracking and generate a personnel motion trajectory dataset;

[0042] The analysis module is used to extract behavioral feature vectors from the personnel movement trajectory data set, perform multivariate statistical analysis on the behavioral feature vectors using quantitative theory, and generate an unsafe behavior response matrix;

[0043] The evaluation module is used to evaluate the risk level in the unsafe behavior response matrix through the fuzzy hierarchical analysis method and issue safety warning notifications based on the risk level.

[0044] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the method for automatic detection of unsafe worker behavior based on machine vision as described in the first aspect of the present invention is implemented.

[0045] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the method for automatic detection of unsafe worker behavior based on machine vision as described in the first aspect of the present invention.

[0046] The beneficial effects of the present invention are as follows: the present invention conducts multivariate statistical analysis on behavioral feature vectors through quantification theory, realizes in-depth quantification and comprehensive evaluation of workers' unsafe behavioral characteristics, and can capture the complex characteristics of workers' behavior from multiple dimensions by converting the behavioral feature vectors into quantitative scores, effectively integrates the correlation between different characteristics, and uses principal component analysis to extract the core variation components of behavioral characteristics, thereby improving analysis efficiency and avoiding the subjectivity of artificially setting weights, enhancing objectivity and reliability, and providing accurate and comprehensive data support for subsequent risk assessments. The unsafe behavior response matrix can clearly reflect the contribution of different behavioral characteristics to risks, facilitates rapid problem location and the adoption of targeted measures, provides a scientific basis for safe production management, and improves the level of safety monitoring at the work site. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 Flowchart of the automatic detection method of unsafe worker behaviors based on machine vision.

[0049] Figure 2 Schematic diagram of the automatic detection system of unsafe worker behaviors based on machine vision.

[0050] Figure 3 Flowchart for detecting bounding boxes for behavioral targets.

[0051] Figure 4 Flowchart of behavioral feature vector. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0053] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0054] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0055] Reference Figures 1 to 4 , is an embodiment of the present invention, which provides a method for automatically detecting unsafe worker behaviors based on machine vision, comprising the following steps:

[0056] S1. Collect operator behavior data and perform preprocessing.

[0057] Operator behavior data includes location information, equipment status, certificate status, and movement trajectory;

[0058] Deploy indoor and outdoor positioning devices at construction sites. Using ultra-wideband (UWB) positioning technology, install UWB base stations and UWB tags worn by workers to record workers' coordinate data in real time and obtain their location information.

[0059] For example, four UWB base stations are deployed at the construction site to ensure that they cover the entire work area (the specific number of UWB base stations is adjusted according to the actual area of ​​the construction site). Each worker is equipped with a unique UWB tag, which is bound to the worker's ID and fixed to the worker's helmet. While the worker is working, the UWB base station scans the UWB tag at regular intervals (e.g., every five minutes) to record the worker's location information.

[0060] RFID tags and IoT sensors are installed on hard hats, safety belts, and protective clothing. The RFID tags are linked to the worker's ID. RFID readers are deployed in key areas of the construction site (such as entrances, high-voltage, and high-altitude areas) to detect whether workers have entered key areas of the construction site. The status of workers' equipment is monitored through IoT sensors.

[0061] Equipment status includes helmet wearing status, safety belt connection status and protective clothing wearing status;

[0062] Install a high-definition camera at the entrance of the work site. Use facial recognition to compare the facial data collected by the high-definition camera with the pre-stored operator ID database to query the validity of the ID and obtain the ID status of the operator.

[0063] Based on the collected position information, the coordinate data of the operator is arranged in sequence by timestamp, the displacement between consecutive time points is calculated, and the motion trajectory is generated;

[0064] Preprocessing includes data cleaning, missing value processing, outlier detection and standardization, as follows:

[0065] Check the collected location information, equipment status, document status, and movement trajectory, and delete records with missing timestamps and operator ID mismatches;

[0066] Use linear interpolation to fill missing coordinate values ​​in location information, and use forward filling to fill missing values ​​in equipment status and document status;

[0067] The coordinate data in the position information is detected based on the 3σ principle for outliers, and coordinate values ​​exceeding 3 times the standard deviation are eliminated. Speed ​​values ​​outside the reasonable range in the motion trajectory are also eliminated, such as speed values ​​greater than 10 m / s.

[0068] Normalize the position information after outlier detection using the formula:

[0069] ;

[0070] in, represents the normalized coordinate value, Represents the original coordinate value, Indicates the minimum value of the original coordinate value, Indicates the maximum value of the original coordinate value;

[0071] The pre-processed location information, equipment status, document status, and motion trajectory are aligned according to timestamps and personnel IDs and merged into operator behavior data.

[0072] S2. Perform unsafe behavior detection on the pre-processed operator behavior data and generate a behavior target detection frame.

[0073] Based on historical worker behavior data, unsafe behaviors are defined as entering the construction site without training, starting work without a briefing, not wearing a safety helmet or safety belt correctly, not holding an operator's license for special work, not holding a permit for hot work, not holding a real-name ID for construction site entry, smoking, working under the influence of alcohol, throwing objects from heights, and trespassing into cordoned and isolated areas.

[0074] Load the pre-trained weight file of the YOLOv5 algorithm from the Ultralytics official open source repository and pre-train the YOLOv5 algorithm using historical worker behavior data, as follows:

[0075] Allocate historical operator behavior data into a training set and a validation set in proportion (e.g., 80%:20%).

[0076] Run the YOLOv5 algorithm using the training set, update the pre-trained weights of the YOLOv5 algorithm, obtain new YOLOv5 algorithm weights, and optimize the YOLOv5 algorithm's ability to detect unsafe behaviors;

[0077] After each round of training, the performance of the YOLOv5 algorithm is evaluated using the validation set and the mAP indicator is used to calculate the validation set accuracy of the target detection box;

[0078] Compare the validation set accuracy after multiple rounds of training, and select the pre-trained weights with the highest validation set accuracy to update the YOLOv5 algorithm;

[0079] It should be noted that the updated YOLOv5 algorithm has used the training set to update weights and has the ability to detect unsafe behaviors on construction sites;

[0080] The convolutional neural network of the updated YOLOv5 algorithm is used to detect the pre-processed operator behavior data and generate a multi-scale behavior feature map, as shown below:

[0081] The convolutional neural network has an input layer, three convolutional layers, and an output layer;

[0082] The input layer inputs the pre-processed operator behavior data into the CSPDarknet53 backbone network of the YOLOv5 algorithm;

[0083] The first convolution layer uses multiple convolution kernels (e.g., 32 3×3 convolution kernels) to perform convolution operations on the preprocessed operator behavior data to extract behavioral features (e.g., color, texture) of the operator behavior data.

[0084] Use SiLU activation function to optimize the extracted behavioral features, retain negative information, and avoid gradient disappearance;

[0085] Use 2×2 maximum pooling (window size is 2×2, stride is 2), take the maximum value in each 2×2 area, reduce the size of the behavior feature map, and obtain a behavior feature map of half size;

[0086] The second convolutional layer uses 64 3×3 convolution kernels (with a stride of 2) to perform another convolution operation on the preprocessed worker behavior data to extract deep behavioral features (such as the curved edge of the helmet and the human body contour);

[0087] Use SiLU activation function to process deep behavioral features to avoid gradient disappearance;

[0088] Among them, the convolution kernel with a stride of 2 can replace the pooling to perform dimensionality reduction processing and obtain a behavioral feature map of 1 / 4 size;

[0089] The third convolutional layer uses 128 3x3 convolution kernels (with a stride of 1) to convolve the 1 / 4-sized behavior feature map to enhance feature expression and identify more detailed structures (such as seat belt status).

[0090] Use the SiLU activation function to optimize the more detailed structure and then insert it into the 2×2 maximum pooling to further reduce the dimension (for example, from 160x160 to 80x80), generating a 1 / 8 size behavioral feature map;

[0091] The output layer integrates the behavioral feature maps of multiple convolutional layers to generate multi-scale behavioral feature maps;

[0092] The detection head of the YOLOv5 algorithm is used to perform target detection on the multi-scale behavior feature map. Based on the target detection results, the specific type of unsafe behavior of the operator is identified, as follows:

[0093] Divide the multi-scale behavioral feature map into multiple grids. For example, a 52×52 multi-scale behavioral feature map is divided into a grid of 52 rows and 52 columns. In each grid cell, multiple bounding boxes (usually 3 bounding boxes with different aspect ratios) are predicted based on the bounding box regression mechanism of the YOLOv5 algorithm.

[0094] The formula for predicting the bounding box is,

[0095] ;

[0096] in, represents the predicted bounding box, represents the sigmoid function, Indicates the offset of the YOLOv5 algorithm, Indicates the coordinates of the upper left corner of the grid;

[0097] It should be noted that the formula for predicting the bounding box is the internal calculation formula of the YOLOv5 algorithm, not the formula defined in this technical solution;

[0098] Use non-maximum suppression (NMS) to filter the predicted bounding box and obtain the final bounding box.

[0099] The screening method is to calculate the intersection over union (IoU) of the predicted bounding boxes for each defined unsafe behavior type, that is, the ratio of the intersection area to the union area of ​​the bounding boxes. The IoU ratios are then sorted from high to low, and bounding boxes with IoU greater than 0.4 are eliminated to obtain the filtered bounding boxes.

[0100] The intersection area refers to the overlapping area of ​​two bounding boxes, and the union area refers to the area after the two bounding boxes are merged;

[0101] Align the filtered bounding box with the operator's behavior data using timestamps to generate a behavior target detection frame.

[0102] It should be noted that the YOLOv5 algorithm is a deep learning model for target detection based on convolutional neural network (CNN), which achieves target positioning and classification through feature extraction and bounding box prediction.

[0103] S3. Input the behavior target detection frame into the target tracking algorithm for multi-target tracking to generate a personnel motion trajectory dataset.

[0104] The coordinates (x, y) of the center point of the target detection box are expressed as discrete position points (t, x, y) of the time series to simulate the event stream of the event camera, where t represents the timestamp, and x and y represent the coordinates of the center point of the target detection box, specifically:

[0105] Extract the center point (x, y) of each target detection box, ignoring the width w and height h (the reason for ignoring them is because they are used for target tracking, so we focus on the coordinate position rather than the size of the target detection box);

[0106] Organize the center point coordinates into a time series according to timestamp t;

[0107] Based on the behavioral target detection frame, the operator ID of the behavioral target detection frame is used as the target identifier;

[0108] The coordinate information of the behavioral target detection frame is input into the target tracking algorithm, and the intersection-over-union (IOU) ratio is used to perform multi-target clustering, and multiple behavioral target detection frames are grouped into independent target clusters. Specifically:

[0109] Based on the target identifier, the target tracking algorithm is used to track the target position change events of the target detection frame coordinate information in real time, and multi-target clustering is performed on the target detection frames where the target position changes.

[0110] A target position change event refers to a change in the coordinate position of the behavioral target detection frame between adjacent time points that exceeds the change threshold, indicating that the operator has moved;

[0111] The change threshold is set to 0.01 based on the historical location information of workers, which is equivalent to a movement distance of 0.5 meters in the actual construction site;

[0112] Calculate the Euclidean distance between adjacent time points of the target detection frame. The formula is:

[0113] ;

[0114] in, represents the Euclidean distance between adjacent time points, and Indicates the coordinate value of the previous time point, and Indicates the coordinate value of the next time point;

[0115] when When it is >0.01, it is recorded as a position change event;

[0116] Traverse the behavior target detection box in timestamp order. If the behavior target detection box is missing (such as occlusion), skip the current time point.

[0117] Density clustering is used to group location change events and generate independent target clusters;

[0118] Kalman filtering is used to predict independent target clusters and generate initial motion trajectories, specifically:

[0119] Initialize the target position and velocity states of the independent target cluster, assuming that the independent target cluster moves at a constant speed;

[0120] Update the target position according to the coordinates of the YOLOv5 detection frame and the behavioral target detection frame, and integrate the noise characteristics of the YOLOv5 detection frame and the behavioral target detection frame (the YOLOv5 detection frame has higher noise, while the behavioral target detection frame has lower noise);

[0121] Iteratively update the state estimate to generate a smooth sequence of target positions;

[0122] Use linear interpolation to fill in the missing points in the target position sequence to form the initial motion trajectory; when there are more than 5 missing points, mark it as an incomplete motion trajectory and remove it;

[0123] The initial motion trajectory is corrected across modalities using the weighted average method to generate a continuous and smooth motion trajectory.

[0124] The target identifier is bound to the corrected initial motion trajectory to generate a personnel motion trajectory dataset.

[0125] S4. Extract behavioral feature vectors from the personnel movement trajectory dataset, perform multivariate statistical analysis on the behavioral feature vectors using quantitative theory, and generate an unsafe behavior response matrix.

[0126] Slice the personnel motion trajectory dataset into sliding time windows (e.g., 10 seconds) and extract the motion trajectory points within each time window.

[0127] The operator's movement speed is calculated based on the displacement and time difference of the movement trajectory points. The formula is:

[0128] ;

[0129] in, Indicates the speed of movement, represents the Euclidean distance between adjacent time points, Indicates the timestamp of the previous time point. The timestamp of the next time point;

[0130] According to the coordinates of the dangerous areas defined at the construction site (such as high-voltage areas and high-altitude working areas, which are defined as rectangular areas), it is determined whether the trajectory point is located in the dangerous area. If it is located in the dangerous area, the cumulative time the trajectory point is in the dangerous area in each time window is counted to obtain the dangerous area residence time (in seconds).

[0131] Based on the type of unsafe behavior and movement speed, the number of unsafe behaviors (such as not wearing a helmet and trespassing in a warning area) within the time window is counted to obtain the frequency of unsafe behaviors;

[0132] Analyze the direction changes of trajectory points and determine the movement mode. The movement modes are divided into straight lines (e.g., cumulative angle change ≤ 15°, small direction change), curved lines (e.g., 15° < angle change ≤ 60°, moderate direction change), and wandering (e.g., angle change > 60°, large direction change).

[0133] The operator's movement speed, time spent in dangerous areas, frequency of unsafe behaviors, and trajectory movement patterns are normalized and scaled to the range of 0 to 1, and then fused into a behavioral feature vector.

[0134] Apply quantitative theory to conduct multivariate statistical analysis on behavioral feature vectors and generate an unsafe behavior response matrix, specifically:

[0135] A behavioral feature matrix was created using the quantification theory type I, where the rows represent the time window of each operator and the columns represent the behavioral feature vector (movement speed, time spent in the hazardous area, frequency of unsafe behaviors, and trajectory movement pattern).

[0136] Each matrix element is the normalized value of the behavior feature vector in the corresponding time window, that is, the quantitative score of the behavior feature vector;

[0137] Analyze the correlation coefficient between the quantitative scores in the behavioral feature matrix and the unsafe behavior types to determine the contribution of the behavioral feature vector to the unsafe behavior;

[0138] The formula for the correlation coefficient is:

[0139] ;

[0140] in, represents the correlation coefficient, Indicates the total number of time windows, represents a quantitative score, Indicates the number of times unsafe behaviors occur. represents the mean of quantitative scores, represents the mean number of unsafe behaviors, represents the cumulative sum of all time windows, An index variable representing the time window;

[0141] Use principal component analysis to construct the response matrix of the behavioral feature vector and extract the variation components of the behavioral feature vector. For example, select the variation components with a cumulative contribution of 85% to reduce the dimension of the behavioral feature vector and retain the main information.

[0142] Each principal component is a linear combination of the original features, representing the influencing factors of unsafe behavior;

[0143] According to the response matrix and correlation coefficient, the unsafe behavior response score of each operator is calculated to generate the unsafe behavior response matrix, which is as follows:

[0144] Project the behavioral feature vector into the principal component space to generate the unsafe behavior response score, which is:

[0145] ;

[0146] in, represents the unsafe behavior response score, represents principal component analysis, represents the correlation between the variation component and the number of unsafe behaviors, represents the number of variation components of the behavioral feature vector, An index variable representing the sequence number of the mutation component;

[0147] The unsafe behavior response score reflects the contribution of the operator's behavior to the unsafe behavior in each time window;

[0148] Use the unsafe behavior response scores to construct an unsafe behavior response matrix, where the rows of the unsafe behavior response matrix represent the time windows of the operators, and the columns represent the unsafe behavior response scores of the operators;

[0149] The matrix elements are the scores of each time window on the corresponding principal component, quantifying the characteristic contribution of unsafe behaviors.

[0150] S5. Evaluate the risk level in the unsafe behavior response matrix through fuzzy hierarchical analysis method, and issue safety warning notifications based on the risk level.

[0151] A safety assessment model is constructed based on the fuzzy analytic hierarchy process (FAHP), which is divided into the scheme layer, the criterion layer and the target layer.

[0152] The solution layer receives the unsafe behavior response matrix;

[0153] The criterion layer uses the unsafe behavior response matrix to determine the behavior feature vector index, and uses the triangular membership function to fuzzify the behavior feature vector index to generate a fuzzy judgment matrix;

[0154] The behavioral feature vector indicators refer to the normalized movement speed, the time spent in the dangerous area, the frequency of unsafe behaviors, and the trajectory movement pattern;

[0155] The parameters of the triangular membership function for fuzzy processing of each behavioral feature are set as follows:

[0156] Movement speed is divided into three levels: low, medium, and high, corresponding to the low, medium, and high intervals of the normalized movement speed value;

[0157] The time spent in the danger zone is divided into three levels: low, medium, and high, which correspond to the short, medium, and long intervals of the normalized time spent in the danger zone.

[0158] The frequency of unsafe behaviors is divided into three levels: low, medium, and high, which correspond to the low, medium, and high intervals of the normalized frequency of unsafe behaviors.

[0159] The trajectory motion mode is divided into three levels: low, medium and high, corresponding to straight line, curve and wandering modes respectively;

[0160] Fuzzy processing refers to using triangular membership function to obtain the membership of the behavior feature vector index in each time window, and integrating it into a fuzzy judgment matrix in sequence according to the membership degree;

[0161] According to the fuzzy judgment matrix, the fuzzy number multiplication operation is performed on each behavior feature vector index to obtain the fuzzy weight;

[0162] The fuzzy weight and the behavioral feature vector index are weighted and summed to obtain the risk level score of the unsafe behavior;

[0163] The target layer divides the risk level of unsafe behaviors according to the risk level score;

[0164] The rules for classifying risk levels are as follows:

[0165] By analyzing historical operator behavior data and the frequency of unsafe behaviors, and combining fuzzy weights, u and w are set as the dividing points to distinguish high, medium and low risks. The specific risk thresholds are adjusted according to actual conditions.

[0166] When the risk level score is ≥ w, the risk level is high risk;

[0167] When u<risk level score<w, the risk level is medium risk;

[0168] When the risk level score is less than u, the risk level is low risk;

[0169] According to the risk level, security warning notifications are issued, specifically:

[0170] When the risk level is high, an on-site alarm is triggered and a text message is sent to the management personnel;

[0171] When the risk level is medium, early warning information will be pushed to operators and managers through the APP;

[0172] When the risk level is low, record security logs and generate security reports regularly.

[0173] This embodiment also provides a system for automatically detecting unsafe worker behaviors based on machine vision, including:

[0174] The acquisition module is used to collect and pre-process operator behavior data, including location information, equipment status, certificate status, and movement trajectory;

[0175] The detection module is used to detect unsafe behaviors on the pre-processed operator behavior data and generate behavior target detection frames;

[0176] The tracking module is used to input the behavior target detection frame into the target tracking algorithm for multi-target tracking and generate a personnel motion trajectory dataset;

[0177] The analysis module is used to extract behavioral feature vectors from the personnel movement trajectory data set, perform multivariate statistical analysis on the behavioral feature vectors using quantitative theory, and generate an unsafe behavior response matrix;

[0178] The evaluation module is used to evaluate the risk level in the unsafe behavior response matrix through the fuzzy hierarchical analysis method and issue safety warning notifications based on the risk level.

[0179] This embodiment also provides a computer device, which is suitable for the automatic detection method of unsafe worker behaviors based on machine vision, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the automatic detection method of unsafe worker behaviors based on machine vision proposed in the above embodiment.

[0180] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0181] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for automatically detecting unsafe worker behaviors based on machine vision as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0182] In summary, the present invention conducts multivariate statistical analysis on behavioral feature vectors through quantitative theory, realizes in-depth quantification and comprehensive evaluation of workers' unsafe behavioral characteristics, and can capture the complex characteristics of workers' behavior from multiple dimensions by converting behavioral feature vectors into quantitative scores, effectively integrating the correlation between different characteristics, and using principal component analysis to extract the core variation components of behavioral characteristics, thereby improving analysis efficiency and avoiding the subjectivity of artificially set weights, enhancing objectivity and reliability, and providing accurate and comprehensive data support for subsequent risk assessments. The unsafe behavior response matrix can clearly reflect the contribution of different behavioral characteristics to risks, facilitate rapid problem location and take targeted measures, provide a scientific basis for safe production management, and improve the level of safety monitoring at the work site.

[0183] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for automatically detecting unsafe worker behaviors based on machine vision, characterized by: include, Collecting and preprocessing operator behavior data, including location information, equipment status, certificate status, and movement trajectory; Perform unsafe behavior detection on the pre-processed operator behavior data and generate behavior target detection frames; Input the behavioral target detection frame into the target tracking algorithm for multi-target tracking to generate a personnel motion trajectory dataset; Extract behavioral feature vectors from the personnel movement trajectory data set, perform multivariate statistical analysis on the behavioral feature vectors using quantitative theory, and generate an unsafe behavior response matrix; The risk level in the unsafe behavior response matrix is ​​evaluated by fuzzy analytic hierarchy process, and safety warning notifications are issued according to the risk level.

2. The method for automatically detecting unsafe worker behaviors based on machine vision according to claim 1, characterized in that: The unsafe behavior detection is performed on the pre-processed operator behavior data to generate a behavior target detection frame. The specific steps are: Define unsafe behavior types based on historical operator behavior data; The convolutional neural network of the YOLOv5 algorithm is used to extract features from the pre-processed operator behavior data and generate a multi-scale behavior feature map; Perform target detection on the multi-scale behavior feature graph and identify the specific type of unsafe behavior of the operator based on the target detection results; Based on the bounding box regression mechanism of the YOLOv5 algorithm, bounding box regression is performed on the specific type of unsafe behavior of the operator to generate the behavior target detection box; The behavior target detection frame refers to the coordinate information of the operator and the specific type of unsafe behavior.

3. The method for automatically detecting unsafe worker behaviors based on machine vision according to claim 1, wherein: The behavior target detection frame is input into the target tracking algorithm for multi-target tracking to generate a personnel motion trajectory dataset. The specific steps are: Assign a target identifier to each worker based on the behavioral target detection box; The behavioral target detection frames are input into the target tracking algorithm, and multiple behavioral target detection frames are grouped into independent target clusters through multi-target clustering; Kalman filtering is used to predict the motion trajectory of independent target clusters and generate initial motion trajectory; The initial motion trajectory is corrected across modalities using the weighted average method; The target identifier is bound to the corrected initial motion trajectory to generate a personnel motion trajectory dataset.

4. The method for automatically detecting unsafe worker behaviors based on machine vision according to claim 3, wherein: The cross-modal trajectory correction refers to the process of smoothing and optimizing the initial motion trajectory by fusing the coordinate information and position information of the behavioral target detection frame through the weighted averaging method.

5. The method for automatically detecting unsafe worker behaviors based on machine vision according to claim 1, wherein: The specific steps of extracting the behavioral feature vector from the personnel motion trajectory data set are as follows: Time slice the personnel motion trajectory dataset to obtain motion trajectory points; Calculate the operator's movement speed and the time they stay in the dangerous area based on the movement trajectory points; Count the frequency and trajectory movement patterns of unsafe behaviors based on the type of unsafe behaviors and movement speed; The operator's movement speed, stay time in the dangerous area, unsafe behavior frequency and trajectory movement pattern are normalized and fused into a behavior feature vector.

6. The method for automatically detecting unsafe worker behaviors based on machine vision according to claim 1, wherein: The method uses quantitative theory to perform multivariate statistical analysis on the behavior feature vector to generate an unsafe behavior response matrix. The specific steps are: Use quantification theory to convert behavioral feature vectors into quantitative scores; Correlation coefficients between quantitative scores and unsafe behavior types were calculated; The variation components of the behavioral feature vector are extracted through principal component analysis; According to the variation components and correlation coefficients, the unsafe behavior response scores of each operator are calculated, and the unsafe behavior response scores are integrated into an unsafe behavior response matrix through a matrix structure.

7. The method for automatically detecting unsafe worker behaviors based on machine vision according to claim 1, wherein: The risk level in the unsafe behavior response matrix is ​​evaluated by the fuzzy analytic hierarchy process, and the specific steps are: Construct a safety assessment model based on the fuzzy analytic hierarchy process (FAHP); The safety assessment model is divided into the scheme layer, the criterion layer and the target layer; The solution layer receives the unsafe behavior response matrix; The criterion layer uses the unsafe behavior response matrix to determine the behavior feature vector index, and performs fuzzy processing to generate a fuzzy judgment matrix; According to the fuzzy judgment matrix, the fuzzy weight of each behavior feature vector indicator is calculated to obtain the risk level score of the unsafe behavior; The target layer divides the risk level of unsafe behaviors according to the risk level score.

8. A system for automatically detecting unsafe worker behaviors based on machine vision, based on the method for automatically detecting unsafe worker behaviors based on machine vision according to any one of claims 1 to 7, characterized in that: include, The acquisition module is used to collect and pre-process operator behavior data, including location information, equipment status, certificate status, and movement trajectory; The detection module is used to detect unsafe behaviors on the pre-processed operator behavior data and generate behavior target detection frames; The tracking module is used to input the behavior target detection frame into the target tracking algorithm for multi-target tracking and generate a personnel motion trajectory dataset; The analysis module is used to extract behavioral feature vectors from the personnel movement trajectory data set, perform multivariate statistical analysis on the behavioral feature vectors using quantitative theory, and generate an unsafe behavior response matrix; The evaluation module is used to evaluate the risk level in the unsafe behavior response matrix through the fuzzy hierarchical analysis method and issue safety warning notifications based on the risk level.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for automatically detecting unsafe worker behaviors based on machine vision according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for automatically detecting unsafe worker behaviors based on machine vision according to any one of claims 1 to 7 are implemented.

Images