Industrial scene-based abnormal behavior detection and intelligent analysis method

Abstract

The invention relates to the technical field of machine vision, in particular to a system and a method for realizing abnormal behavior detection and intelligent analysis in an industrial scene based on a machine vision technology. According to the system, workshop image data are collected through a high-precision camera, and the action and posture of a worker and the operation state of equipment are analyzed through an image recognition algorithm. In order to ensure accurate detection, a multivariate feature model is constructed to mine behavior features, and a deep learning neural network is adopted to optimize recognition accuracy and speed. The method is characterized in that the image quality is improved through data preprocessing, accurate behavior positioning and recognition are realized through fusion of multiple algorithms, and an intelligent analysis module is designed to generate a report to assist decision making. In industrial production, the system can timely discover abnormal conditions such as illegal operation, deviation from a safe area, fatigue operation and the like, give an alarm and provide an analysis report, so that accidents are avoided, and production benefits are improved. Through testing, the abnormal behavior detection accuracy is high, the system response is rapid, and industrial safety management and production process optimization can be effectively assisted.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and deep learning technology, and in particular to an abnormal behavior detection system and intelligent analysis method based on industrial scenarios Background Art

[0002] With the acceleration of the industrialization process, industrial production scenes are becoming increasingly complex. Some violations by employees, such as smoking, playing with mobile phones, sleeping on the job, and not wearing safety equipment, not only pose a threat to their own safety, but may also affect production efficiency and even cause serious safety accidents. Traditional monitoring methods mainly rely on manual inspections, hardware equipment, and wireless signals or video monitoring, but these methods have many limitations. Manual inspections are time-consuming and labor-intensive, difficult to fully cover, and prone to missed inspections or false alarms; video monitoring requires a large amount of storage and bandwidth, and manual analysis is inefficient and prone to errors; hardware equipment and wireless signal detection methods have poor adaptability and are not effective in special scenarios. In addition, traditional methods lack timeliness and initiative, and are mostly processed after the behavior occurs, making it difficult to play a preventive warning role.

[0003] With the rapid development of deep learning technology, computer vision technology has made significant progress, and many scholars have applied it to fields such as behavior recognition. However, in the detection of abnormal behavior in industrial scenarios, there are still many challenges, such as the accuracy of small target detection (such as cigarette butts, mobile phones, etc.), the robustness of behavior recognition in complex environments, and the real-time and accuracy of the detection system. These problems need to be solved urgently. At the same time, the existing technology lacks a unified standard and evaluation system, and there are insufficient public data sets and benchmarks. The understanding of the nature and mechanism of abnormal behavior is not deep enough, which limits the development of more effective and general detection methods. Summary of the invention

[0004] The present invention aims to overcome the shortcomings of the prior art and provide an abnormal behavior detection system and an intelligent analysis method based on industrial scenarios, so as to realize efficient, accurate and real-time monitoring and early warning of abnormal behaviors in industrial scenarios and improve the level of industrial production safety management.

[0005] Step 1: Data collection and preprocessing

[0006] Collect image and video data of human behavior in industrial scenarios through online search, AIGC, and simulation scenarios, covering normal behavior and abnormal behaviors such as smoking, playing with mobile phones, sleeping on the job, and not wearing safety clothing. Accurately annotate the collected data to clarify whether there is abnormal behavior in each image or video clip and the specific type of abnormal behavior (such as smoking, playing with mobile phones, etc.), providing accurate supervision information for subsequent model training. Crop and scale all images to unify their sizes to the model adaptation size to ensure consistent input data formats. According to model requirements, convert images into appropriate color spaces to optimize data feature expression.

[0007] Step 2: Model selection and improvement

[0008] An improved model based on the YOLO framework is constructed and integrated with the BGCNet network. BGCNet effectively captures global context information and improves the ability to detect complex behaviors through unique context modeling, feature conversion, and fusion mechanisms, with small number of parameters and computation. The lightweight backbone network ShuffleNetV2 is introduced to replace the traditional backbone network, reduce computational complexity, and ensure the real-time response performance of the model on resource-constrained devices while maintaining detection accuracy.

[0009] The labeled data is divided into training set, validation set and test set in proportion of 70%, 15% and 15% to ensure the effectiveness of model training and evaluation. A variety of loss functions are integrated, including weighted binary cross entropy loss, smooth L1 loss, etc., which are used to constrain the prediction of different aspects such as model center, displacement, segment matching, etc. At the same time, geometric information such as needle axis length, angle, segment segmentation, etc. are considered, and multiple loss collaborative training models are designed to ensure accurate prediction of abnormal behavior. Adaptive learning rate strategies such as cosine annealing decay and Adam optimizer are used to set reasonable parameters, control the training batch size and number of iterations, and use the loss sum back propagation to update model parameters until the model converges.

[0010] Step 3: System design and implementation

[0011] Based on YOLO target detection technology and deep learning models, real-time identification of employee behavior in industrial production areas is performed. The system can detect elements such as people, hands, mobile phones, cigarette butts, safety wearable devices, and judge whether the behavior is abnormal by the positional relationship and relative movement between these elements. For example, by calculating the IoU (intersection over union) between the hand and the mobile phone, it can determine whether the mobile phone is played (IoU>0.5 and continuous detection for a period of time, such as 2 seconds), and by the IoU between the cigarette butt and the hand, it can determine whether smoking. The system also supports custom area detection, which allows users to accurately manage high-risk areas and key prevention and control areas.

[0012] When a violation is detected (such as smoking, playing with mobile phones, not wearing safety clothing, etc.), the system will issue an alarm in real time. The alarm signal can be sent to the management personnel in a variety of ways, such as client pop-up notification, connecting to the alarm device to send out an audible alarm, SMS push, etc., to ensure that the management personnel receive the information in time and take measures to effectively prevent the occurrence of safety hazards.

[0013] Step 4: History query and intelligent analysis

[0014] Detailed records of violation information such as time, location, content, and video clips are recorded, and historical records can be queried by time period and region for easy retrospective analysis. The data from the previous day is automatically analyzed daily, and Pandas data analysis tools and algorithms such as Isolation Forest and DBSCAN are used to explore the patterns of violations and risk areas, generate trend charts, pie charts and other visual reports, and provide optimization suggestions through natural language generation technology to assist management decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments:

[0016] Figure 1 The overall architecture diagram of the detection system.

[0017] Figure 2 Schematic diagram of the improved YOLO model structure,

[0018] Figure 3 Schematic diagram of abnormal behavior detection process. DETAILED DESCRIPTION

[0019] The following will be combined with the attached embodiment of the present invention Figure 1 , which are clearly and comprehensively described in the technical solutions of the embodiments of the present invention. Obviously, the embodiments described here are only a part of the many embodiments of the present invention, not all of them. Without involving creative work, all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention also fall within the scope of protection of the present invention.

[0020] Example:

[0021] (1) Data preparation stage

[0022] Long-term video data is collected from multiple surveillance cameras in steel plants, production workshops, warehouses, etc., covering different working hours and personnel activity status. Web crawlers are used to obtain supplementary data from industrial safety-related websites and video platforms, and AIGC technology is used to generate data simulating special scenarios (such as extreme lighting and dense crowds) to ensure the diversity of the data set. A professional annotation team is organized to use customized annotation tools to annotate the collected videos frame by frame, accurately annotate the positions and behavior categories of elements such as people, hands, mobile phones, cigarette butts, and safety wearable devices. The annotation process follows strict specifications to ensure the quality of annotation. The annotated data is operated according to the above preprocessing method, the cropping area and scaling ratio are set, the appropriate color space conversion method is selected according to the model requirements, and the data set is divided into training set, validation set, and test set in a ratio of 7:2:1 to prepare for model training.

[0023] (2) Model training phase

[0024] The model is built based on the YOLO framework, integrating the BGCNet network and the ShuffleNetV2 backbone network. The network parameters (such as the convolution kernel size in the context modeling stage in BGCNet, the number of channels in ShuffleNetV2, etc.) are adjusted according to the actual scenario requirements. The initial learning rate is set to 0.001, the cosine annealing decay strategy is adopted, the momentum is 0.9, the weight decay coefficient is 0.0005, the training batch size is 16, and the number of iterations is 100. According to the characteristics of the dataset and the performance of the model, these parameters are adjusted in time to optimize the training process. When calculating the weighted binary cross entropy loss and smoothed L1 loss, reasonable weight parameters are set according to the difficulty and importance of different types of behavior annotations. When calculating the matching loss, the line segments are strictly selected according to the matching conditions, and the geometric distance loss is accurately calculated to ensure that the loss function can effectively guide the model training. The training set data is input into the model for training. After every 10 training iterations, the model performance is evaluated using the validation set, and indicators such as mAP value, precision and recall are calculated. Observe the trend of indicator changes. If the performance no longer improves or the improvement is not obvious and the loss value tends to be stable, stop training and save the model.

[0025] (3) System implementation and deployment phase

[0026] Select a high-performance inference server, install the operating system Ubuntu Server and the deep learning framework PyTorch and its dependent libraries, deploy the trained model on the server, and optimize the model inference performance.

[0027] Install cameras and sensor devices in industrial scenarios to ensure coverage of key areas and normal operation. Connect devices to servers through the network, configure data transmission protocols and parameters, and ensure stable transmission of video streams and sensor data.

[0028] Develop a fully functional system management client, including user login authentication, real-time monitoring screen display, alarm information reception and processing, historical record query and export, intelligent analysis report viewing and other functional modules. Optimize the client interface design to ensure convenient operation and friendly interaction, and realize stable communication and data interaction between the client and the server.

[0029] (4) Historical record query and intelligent analysis

[0030] The system records every detected violation in detail, including the time t, location l, specific content c, and related video clips v. Managers can query the violation history of a specific area A according to the time period [t1, t2], which is convenient for backtracking and analysis.

[0031] The system has intelligent analysis capabilities and automatically analyzes the previous day's violation data at midnight every day. Use recurrent neural networks or long short-term memory networks in deep learning to perform time series analysis on historical behavior data to explore the potential patterns and laws of employee behavior in the time dimension. Suppose the employee behavior sequence is X = {x1, x2, ..., x n}, where x i Represents the behavior feature vector at time i, and updates the hidden state formula h through RNN or LSTM t =f(h t-1 , x t ), (f is a nonlinear activation function, such as f(x) = tanh(x)) to analyze the behavior pattern and identify that employees are more likely to have fatigue-related abnormal behaviors in the afternoon (12≤t<14) (assuming that the set of fatigue-related abnormal behaviors is F, when x t When certain conditions are met, x t ∈F), or after equipment maintenance (assuming that the equipment maintenance time interval is [m1, m2], when t∈[m1, m2], it is easy to ignore safety regulations (assuming that the set of behaviors that ignore safety regulations is S, when the corresponding conditions are met, x t ∈S).

[0032] Cluster analysis algorithms such as DBSCAN and K-Means++ are used to perform spatial dimension cluster analysis on employee behaviors in different regions r and different positions p, and the similarities and differences in behavioral characteristics of different groups are found. For example, employees in high temperature and high noise areas (assuming that the conditions for high temperature and high noise areas are T>T0 and N>N0, T is temperature, T0 is the high temperature threshold, N is the noise level, and N0 is the high noise threshold) may be more inclined to gather in a specific location to rest, thereby increasing the risk of illegal behaviors such as smoking. Suppose the employee behavior data point is y i =(r i , p i ,b i )(b i The core point of DBSCAN is defined as a point that contains at least MinPts data points within a radius of ε. The clustering process calculates the Euclidean distance between data points. (m is the feature dimension) to discover the clustering patterns of employee behaviors in different regions and positions, and then analyze the group behavior characteristics.

[0033] The Apriori association rule mining algorithm is introduced to analyze the potential association between abnormal behavior a and environmental factors e (temperature T, humidity H, noise level N), equipment operation status s (normal operation s1, failure s2, shutdown s3, etc.), and work task type t (emergency task t1, routine task t2, etc.). For example, when the equipment temperature is too high (T>T1, T1 is the high temperature threshold) and an emergency task is being performed (t=t1), the probability of employees performing illegal operations (assuming that the set of illegal operations is O) increases significantly. The support calculation formula for the association rule is: is the number of records where a, e, s, and t appear at the same time, and N is the total number of records), the confidence calculation formula is:

[0034] Using the Bayesian network causal inference algorithm, we try to determine the causal relationship between abnormal behavior and other factors, and provide a basis for targeted intervention measures. For example, if long-term continuous work w is the cause, employee fatigue f is the intermediate state, and the behavior s of sleeping on the job P(s|f, w) = P(s, f, w) / P(f, w) is the result, the conditional probability formula in the Bayesian network P(s|f, w) = P(s, f, w) / P(f, w) can be used to analyze the causal chain of long-term continuous work (assuming continuous working time w>W, W is the long-term threshold) leading to employee fatigue (assuming the fatigue state judgment function is F(w), when F(w) = 1 indicates fatigue), which in turn triggers the behavior of sleeping on the job (assuming the judgment function of sleeping on the job is G(f) = 1, indicating sleeping on the job), thus providing a reference for the reasonable arrangement of working time (such as setting a reasonable shift system, assuming the shift length is B, which should satisfy B≤W).

[0035] The system has intelligent decision support and optimization suggestion generation. Based on the employee's historical behavior record H and personal characteristics P (age a, work experience e, job risk level r), a personalized risk assessment model is built for each employee. Let the risk assessment function be R = g(H, P). By calculating the employee's risk in different behavior dimensions (compliance with safety regulations b s 、Work efficiency w , Fatigue level l ) to accurately assess the risk of individual employees. Based on the results of personalized risk assessment, a dynamic early warning strategy is formulated. For high-risk employees (assuming that the risk score R>R high , R high For employees with low risk (R<R), the early warning threshold θ is lowered (e.g., θ=θ0-ɑR, θ0 is the initial threshold, a is the adjustment coefficient), the early warning frequency f (e.g., f=f0+βR, f0 is the initial frequency, β is the increase coefficient) and the method (e.g., using pop-up windows, text messages, voice reminders, etc.) are increased to ensure timely intervention; for employees with low risk (R<R low , R lowFor low risk thresholds), appropriately relax the warning conditions (such as increasing the warning threshold θ and reducing the warning frequency f) to reduce unnecessary interference and improve management efficiency.

[0036] The above embodiment is only a specific implementation of the present invention, and can be adjusted and optimized according to specific industrial scenarios and needs in actual applications. Without departing from the spirit and scope of the present invention, various modifications and equivalent replacements of the embodiments should be included in the protection scope of the present invention.

Claims

1. A system and method for abnormal behavior detection and intelligent analysis in industrial scenarios based on machine vision technology. To ensure the efficient and safe operation of the production process and to detect abnormal operation behaviors of workers in a timely manner, the system uses high-precision cameras to collect real-time image data in the workshop in all directions, and uses advanced image recognition algorithms to accurately analyze the workers' movements, postures, and the operating status of equipment. Intelligent analysis of historical records assists managers in making quick decisions, effectively avoids potential safety accidents, and improves the overall benefits of industrial production. It is characterized by: The following steps are involved: Step 1: Data collection and preprocessing: Collect image and video data of human behavior in industrial scenarios through network search, AIGC and simulation scenarios, accurately annotate the collected data, identify abnormal behaviors and types, crop, scale and convert color space of images, and divide the annotated data into training set, validation set and test set in proportion. Step 2: Model selection and improvement: Build an improved model based on the YOLO framework, integrate the BGCNet network, introduce the lightweight backbone network ShuffleNetV2, integrate multiple loss functions, use adaptive learning rate strategy and optimizer to train the model, and use loss sum back propagation to update the model parameters until convergence. Step 3: System design and implementation: Based on YOLO target detection technology and deep learning models, real-time identification of employee behavior in industrial production areas is performed, custom area detection is supported, and when violations are detected, real-time alarms are issued in a variety of ways. Step 4: Historical record query and intelligent analysis: Record violation information in detail, support historical record query by time period and region, automatically analyze the previous day's data every day, use a variety of algorithms to explore violation patterns and risk areas, generate visual reports, provide optimization suggestions, and assist management decisions.

2. The machine vision-based industrial scene abnormal behavior detection system and intelligent analysis method according to claim 1 is characterized in that: In the data acquisition and preprocessing steps, the data collection sources include collecting video data from industrial scene monitoring cameras, obtaining supplementary data from web crawlers, and generating simulated data using AIGC technology, and the labeling process follows strict specifications to ensure the labeling quality.

3. The abnormal behavior detection system and intelligent analysis method for industrial scenes based on machine vision according to claim 1 is characterized in that: In the model selection and improvement steps, the BGCNet network captures global context information through a unique mechanism, ShuffleNetV2 reduces computational complexity, the loss function includes weighted binary cross entropy loss and smoothed L1 loss, the learning rate strategy adopts cosine annealing attenuation, and the optimizer is the Adam optimizer.

4. The abnormal behavior detection system and intelligent analysis method for industrial scenes based on machine vision according to claim 1 is characterized in that: In the system design and implementation steps, the IoU between the elements is calculated to determine whether the behavior is abnormal. The alarm methods include client pop-up notification, sound alarm from the connected alarm device, and SMS push.

5. The abnormal behavior detection system and intelligent analysis method for industrial scenes based on machine vision according to claim 1 is characterized in that: In the historical record query and intelligent analysis steps, the violation information includes the occurrence time t, location I, specific content c and related video clips v. The historical record query supports querying violations in a specific area A according to time periods t1 and t2.

6. The industrial scene abnormal behavior detection system and intelligent analysis method based on machine vision according to claim 1 is characterized in that: In the historical record query and intelligent analysis steps, intelligent analysis includes time series analysis, cluster analysis, association rule mining, causal inference analysis and intelligent decision support, specifically: Step 1: Time series analysis: Using recurrent neural network (RNN) or long short-term memory network (LSTM), let the employee behavior sequence be X = {x1, x2, ..., x n }, update the formula h through the hidden state t =f(h t-1 , x t ) Analyze behavioral patterns and identify abnormal behaviors that are prone to occur in different time periods. Step 2: Cluster analysis: Use cluster analysis algorithms such as DBSCAN and K-Means++, and set the employee behavior data point as y i =(r i , p i , b i ), by calculating the Euclidean distance (m is the feature dimension) to achieve clustering and analyze the clustering patterns of employee behaviors in different regions and positions. Step 3: Association rule mining: Introduce the Apriori association rule mining algorithm to analyze the potential association between abnormal behavior a and environmental factors e (temperature T, humidity H, noise level N), equipment operating status s (normal operation s1, fault s2, shutdown s3, etc.). The support calculation formula is: (count(a, e, s, t) is the number of records where a, e, s, and t appear at the same time, and N is the total number of records), the confidence calculation formula is: Step 4: Causal inference analysis: Using the Bayesian network causal inference algorithm, assume that long-term continuous work w is the cause, employee fatigue f is the intermediate state, and sleeping on the job s is the result. The causal relationship is analyzed through the conditional probability formula P(s|f, w) = P(s, f, w) / P(f, w) to provide a basis for intervention measures. Step 5: Based on the employee's historical behavior record H and personal characteristics P (age a, work experience e, job risk level r), build a personalized risk assessment model R = g (H, P) for each employee. Develop a dynamic early warning strategy based on the risk score, lower the early warning threshold θ = θ0-ɑR (θ0 is the initial threshold, a is the adjustment coefficient) for high-risk employees, increase the early warning frequency (f0 is the initial frequency, β is the increase coefficient) and method, and appropriately relax the early warning conditions for low-risk employees.

Images