Intelligent worker behavior management and control method and system based on multi-modal sign recognition

Abstract

The invention relates to the technical field of intelligent management and control of worker behaviors, and discloses an intelligent management and control method and system for worker behaviors based on multi-modal sign recognition, and the system is composed of a multi-modal sensor array and a behavior analysis server which are arranged at set intervals in an operation area. The method comprises the steps that a sensor array collects physical sign state and environment interaction data, and a behavior characteristic data set is generated; the server receives the real-time sign fusion data to construct a matrix, extracts mode features in combination with a data set after processing, predicts a behavior mode according to a physiological state change gradient and an action trajectory, outputs time-space distribution features through a behavior association model, updates the data set, and determines classification features; and dynamically configuring a security policy according to the classification characteristics, wherein the security policy comprises execution actions such as region identifier mapping and control protection equipment activation, and dynamically allocating protection resources. According to the scheme, multi-modal data fusion analysis and security policy dynamic adjustment are realized, and the intelligence and accuracy of operation security management and control are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of worker behavior, and specifically to a method and system for intelligent control of worker behavior based on multimodal vital sign recognition. Background Art

[0002] In various work scenarios such as industrial production and construction, the safety of workers' work behavior is directly related to production efficiency and the safety of life and property. Traditional methods of worker behavior control mainly rely on manual inspections, video surveillance, and limited sensor monitoring, which have many limitations.

[0003] From the perspective of monitoring methods, traditional single-mode sensors (such as devices that only monitor position or a single physiological parameter) cannot fully capture the physical status of workers and environmental interaction information. For example, it is difficult to obtain real-time changes in physiological parameters such as heart rate and body temperature by monitoring workers' movements only through cameras, and it is impossible to timely determine whether workers are in dangerous states such as fatigue and stress; and relying solely on physiological sensors cannot accurately analyze the relationship between workers' movement trajectories and environmental risks. The one-sidedness of this single-mode monitoring makes it impossible to build a complete behavioral feature model, making it difficult to accurately analyze and predict workers' behavior.

[0004] At the data analysis level, traditional methods often use static rule matching or simple threshold judgment to identify abnormal behavior. For example, a speed threshold for a certain action is set, and an alarm is triggered when the action speed is detected to exceed the threshold. However, this method ignores the individual differences between different workers, the changes in the reasonable range of actions in different work scenarios, and the dynamic relationship between physiological state and action. At the same time, traditional systems lack the ability to integrate multi-source data, and cannot effectively integrate multi-dimensional information such as physiological parameters, action trajectories, and environmental data, making it difficult to explore the potential behavior patterns and risk trends behind the data.

[0005] From the perspective of security policy execution, the security policy configuration of traditional management and control systems is usually fixed and passive. For example, protective equipment is fixedly set up in specific areas and cannot be dynamically adjusted according to real-time worker behavior patterns and risk distribution. When worker behavior patterns change or new risk scenarios emerge, traditional systems cannot respond in a timely manner, resulting in unreasonable allocation of protection resources, and there may be insufficient protection in key risk areas and waste of resources in low-risk areas. In addition, the linkage between the components of traditional systems is poor. For example, there is a lack of effective data interaction and coordination mechanism between monitoring equipment and protective equipment, which makes it impossible to achieve real-time risk response and linkage control.

[0006] With the development of industrial intelligence, higher requirements are put forward for the accuracy, real-time performance, and intelligence of worker behavior control. The existing technical means are difficult to meet the safety control requirements in complex working environments. There is an urgent need for an intelligent control method and system for worker behavior that can integrate multi-modal data, realize intelligent analysis of behavior patterns, and dynamically configure safety strategies, so as to improve operation safety and production efficiency and reduce the occurrence of safety accidents. Summary of the Invention

[0007] The purpose of the present invention is to provide an intelligent control method and system for worker behavior based on multi-modal physical sign recognition to solve the problems raised in the above background technology.

[0008] To achieve the above purpose, the present invention provides the following technical solution: An intelligent control method for worker behavior based on multi-modal physical sign recognition, which is applied to a behavior monitoring system. The system includes a plurality of multi-modal sensor arrays deployed in the working area and a behavior analysis server connected to the multi-modal sensor arrays. The distance between adjacent two multi-modal sensor arrays is set at a certain interval. The method includes: Collect the physical sign state data and environmental interaction data of the operators in the area through the multi-modal sensor arrays to generate a behavior feature data set; Receive the real-time physical sign fusion data of a plurality of multi-modal sensor arrays through the behavior analysis server to construct a physical sign fusion matrix; According to the behavior feature data set and the real-time data of each multi-modal sensor array, determine the classification features of the operation behavior mode. Among them, determining the classification features of the operation behavior mode includes: processing the physical sign fusion matrix, extracting mode features in combination with the behavior feature data set, predicting the behavior mode according to the physiological state change gradient and action trajectory information, outputting the spatio-temporal distribution features of the operation behavior mode through the behavior association model, and updating the behavior feature data set according to the spatio-temporal distribution features; Dynamically configure the safety strategy for the working area according to the classification features.

[0009] Preferably, determining the classification features of the operation behavior mode includes: Process the physical sign fusion matrix to extract the physiological parameter distribution, action trajectory features, and operation behavior trends; Conduct state stability modeling on the physical sign fusion matrix according to the physiological parameter distribution and action trajectory features, divide the working area into multiple sub-areas and mark area identifiers, perform correlation matching between the physiological parameters of the sub-areas and the behavior feature data set, and mark the area identifiers in the behavior feature data set; Calculate the physiological state change gradient according to the positions of the multimodal sensor arrays, predict the distribution of behavior patterns based on the physiological state change gradient and the operation behavior trend, and calculate the behavior prediction information for each sub-region; Construct a behavior association model, use the behavior prediction information as the input parameter of the behavior association model, perform spatial association modeling on the behavior prediction information through the behavior association model, and output the spatio-temporal distribution characteristics of the operation behavior pattern; Update the behavior feature dataset according to the spatio-temporal distribution characteristics to obtain the classification features of the operation behavior pattern.

[0010] Preferably, the processing of the sign fusion matrix includes: Normalize the sign fusion matrix, intercept the behavior hot spot area in the matrix through a sliding window, filter out the outliers in the hot spot area, and calculate the operation behavior trend through an orthogonal decomposition algorithm; Calculate the spatial association features of the sign fusion matrix, calculate the action coupling coefficient, posture stability index and risk area marker between regions according to the spatial association features, construct a feature fusion network, and calculate the action trajectory features through the feature fusion network; Extract the time-domain features and spatial-domain features collected by each multimodal sensor array, calculate the behavior feature vector of the array according to the phase difference between the time-domain features and the spatial-domain features, perform feature matching on the multimodal sensor arrays at different positions according to the behavior feature vector, and calculate the operation behavior trend.

[0011] Preferably, the state stability modeling of the sign fusion matrix includes: Extract the physiological parameter sampling points in each frame of data according to the physiological parameter distribution, perform association mapping according to the sampling points and the action trajectory features to generate a state stability map, spatially align the state stability maps collected by multiple arrays, and calculate the state stability distribution of the region; Set a risk determination threshold, locate the risk source according to the physiological parameter values of multiple frames of sign fusion matrices, calculate the risk intensity difference. If the risk intensity difference is greater than or equal to the risk determination threshold, it indicates that there is a behavior abnormality in this region. Perform kinematic model constraint compensation on the current region, perform iterative correction on the state stability distribution of the current region according to the corresponding attitude transformation model of the current region, and calculate the stability compensation value of the abnormal region according to the correction result; Perform state stability modeling on the sign fusion matrix according to the state stability distribution, and perform trajectory annotation on the stability model of the region through the action trajectory features.

[0012] Preferably, the calculation of the physiological state change gradient according to the positions of the multimodal sensor arrays includes: Extract the change points of physiological parameters based on multiple groups of fused physical sign data, map the change points to a unified spatial coordinate system according to the deployment positions of the arrays, and fit the change points through a spatial interpolation algorithm to generate a behavior field model of the region; Perform equally-spaced sampling along the movement trajectory of the behavior field model, calculate the change rate of physiological parameters, the action fluctuation index, and the state change slope of the trajectory according to the sampling results, and calculate the state change parameters according to the change rate of physiological parameters, the action fluctuation index, and the state change slope; Project the distribution characteristics of the operation behavior in each frame of data onto the behavior field model according to the deployment parameters and acquisition accuracy of the multimodal sensor arrays, partition along the movement direction of the behavior field model according to the number of arrays, analyze the change rules of the operation behavior within the partitions, and calculate the behavior distribution characteristics according to the change rules; Calculate the physiological state change gradient according to the state change parameters and the behavior distribution characteristics. The calculation process of the physiological state change gradient includes: based on the spatial position range from the first multimodal sensor array to the last multimodal sensor array, select spatial coordinate points in the array deployment direction, cumulatively calculate the product of the behavior field characteristic weight value and the operation behavior distribution characteristic weight value within the spatial resolution range, and superimpose the influence value of the array acquisition frequency on the physiological state change rate.

[0013] Preferably, the calculation of the behavior prediction information for each sub-region includes: Take the main movement trajectory of the behavior field model as the reference line, take the peak position of the operation behavior in each frame of data as the reference point, calculate the behavior offset, and draw the behavior distribution curve according to the spatial coordinates; Correct the change rate and direction in the operation behavior trend according to the physiological state change gradient; Starting from the nearest behavior distribution point, continue to draw the distribution curve according to the corrected results of the change rate and direction to generate the behavior distribution points in the next time period until the distribution points cover the entire target region to generate the behavior prediction information.

[0014] Preferably, the construction of the behavior association model includes: An input layer for organizing the behavior prediction information into spatial distribution data and performing normalization processing; A feature fusion layer for extracting the regional association features of the behavior by processing the spatial distribution data and constructing the dependency relationship between spatial units; A policy generation layer for integrating the association relationships of the operation behavior on the spatial units to generate a security policy configuration sequence.

[0015] Preferably, the obtaining of the classification features of the operation behavior patterns includes: Corresponding to the spatio-temporal distribution characteristics of the operation behavior pattern output according to the behavior association model, the identification of the sub-region is associated with the spatio-temporal distribution characteristics; Reorganize the regional data in the behavior feature dataset according to the spatio-temporal characteristics to generate a regional distribution map sorted by behavior risk level; According to the reorganized regional distribution map, output the optimized classification features of the behavior pattern.

[0016] Preferably, the dynamic configuration of the safety policy for the operation area includes: Map the regional identification to the region of the classification features of the operation behavior pattern; According to the spatio-temporal distribution characteristics of the behavior pattern, control the execution actions of the safety policy, including the activation of protection equipment, the adjustment of the operation path, and the configuration operation of the alarm level; According to the spatial distribution of the behavior pattern and the preset safety policy rules, dynamically allocate protection resources to the corresponding spatial regions; The control of the execution actions of the safety policy includes: When the behavior pattern reaches the preset risk level threshold in the target area, trigger the linkage command of the protection equipment in the adjacent area; Dynamically combine the boundaries of the safety area according to the operation path policy to generate an alarm level vector; Based on the alarm level vector, adjust the sensor parameters of the monitoring nodes in the target area.

[0017] Preferably, the present invention further includes an intelligent control system for workers' behavior based on multi-modal physical sign recognition, which is used to implement an intelligent control method for workers' behavior based on multi-modal physical sign recognition as described above. The system includes a plurality of multi-modal sensor arrays deployed in the operation area at a set interval, and a behavior analysis server connected to the multi-modal sensor arrays. Among them, the multi-modal sensor arrays are used to collect the physical sign state data and environmental interaction data of the operation personnel in the area to generate a behavior feature dataset; the behavior analysis server is used to receive the real-time physical sign fusion data of the plurality of multi-modal sensor arrays, construct a physical sign fusion matrix, process the physical sign fusion matrix, extract pattern features in combination with the behavior feature dataset, and perform behavior pattern prediction according to the physiological state change gradient and action trajectory information, output the spatio-temporal distribution characteristics of the operation behavior pattern through the behavior association model, update the behavior feature dataset according to the spatio-temporal distribution characteristics to determine the classification features of the operation behavior pattern, and dynamically configure the safety policy for the operation area according to the classification features.

[0018] Compared with the prior art, the beneficial effects of the present invention are: In terms of data collection and fusion, the multi-modal sensor array can simultaneously collect the physical sign data of the operator (such as heart rate, body temperature, posture, etc.) and environmental interaction data (such as lighting, temperature, equipment operation status, etc. in the operation area), generating a comprehensive and rich behavior feature dataset. Compared with traditional single-modal sensors, the fusion of multi-modal data can more truly reflect the actual operation status of workers and environmental influencing factors, providing a more reliable basis for subsequent behavior analysis. The behavior analysis server constructs a physical sign fusion matrix, performs standardized processing and deep fusion on multi-source data, effectively solves the problem of isolated multi-source data and difficult correlation analysis in traditional systems, and improves the utilization value of data.

[0019] In the link of behavior pattern analysis and prediction, through the processing of the physical sign fusion matrix, key information such as the distribution of physiological parameters, action trajectory features, and operation behavior trends is extracted, and a series of algorithms such as state stability modeling, physiological state change gradient calculation, and behavior association model are combined to achieve accurate classification and spatio-temporal distribution prediction of operation behavior patterns. For example, through state stability modeling, abnormal behavior conditions in the area can be detected in a timely manner, and kinematic model constraint compensation and stability correction are carried out, improving the detection accuracy and response speed of abnormal behaviors; the calculation of physiological state change gradient takes into account factors such as the position of the multi-modal sensor array, acquisition accuracy, and behavior field model, and can more accurately reflect the dynamic change trends of physiological states and action trajectories, providing a scientific basis for behavior pattern prediction. The behavior association model outputs the spatio-temporal distribution characteristics of operation behavior patterns through spatial association modeling, and updates the behavior feature dataset according to this feature, enabling the system to continuously learn and adapt to new behavior patterns, and improving the self-adaptability and prediction accuracy of the system.

[0020] In terms of dynamic configuration of safety policies, according to the classification characteristics and spatio-temporal distribution characteristics of behavior patterns, dynamic adjustment and optimization of safety policies for the operation area are realized. Specifically, mapping the area identifier with the classification characteristics of behavior patterns can accurately locate risk areas; controlling execution actions such as activation of protection equipment, adjustment of operation paths, and configuration of warning levels realizes real-time response and accurate execution of safety policies; dynamically allocating protection resources to corresponding spatial areas improves the utilization efficiency of protection resources and avoids resource waste and insufficient protection problems under traditional fixed policies. For example, when the behavior pattern reaches the preset risk level threshold in the target area, a linkage instruction for protection equipment in the adjacent area is triggered, which can form a collaborative protection system and enhance the ability to respond to sudden risks; dynamically combining the boundaries of safety areas according to the operation path policy and generating a warning level vector can provide more reasonable path guidance and risk warnings for operators, reducing operation risks.

[0021] From the perspective of the overall system performance, this method and system achieve intelligent control of the entire process from data collection, analysis to strategy execution, with high real-time performance and reliability. The fusion of multi-modal data and the application of advanced algorithms enable the system to deeply mine the potential information in behavior data, predict the development trend of behavior patterns in advance, and realize the transformation from passive monitoring to active prevention. At the same time, the self-adaptability and dynamic adjustment ability of the system enable it to adapt to changes in different operation scenarios and environments, with broad application prospects. Through the application of this system, the efficiency and level of operation safety management can be effectively improved, the occurrence of safety accidents can be reduced, the life safety and physical health of operators can be guaranteed, and at the same time, the production process can be optimized and the production efficiency can be improved, with significant economic and social benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is the working principle diagram of the method and system for intelligent control of workers' behavior based on multi-modal sign recognition according to the present invention; Figure 2 is the flow chart for determining the classification features of operation behavior patterns; Figure 3 is the flow chart for modeling the state stability of the sign fusion matrix; Figure 4 is the flow chart for dynamic configuration of safety strategies in the operation area. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0024] Please refer to Figures 1 - 4 , a method for intelligent control of workers' behavior based on multi-modal sign recognition according to the present invention is applied to a behavior monitoring system including multiple multi-modal sensor arrays and a behavior analysis server, and the distance between adjacent multi-modal sensor arrays is a set interval. The specific implementation steps are as follows: Through the multi-modal sensor arrays deployed in the operation area, the sign state data of the operators (such as physiological parameters such as heart rate, body temperature, and limb movement trajectories) and the environmental interaction data (such as operation equipment operation signals, environmental temperature and humidity, etc.) are collected in real time. Each array synchronously collects data at a preset frequency, and after preliminary preprocessing (such as denoising, format unification), a behavior feature data set including time stamps and spatial position identifiers is generated and stored in the database of the behavior analysis server.

[0025] The behavior analysis server receives the real-time physical sign data transmitted by each multi-modal sensor array, aligns the data in the order of spatial position and time series, and constructs a multi-dimensional physical sign fusion matrix. The row dimension of the matrix is the sensor array number (corresponding to the spatial position), the column dimension is the type of acquisition parameter (such as heart rate, acceleration, etc.), and the element value is the real-time monitoring value at each moment, forming a physical sign data matrix structure in the space-time dimension.

[0026] After preprocessing the physical sign fusion matrix, such as noise reduction and standardization, combined with the historical behavior feature dataset, the classification features of the operation behavior are extracted through a pattern recognition algorithm. Specifically, it includes: using matrix decomposition technology to extract the physiological state change gradient (such as heart rate change rate, action acceleration gradient) and action trajectory information (such as limb movement path, equipment operation sequence), and inputting them into the behavior association model for spatio-temporal distribution prediction. The model output includes the time evolution and spatial distribution characteristics of the operation behavior patterns in each sub-region (such as the spatio-temporal hot spot area of high-risk behaviors), and accordingly updates the behavior feature dataset to form a dynamically optimized classification feature library.

[0027] According to the determined classification features, the operation area is divided into sub-regions with different risk levels, and corresponding differential safety strategies are configured. For example, increase the sensor monitoring frequency in high-risk areas, activate the surrounding protection equipment (such as safety fences, warning lights), or adjust the movement route of the operation personnel through the path planning algorithm to achieve real-time dynamic adjustment of the safety strategy.

[0028] The present invention will be further described below in conjunction with Embodiments 1 to 5: Embodiment 1: When determining the classification features of the operation behavior pattern, it is necessary to perform multi-dimensional processing on the physical sign fusion matrix and combine the behavior feature dataset to achieve dynamic analysis. The specific implementation method is as follows: Process the physical sign fusion matrix to extract key features. By analyzing the data distribution in the matrix, identify the distribution characteristics of physiological parameters, such as the aggregation or dispersion state of physiological indicators such as the heart rate and body temperature of the operation personnel in space, and the fluctuation law in different time periods. At the same time, extract the action trajectory features, including dynamic parameters such as the curvature and speed change of the limb movement path and tool operation trajectory of the operation personnel, and the operation behavior trend, such as the continuous operation duration and the change law of the action frequency over time. The extraction of these features is achieved through the time-domain and space-domain analysis of the matrix data. For example, perform time series analysis on the heart rate data to identify abnormal fluctuation points, and perform spatial trajectory reconstruction on the action acceleration data to obtain the movement path curve.

[0029] Based on the distribution of physiological parameters and the characteristics of action trajectories, a state stability model is built for the sign fusion matrix. The operation area is divided into multiple sub-areas. The division of sub-areas can be determined according to the physical layout of the operation area (such as workshop work station partitioning, building floor structure, etc.) or the deployment location of the sensor array. Each sub-area is assigned a unique area identifier (such as Area A, Area B1, etc.). By correlating and matching the physiological parameter data of each sub-area with the historical data in the behavior feature dataset, for example, comparing the distribution of abnormal heart rate points in the current sub-area with the physiological parameter pattern in the "equipment fault troubleshooting" scenario in the historical data, the corresponding area identifier is marked in the behavior feature dataset, and the mapping relationship between the current real-time data and the historical behavior pattern is established. This process is achieved through pattern recognition algorithms. For example, the K-nearest neighbor algorithm (K-NN) is used to retrieve the most similar behavior pattern category in the historical data.

[0030] Calculate the gradient of physiological state changes based on the position of the multi-modal sensor array. The specific steps include: First, extract the change points of physiological parameters from multiple groups of sign fusion data, such as the time points when the heart rate suddenly increases, the position points where the action acceleration suddenly changes, etc. According to the deployment location of the sensor array, map these change points to a unified spatial coordinate system to form a discrete set of spatial data points. Fit these discrete points through a spatial interpolation algorithm (such as the inverse distance weighted interpolation method) to generate a continuous regional behavior field model, which describes the spatial distribution trend of physiological parameter changes in the form of a three-dimensional spatial field. Perform equidistant sampling along the movement trajectory of the behavior field model to obtain the change rate of physiological parameters (such as the change value of heart rate per unit time), the action fluctuation index (a dimensionless parameter reflecting action smoothness), and the state change slope (the spatial gradient of parameter change) at each sampling point, and calculate the state change parameter by integrating these parameters to quantify the change intensity of physiological state in space and time.

[0031] Project the distribution characteristics of operation behaviors in each frame of data (such as the frequency of action occurrence, the spatial distribution of intensity) onto the behavior field model according to the deployment parameters (such as array spacing, coverage range) and acquisition accuracy of the multi-modal sensor array. Along the movement direction of the behavior field model, divide the model into multiple partitions according to the number of sensor arrays, and analyze the change rules of operation behaviors in each partition, such as whether the action frequency in the same partition shows periodic fluctuations, the difference in action intensity between different partitions, etc. Then calculate the behavior distribution characteristics, such as distribution entropy value, concentration index, etc., to describe the distribution pattern and complexity of operation behaviors in space.

[0032] The calculation of the physiological state change gradient comprehensively considers the state change parameters and the characteristics of behavior distribution. The specific process is as follows: within the spatial position range from the first multi-modal sensor array to the last sensor array, multiple spatial coordinate points are selected along the array deployment direction. For each coordinate point, the product of the behavior field characteristic weight value and the operation behavior distribution characteristic weight value within the spatial resolution range where the point is located is calculated cumulatively, where the weight values are preset according to the importance of the parameters (for example, the weight of the physiological parameter change rate is 0.6, and the weight of the behavior distribution entropy is 0.4). In addition, the influence value of the sensor array acquisition frequency on the physiological state change rate is superimposed. For example, an array with a higher acquisition frequency has a higher requirement for real-time performance, and the weight of its data change rate needs to be increased accordingly. Through the above steps, a gradient value reflecting the spatial change trend of the physiological state is finally obtained, and this gradient value can be used to judge the stability and potential risks of the behavior pattern.

[0033] When calculating the behavior prediction information of each sub-region, the main movement trajectory of the behavior field model is used as the reference line, and the main movement trajectory is determined through the statistical analysis of historical action trajectory data. For example, the frequently occurring movement path is extracted as the reference. The peak position of the operation behavior in each frame of data (such as the spatial point where the action occurs most frequently) is used as the reference point, and the behavior offset between the reference point and the reference line is calculated. This offset reflects the deviation degree of the current behavior from the typical behavior pattern. These reference points and offsets are plotted into a behavior distribution curve according to the spatial coordinates, and the shape of the curve (such as slope, curvature) can intuitively display the diffusion or convergence trend of the behavior in space.

[0034] According to the physiological state change gradient, the change rate and direction in the operation behavior trend are corrected. For example, if the physiological state change gradient of a certain sub-region shows that the heart rate change rate increases significantly and the action fluctuation index increases, it indicates that there may be an abnormal behavior trend in this region, and the original operation behavior change rate (such as action speed) and direction (such as the deflection angle of the movement trajectory) need to be adjusted, and the correction amplitude is determined according to the magnitude of the gradient value.

[0035] Starting from the nearest behavior distribution point, continue to draw the distribution curve according to the corrected change rate and direction, and gradually generate the behavior distribution points in the next time period. The drawing process uses a recursive method, and the position of each step of the distribution point is calculated based on the position of the previous point and the corrected parameters until the distribution points cover the entire target area, forming complete behavior prediction information. This information is stored in a grid form, and each grid cell corresponds to a sub-region, containing the type and probability value of the behavior pattern that may appear in this region in the next time period. For example, a certain grid cell is marked as "equipment maintenance, probability 75%", providing basic data for the subsequent prediction of the spatio-temporal distribution of the behavior pattern.

[0036] When constructing a behavior association model, the input layer of the model is responsible for organizing behavior prediction information into spatially distributed data. For example, the probability values of behavior patterns in each sub-region are arranged into a two-dimensional matrix according to spatial coordinates and normalized to unify the data range to the interval [0, 1] to eliminate the influence of different parameter dimensions. The feature fusion layer processes the spatially distributed data through deep learning models such as convolutional neural networks (CNNs) or graph neural networks (GNNs) to extract the regional association features of behaviors. For example, the correlation of behavior patterns between adjacent sub-regions (such as when a certain region is performing high-altitude operations, tool transfer behaviors may synchronously occur in adjacent regions), and constructs a dependency matrix between spatial units, which records the influence degree of the behavior patterns of each sub-region on other regions. The policy generation layer integrates the association relationships of operation behaviors on spatial units and generates a sequence of safety policy configurations through recurrent neural networks (RNNs) or Transformer models. This sequence contains the types and execution orders of safety policies for different sub-regions. For example, the protection equipment is activated for high-risk regions first, and then the path is adjusted for adjacent regions.

[0037] According to the spatio-temporal distribution characteristics of the operation behavior patterns output by the behavior association model, the identifiers of the sub-regions are corresponded to the spatio-temporal distribution characteristics. For example, area A1 at time t is marked as "high-risk operation, duration 30 minutes". The regional data in the behavior feature dataset is reorganized according to spatio-temporal characteristics. For example, the behavior pattern data of each sub-region is arranged in chronological order, and the regional data is sorted according to the behavior risk level (such as low, medium, high) to generate a regional distribution map. In the map, the risk levels and behavior patterns of different regions are intuitively displayed through the depth of color or the type of icon. According to the reorganized regional distribution map, the optimized classification features of behavior patterns are extracted. For example, the rule "high-risk behaviors are mainly concentrated in areas B2 and C3 from 3 pm to 5 pm, which coincides with the period of equipment fatigue operation" is summarized to provide a basis for the dynamic configuration of safety policies.

[0038] Example 2: The processing steps of the sign fusion matrix realize the conversion from raw monitoring data to structured behavior features through multi-stage data cleaning, feature extraction, and association analysis. The specific implementation methods are as follows: Standardize the sign fusion matrix to eliminate the dimensional differences of different physiological parameters and environmental data. The standardization process uses the Z-score method. For each parameter dimension in the matrix (such as heart rate, acceleration, environmental temperature, etc.), calculate the mean and standard deviation of the data in this dimension, and convert each data point into a dimensionless form of (original value - mean) / standard deviation, so that the data in each dimension follows the standard normal distribution. After standardization, intercept the behavior hot spot area in the matrix through the sliding window technique: set the window size to T frames of data (for example, T = 100, corresponding to a 10-second acquisition duration), slide on the matrix time axis with a fixed step size (such as 1 frame / second), calculate the variance or information entropy of each parameter dimension within each window, and identify the window with variance exceeding the preset threshold (such as 1.5 times the global variance) or a sudden increase in information entropy as the behavior hot spot area. These areas usually correspond to the key operation periods or abnormal behavior occurrence periods of the operator.

[0039] When filtering outliers in the hot spot area, use the DBSCAN (density clustering) algorithm to identify and remove noise points. The specific steps are as follows: construct a multi-dimensional feature space for the data points in the hot spot area according to the spatial position (sensor array number) and parameter values, set the neighborhood radius ε and the minimum number of samples MinPts (such as ε = 2 array spacings, MinPts = 5), check the number of samples within the ε neighborhood of each data point, and if it is less than MinPts, it is determined as a noise point and filtered. The data in the filtered hot spot area retains the true physiological parameter outliers and action trajectory mutation points, such as the points where the operator's heart rate suddenly increases abnormally or the sharp turning points of the tool operation trajectory.

[0040] When calculating the operation behavior trend through the orthogonal decomposition algorithm (such as singular value decomposition SVD), decompose the sign fusion matrix into the product form of singular values, left singular vectors, and right singular vectors, extract the singular vector combination corresponding to the first k largest singular values, reconstruct the matrix to retain the main feature components, and remove high-frequency noise and secondary interference factors. The reconstructed matrix reflects the low-frequency trend component of the operation behavior, such as the trend of gradually decreasing action amplitude during continuous operation or the slow upward trend of physiological parameters with the increase of labor intensity. By analyzing the time series changes of the singular vectors, the overall behavior trend direction of the operator is deduced.

[0041] When calculating the spatial correlation features of the vital sign fusion matrix, an undirected graph model is constructed based on the physical deployment locations of the sensor arrays. Each array node represents a spatial location, and the edge weights represent the correlation of physiological parameters or action trajectories between nodes (such as the Pearson correlation coefficient). Through graph neural network (GNN) or graph Laplacian matrix analysis, the action coupling coefficient between regions is calculated, which quantifies the degree of cooperation of the actions of workers in adjacent regions (such as the action synchronization of workers in front and back processes in assembly line operations); at the same time, the posture stability index is calculated, and the stability of the posture of the worker is evaluated by analyzing the time series fluctuations of the data of a single array node (such as the root mean square value of the acceleration signal). The smaller the index value, the more stable the posture. According to the action coupling coefficient and the posture stability index, risk areas are marked. For example, areas with high coupling coefficients and low stability indices may pose risks of personnel cooperation errors or posture imbalance.

[0042] When constructing the feature fusion network, a multi-layer perceptron (MLP) or a convolutional neural network (CNN) is used to aggregate multi-dimensional features. The inputs include standardized physiological parameters, filtered hot spot area data, trend components obtained by orthogonal decomposition, and spatial correlation features (coupling coefficient, stability index, etc.). The network automatically learns the interaction relationships between features through non-linear transformation. For example, the combination of abnormal physiological parameters and high action coupling coefficients may indicate safety risks in collaborative operations, and outputs an action trajectory feature vector, which contains comprehensive features such as the curvature, direction change rate, and spatial coverage of the trajectory.

[0043] When extracting the time domain features and spatial domain features collected by each multi-modal sensor array, the time domain features include the statistics of physiological parameters (mean, variance, kurtosis, skewness), frequency domain analysis results (such as the power spectral density of heart rate variability HRV), the rising edge / falling edge time of action signals, etc.; the spatial domain features include the spatial coordinate range of the action trajectory, the trajectory length, the relative position relationship with the operating equipment, etc. By calculating the phase difference between the time domain features and the spatial domain features, such as the time interval between the moment of sudden change in heart rate of an array and the inflection point of the action trajectory, a behavior feature vector is generated, which reflects the time correlation between physiological state changes and action execution.

[0044] When performing feature matching on multi-modal sensor arrays at different positions according to the behavior feature vector, the dynamic time warping (DTW) algorithm or cosine similarity is used to calculate the similarity of the feature vectors of different arrays. For example, if adjacent arrays show similar increases in heart rate variability and increases in the curvature of the action trajectory within a similar time period, it is determined as feature matching, indicating that there may be collaborative actions or chain reactions. Through the feature matching results, the propagation path and collaborative mode of the operation behavior in space are analyzed, and the operation behavior trend is deduced. For example, when feature matching continuously appears in multiple adjacent arrays, it may indicate the progress of a certain operation process or the spread of abnormal behavior.

[0045] In the process of processing the sign fusion matrix, each step is closely connected: standardization and hotspot extraction ensure that subsequent analysis focuses on the key data range, outlier filtering improves data quality, orthogonal decomposition and spatial correlation analysis separate behavior trends and spatial interaction features, the feature fusion network realizes the deep aggregation of multi-source data, and time-frequency domain feature matching reveals the spatial coordination law of behaviors. Through the above processing, the sign fusion matrix is transformed from the original multi-dimensional data set into a structured feature set containing physiological parameter distribution, action trajectory features, operation behavior trends and spatial correlation relationships, providing reliable input data for subsequent state stability modeling, behavior pattern prediction and safety strategy configuration. The whole process is based on technologies such as signal processing, machine learning and graph theory, realizing multi-dimensional quantitative analysis of the operator's behavior, avoiding the limitations of subjective judgment, and improving the accuracy and comprehensiveness of behavior feature extraction.

[0046] Example 3: When performing state stability modeling, it is necessary to combine the physiological parameter distribution and action trajectory features to realize the dynamic risk assessment of the operation area. The specific implementation method is as follows: According to the physiological parameter distribution, physiological parameter sampling points are extracted from each frame of sign fusion matrix data. The selection of sampling points is based on the significance of parameter changes, such as points where the heart rate exceeds a preset threshold (such as 1.2 times the resting heart rate), points where the absolute value of the action acceleration is greater than a specific value (such as 2m / s²), etc. These sampling points are associated and mapped with the action trajectory features. For example, the timestamp corresponding to a sampling point is aligned with the inflection point timestamp in the action trajectory to generate a state stability map. The map is presented in a two-dimensional grid form, where the horizontal and vertical axes correspond to the spatial coordinates of the operation area, and the color or gray value of each grid unit reflects the abnormal degree of physiological parameters or the mutation intensity of the action trajectory in that area, forming an intuitive visualization result of real-time risk distribution.

[0047] When spatially aligning the state stability maps collected by multiple sensor arrays, first establish a unified world coordinate system based on the physical deployment positions of each array, and transform the local maps of each array to the world coordinate system through coordinate transformation (translation, rotation, scaling) to ensure that the edge areas of adjacent maps coincide. After alignment, the data in the overlapping area is fused, for example, using the weighted average method (the weights are determined according to the acquisition accuracy of the array), to generate a global state stability distribution map, which integrates the physiological parameters and action trajectory information of the entire operation area and eliminates the limitations of a single array perspective.

[0048] When setting the risk judgment threshold, factors such as operation type and personnel physiological characteristics are comprehensively considered. For example, the heart rate threshold in the high-altitude operation scenario is set to 110 beats per minute, and the threshold for ordinary workstation operations is set to 90 beats per minute. Locate the risk source according to the physiological parameter values of the multi-frame physical signs fusion matrix. By continuously monitoring multiple frames of data (such as 50 frames, corresponding to 5 seconds), identify the area where the physiological parameters continuously exceed the threshold as the potential risk source. Calculate the difference in risk intensity between adjacent frames. The risk intensity is represented by a normalized value, ranging from 0 to 1, and the calculation formula is:

[0049] where ΔR is the difference in risk intensity between adjacent frames, and R t is the risk intensity value of the t-th frame, and R t-1 is the risk intensity value of the (t - 1)-th frame. If ΔR is greater than or equal to the risk judgment threshold (such as 0.3), it indicates that the risk state of this area has changed significantly, and there may be abnormal behavior.

[0050] When it is detected that the difference in risk intensity exceeds the standard, perform kinematic model constraint compensation on the current area. First, call the attitude transformation model corresponding to the current area. This model is established in advance based on the operation type. For example, the attitude model corresponding to high-altitude operations describes the center-of-gravity offset and balance parameters of personnel in different climbing postures, and the model corresponding to mechanical operations describes the dynamic relationship between limb movements and equipment operations. Calculate the deviation between the theoretical value and the actual monitored value of the current motion trajectory through the model, and iteratively correct the state stability distribution based on the deviation value: adjust the parameter weights of the abnormal area in the map, suppress noise interference, and enhance the representation of real risk signals. After correction, calculate the stability compensation value of the abnormal area, which is used for parameter adjustment of subsequent safety strategies (such as adjusting the response sensitivity of safety equipment).

[0051] When performing state stability modeling on the physical signs fusion matrix according to the state stability distribution, associate each element in the matrix with the corresponding area of the stability distribution map. For example, the heart rate data of a certain sensor array in the matrix corresponds to the stability level (low, medium, high) of the position of this array in the distribution map. Through the dynamic time warping technique, align the time series data of the matrix with the change trend of the stability level to construct a stability model that evolves over time. The model output includes the risk level and state transition probability of each area at different times (such as the probability of changing from "medium risk" to "high risk").

[0052] When performing trajectory annotation on the stability model of an area through action trajectory features, the coordinates of the key nodes (starting point, inflection point, ending point) of the action trajectory are matched with the spatial grid of the stability model, and the areas passed by the trajectory and their corresponding stability states are marked in the model. For example, if the action trajectory of a worker moves from a low-risk area A to a high-risk area B, the trajectory segment in the model will be marked as "A (low risk) → B (high risk)", and the timestamps of the areas passed by the trajectory are recorded simultaneously to form stability model data with spatio-temporal tags. This annotation process provides the correlation basis between the trajectory and the risk state for subsequent behavior pattern prediction, such as predicting the change trend of physiological parameters when a person enters a high-risk area.

[0053] In the entire process of state stability modeling, the accurate extraction of physiological parameter sampling points is the basis. The relevance between the sampling points and the action trajectory is ensured through threshold judgment and time alignment; the generation of maps and spatial alignment achieve the spatial integration of multi-source data, improving the accuracy of global risk assessment; the risk determination and compensation mechanism enhance the robustness of the system by dynamically correcting the model to handle sudden abnormal situations; stability modeling and trajectory annotation provide structured data in the spatio-temporal dimension for behavior analysis, supporting subsequent prediction and decision-making. This process closely combines real-time monitoring data with prior models and realizes the dynamic quantitative assessment of the state stability of the working area through multi-step processing such as data fusion, threshold judgment, and model correction, providing key intermediate-layer data support for the intelligent control of workers' behaviors.

[0054] Example 4: When calculating the behavior prediction information of each sub-area, it is necessary to combine the behavior field model, the gradient of physiological state changes, and the trend of operation behaviors, and dynamically estimate the future behavior distribution through spatial mapping, trend correction, and recursive drawing. The specific implementation method is as follows: Taking the main movement trajectory of the behavior field model as the reference line, the determination of the main movement trajectory is based on the statistical analysis of historical action trajectory data. By collecting a large amount of action trajectory data of workers in normal working scenarios and using density clustering algorithms (such as DBSCAN) to identify the frequently occurring movement paths, the center line or average path of these paths is used as the main movement trajectory. For example, in the assembly line working scenario, the main movement trajectory may appear as a straight line or a specific curve extending along the production line, reflecting the regular movement route of workers between workstations.

[0055] Take the peak position of the operation behavior in each frame of data as the reference point. The identification of the peak position is based on the frequency or intensity of the action occurrence. For each sub-region, calculate the number of action events occurring per unit time or the cumulative amount of action parameters (such as acceleration, force value), and determine the position point with the highest number or cumulative amount as the peak position. For example, if an operator frequently operates a certain device within a sub-region, the position of this device is marked as the peak position. Calculate the behavior offset between the reference point and the baseline. The calculation of the offset includes spatial distance offset and direction angle offset: the spatial distance offset is the vertical distance from the reference point to the baseline, and the direction angle offset is the included angle between the moving direction of the reference point and the tangent direction of the baseline. By quantifying the offset, the deviation degree of the current behavior relative to the typical behavior pattern can be judged. For example, a larger offset may indicate that the operator deviates from the standard process or has abnormal actions.

[0056] When drawing the behavior distribution curve according to spatial coordinates, use time as the horizontal axis and spatial position as the vertical axis, and connect the reference points and their offsets of each frame into a curve in sequence. The shape of the curve reflects the diffusion or convergence trend of the behavior in space. For example, if the curve diverges to both sides of the baseline, it may indicate that the action range of the operator expands, and if the curve converges to the baseline, it may indicate that the action tends to be standardized. By analyzing the changes in the slope and curvature of the curve, the stability of the behavior pattern can be further identified. For example, a sudden change in the slope may correspond to a sudden change in the action direction, and an increase in the curvature may correspond to the complication of the action trajectory.

[0057] When correcting the change rate and direction in the operation behavior trend according to the physiological state change gradient, first extract the key parameters in the physiological state change gradient, such as the change rate of physiological parameters, action fluctuation index, state change slope, etc. These parameters reflect the dynamic relationship between the physiological state and action execution. For example, an increase in the change rate of physiological parameters may be accompanied by a decrease in action stability. For the change rate (such as the increase or decrease rate of action speed) and direction (such as the deflection angle of the movement trajectory) in the operation behavior trend, adjust according to the numerical value and sign of the gradient parameters: if the change rate of physiological parameters is positive (indicating an increase in the parameter) and the action fluctuation index exceeds the threshold, then reduce the expected change rate of the action speed or increase the correction value of the movement trajectory deflection angle to reflect the potential impact of the physiological state on action execution. The correction rules are obtained based on prior knowledge or historical data training. For example, establish a mapping relationship model between physiological parameters and action parameters through regression analysis.

[0058] Starting from the nearest behavior distribution point, continue to draw the distribution curve according to the correction results of the change rate and direction. The drawing process uses a recursive method, and the position of each distribution point is calculated based on the position of the previous point, the corrected change rate, and direction. The specific steps are as follows: Determine the coordinates (x t , y t), calculate the velocity increment Δv at the next moment according to the corrected action speed change rate, and combine it with the current velocity v t to obtain the velocity v at the next moment t+1 = v t + Δv; according to the corrected direction deflection angle Δθ, update the motion direction angle θ t+1 = θ t + Δθ; based on the velocity and direction, calculate the position coordinates at the next moment . Repeat this process until the distribution points cover the entire target area, forming complete behavior prediction information.

[0059] The generation of behavior prediction information needs to consider the physical boundaries and equipment layouts of the target area. For working environments with obstacles or prohibited areas, these areas are automatically avoided when drawing the distribution curve. For example, by setting a spatial mask matrix, the coordinate points of the prohibited areas are set to invalid values. When the position calculated by recursion falls into the invalid area, the motion direction is automatically adjusted to bypass. In addition, for the multi-operator collaboration scenario, the behavior prediction information of each person needs to consider the action coupling between them. For example, by cross-verifying the predicted trajectories of adjacent personnel, conflicting behavior distribution points (such as predicting that two people appear at the same position at the same time) are avoided.

[0060] The generated behavior prediction information is stored in the form of a grid. Each grid cell corresponds to a sub-area, containing the types and probability values of the behavior patterns that may appear in this area in the next time period. The classification of behavior pattern types is based on the clustering results of the historical behavior feature dataset. For example, it is divided into categories such as "equipment debugging", "material handling", "abnormal stay", etc. The probability value is calculated through a logistic regression model or a Naive Bayes classifier, reflecting the possibility of this behavior pattern occurring during the target time period. For example, the prediction information of a certain grid cell may be "material handling, probability 65%; abnormal stay, probability 20%; others, probability 15%", providing input data for the subsequent behavior association model.

[0061] In the entire calculation process of behavior prediction information, the determination of the main motion trajectory lays the benchmark framework for behavior distribution. The calculation of reference points and offsets realizes the comparative analysis of real-time data and typical patterns. The correction mechanism of the physiological state change gradient introduces the dynamic association between physiology and behavior. Recursive drawing and spatial constraints ensure the physical feasibility of the prediction results. Through multi-step data processing and model calculation, this process combines real-time monitoring data, historical behavior patterns, and physiological state characteristics to generate behavior prediction results with spatio-temporal dimensions, providing a forward-looking basis for the extraction of spatio-temporal distribution characteristics of operation behavior patterns and the early configuration of safety strategies. By continuously iteratively updating the prediction information, the system can dynamically adapt to changes in the working environment and personnel behavior, improving the timeliness and accuracy of intelligent management and control.

[0062] Example 5: When dynamically configuring the safety policy for the operation area, precise response to safety protection needs to be achieved through links such as area mapping, policy triggering, path adjustment, and resource allocation. The implementation methods in specific application scenarios are as follows: Taking an assembly workshop in a factory as an example, there are multiple operation areas in the workshop, such as a material area, an assembly area, and a debugging area. Multimodal sensor arrays are deployed in each area to continuously monitor the heart rate, movement trajectory, and equipment operation signals of the operators. When the system analyzes through the sign fusion matrix and finds that the operators in a certain sub - area (such as Area C3) in the assembly area have had a heart rate continuously higher than 100 beats per minute and a movement trajectory frequently deviating from the standard process for 10 consecutive minutes, according to the spatio - temporal distribution characteristics output by the behavior correlation model, it is determined that the behavior pattern in this area reaches the preset risk level threshold (such as Level 3 high risk). At this time, the system performs the following dynamic safety policy configuration operations: Map the area identifier and classification features. Bind the area identifier of Area C3 to the "high - risk operation" category in the classification features. At the same time, retrieve the historical behavior feature data of this area and confirm that this behavior pattern has been associated with abnormal start - stop or operation error events of equipment many times. Based on this, the system triggers the linkage command for the protection equipment in adjacent areas: activate the audible and visual alarms in Areas D3 and C2 around Area C3, emit red warning lights and intermittent beeps to alert nearby personnel of potential risks; at the same time, send a vibration warning to the smart safety helmets of the operators in this area to prompt them to adjust their operation status.

[0063] In terms of operation path adjustment, the system dynamically combines the boundaries of safety areas according to the spatio - temporal distribution characteristics of the behavior pattern. For example, if the analysis shows that high - risk behaviors are concentrated in the middle of Area C3 and spread to the west, the system calculates and generates a detour path through the A* algorithm: guide the personnel who originally planned to pass through the west - side passage of Area C3 to move through the north - side standby passage to avoid the risk - spreading area. The path adjustment instructions are pushed synchronously through the LED displays in the workshop and the hand - held terminals of the operators. During the path planning process, areas where materials are stacked and equipment operation trajectories are avoided to ensure the safety and passing efficiency of the new path.

[0064] During the alarm level configuration operation, the system generates an alarm level vector with different priorities based on the risk level. Taking the C3 area as an example, the risk level Level 3 corresponds to the highest priority alarm. The system marks the alarm level vector of this area as (red, emergency, 10 minutes), where "red" represents the visual warning color, "emergency" represents the response level, and "10 minutes" represents the continuous monitoring duration. Based on this vector, adjust the sensor parameters of the monitoring nodes in the target area: increase the camera acquisition frame rate of the C3 area and adjacent arrays from 25 frames / second to 50 frames / second to enhance the ability to capture action details; increase the sampling frequency of the heart rate monitoring sensor from 1 time / second to 2 times / second to ensure real-time acquisition of physiological parameter changes.

[0065] In the dynamic allocation of protective resources, the system uses intelligent scheduling algorithms to optimize resource allocation based on the spatial distribution of behavioral patterns. For example, after detecting high-risk operations in the C3 area, the system automatically prioritizes the positioning tags of equipment such as helmets and protective gloves in the nearby emergency material cabinets, sends instructions to the inspection robot through the IoT gateway, and guides it to move to the entrance of the C3 area to stand by in order to quickly respond to personnel needs; at the same time, the automatic fire extinguishing system in the workshop is partitioned, and the fire extinguishing agent injection threshold in the C3 area is reduced from 80% of the standard value to 60%, shortening the response time in emergency situations.

[0066] In another application scenario, a sensor array is deployed in the high-altitude operation area of a construction site to monitor the workers' body movements and safety belt wearing status in real time. When the system detects that a worker (whose sensor corresponds to the E5 area of the array) has a significant increase in the movement fluctuation index during climbing the scaffold (such as three consecutive foot steps causing a sudden change in acceleration), and the gradient of the physiological state change shows that the heart rate change rate exceeds the threshold of 0.5 times / second, it is judged as a "high-altitude fall risk" warning. The system immediately implements the following strategies: trigger the automatic tensioning device of the safety net in the E5 area and the adjacent E4 and F5 areas to increase the load-bearing strength of the safety net; send a voice warning to the worker through the broadcasting system, prompting him to stop the action and check the safety belt connection; at the same time, send an alarm message containing the specific location and risk type to the terminal of the ground safety manager to start the on-site emergency response process.

[0067] The linkage of security policy configuration is also reflected in the coordination of multiple systems. For example, in a chemical workshop scenario, when the behavior analysis server detects abnormal physiological characteristics of poisoning (such as a sudden drop in respiratory rate and abnormal skin conductance) in a certain area, in addition to activating local protective equipment, it also sends instructions to the plant environment monitoring system through the industrial Internet interface to start real-time monitoring of toxic gas concentrations in the area, and transmits risk location information to the fire control system, reserving an interface for subsequent emergency disposal of toxic gas leaks.

[0068] The entire dynamic configuration process is centered around the spatio-temporal distribution characteristics of the behavior pattern. Through the precise mapping of area identifiers and risk levels, it realizes multi-dimensional policy responses such as activation of protection equipment, adjustment of operation paths, and upgrade of warning levels. The priority of policy execution and parameter settings are based on a rule base trained with historical data. For example, different risk thresholds and response action combinations are preset for different operation types (such as mechanical operations, high-altitude operations, chemical processing). Through real-time data-driven policy generation and dynamic adjustment, the system can minimize safety risks without interfering with normal operations, realizing the transformation from passive warning to active protection. This process closely combines the physical characteristics of the operation scenario and the behavior patterns of personnel, and through hierarchical and regional refined management and control, it improves the pertinence and effectiveness of safety policies.

[0069] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or device.

[0070] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent control method for worker behavior based on multi-modal sign recognition, which is applied to a behavior monitoring system. The system includes a plurality of multi-modal sensor arrays deployed in an operation area and a behavior analysis server connected to the multi-modal sensor arrays. The distance between two adjacent multi-modal sensor arrays is set at a certain interval. It is characterized in that The method includes: Collecting the physical sign state data and environmental interaction data of the operators in the area through the multi-modal sensor array to generate a behavior feature data set; Receiving the real-time physical sign fusion data of multiple multi-modal sensor arrays through the behavior analysis server to construct a physical sign fusion matrix; Determining the classification features of the operation behavior pattern according to the behavior feature data set and the real-time data of each multi-modal sensor array. Among them, determining the classification features of the operation behavior pattern includes: processing the physical sign fusion matrix, extracting pattern features in combination with the behavior feature data set, predicting the behavior pattern according to the physiological state change gradient and the action trajectory information, outputting the spatio-temporal distribution features of the operation behavior pattern through the behavior association model, and updating the behavior feature data set according to the spatio-temporal distribution features; Dynamically configuring the safety policy of the operation area according to the classification features.

2. The intelligent control method for worker behavior based on multi-modal sign recognition according to claim 1, characterized in that The determination of the classification features of the operation behavior pattern includes: Processing the physical sign fusion matrix to extract the physiological parameter distribution, action trajectory features and operation behavior trends; Conducting state stability modeling on the physical sign fusion matrix according to the physiological parameter distribution and action trajectory features, dividing the operation area into multiple sub-areas and marking area identifiers, associating and matching the physiological parameters of the sub-areas with the behavior feature data set, and marking area identifiers in the behavior feature data set; Calculating the physiological state change gradient according to the position of the multi-modal sensor array, predicting the distribution of the behavior pattern according to the physiological state change gradient and the operation behavior trend, and calculating the behavior prediction information of each sub-area; Constructing a behavior association model, using the behavior prediction information as the input parameter of the behavior association model, conducting spatial association modeling on the behavior prediction information through the behavior association model, and outputting the spatio-temporal distribution features of the operation behavior pattern; Updating the behavior feature data set according to the spatio-temporal distribution features to obtain the classification features of the operation behavior pattern.

3. The intelligent control method for worker behavior based on multi-modal sign recognition according to claim 2, characterized in that, The processing of the physical sign fusion matrix includes: Normalizing the physical sign fusion matrix, intercepting the behavior hot spot area in the matrix through a sliding window, filtering out the outliers in the hot spot area, and calculating the operation behavior trend through an orthogonal decomposition algorithm; Calculating the spatial association features of the physical sign fusion matrix, calculating the action coupling coefficient, posture stability index and risk area marking between regions according to the spatial association features, constructing a feature fusion network, and calculating the action trajectory features through the feature fusion network; Extracting the time domain features and spatial domain features collected by each multi-modal sensor array, calculating the behavior feature vector of the array according to the phase difference between the time domain features and the spatial domain features, performing feature matching on the multi-modal sensor arrays at different positions according to the behavior feature vector, and calculating the operation behavior trend.

4. The intelligent control method for workers' behavior based on multi-modal sign recognition according to claim 3, characterized in that, The state stability modeling of the physical sign fusion matrix includes: Extracting the physiological parameter sampling points in each frame of data according to the physiological parameter distribution, performing association mapping according to the sampling points and the action trajectory features to generate a state stability map, spatially aligning the state stability maps collected by multiple arrays, and calculating the state stability distribution of the region; Set a risk judgment threshold, locate the risk source according to the physiological parameter values of the multi-frame sign fusion matrix, calculate the risk intensity difference. If the risk intensity difference is greater than or equal to the risk judgment threshold, it indicates that there is an abnormal behavior in this area. Perform kinematic model constraint compensation on the current area, and according to the attitude transformation model corresponding to the current area, iteratively correct the state stability distribution of the current area, and calculate the stability compensation value of the abnormal area according to the correction result; Perform state stability modeling on the sign fusion matrix according to the state stability distribution, and perform trajectory annotation on the stability model of the area through the action trajectory characteristics.

5. The intelligent control method for workers' behaviors based on multi-modal sign recognition according to claim 4, characterized in that, The calculating the physiological state change gradient according to the multi-modal sensor array positions includes: Extract the physiological parameter change points according to multiple groups of sign fusion data, map the change points to a unified space coordinate system according to the deployment positions of the arrays, and fit the change points through a spatial interpolation algorithm to generate the behavior field model of the area; Perform equally spaced sampling along the motion trajectory of the behavior field model, calculate the physiological parameter change rate, action fluctuation index and state change slope of the trajectory according to the sampling results, and calculate the state change parameters according to the physiological parameter change rate, action fluctuation index and state change slope; According to the deployment parameters and acquisition accuracy of the multi-modal sensor array, project the distribution characteristics of the operation behavior in each frame of data onto the behavior field model, partition according to the number of arrays along the motion direction of the behavior field model, analyze the change rules of the operation behavior in the partition, and calculate the behavior distribution characteristics according to the change rules; Calculate the physiological state change gradient according to the state change parameters and the behavior distribution characteristics. The calculation process of the physiological state change gradient includes: based on the spatial position range from the first multi-modal sensor array to the last multi-modal sensor array, select spatial coordinate points in the array deployment direction, accumulate and calculate the product of the behavior field characteristic weight value and the operation behavior distribution characteristic weight value within the spatial resolution range, and superimpose the influence value of the array acquisition frequency on the physiological state change rate.

6. The intelligent control method for workers' behaviors based on multi-modal sign recognition according to claim 5, wherein, The calculating the behavior prediction information of each sub-region includes: Taking the main motion trajectory of the behavior field model as the reference line, taking the peak position of the operation behavior in each frame of data as the reference point, calculate the behavior offset, and draw the behavior distribution curve according to the spatial coordinates; Correct the change rate and direction in the operation behavior trend according to the physiological state change gradient; Starting from the nearest behavior distribution point, continue to draw the distribution curve according to the correction results of the change rate and direction to generate the behavior distribution points in the next time period until the distribution points cover the entire target area to generate the behavior prediction information.

7. The intelligent control method for worker behavior based on multi-modal sign recognition according to claim 2, characterized in that, The constructing the behavior association model includes: An input layer for organizing the behavior prediction information into spatial distribution data and performing normalization processing; A feature fusion layer for extracting the regional association features of the behavior by processing the spatial distribution data and constructing the dependence relationship between spatial units; A strategy generation layer for integrating the association relationships of the operation behavior on the spatial units to generate a security policy configuration sequence.

8. The intelligent control method for worker behavior based on multi-modal sign recognition according to claim 2, characterized in that, The obtaining the classification features of the operation behavior pattern includes: Correspond the identification of the sub-region with the spatio-temporal distribution characteristics of the job behavior pattern output according to the behavior association model; Reorganize the regional data in the behavior feature dataset according to spatio-temporal characteristics to generate a regional distribution map sorted by behavior risk level; Output the optimized behavior pattern classification features according to the reorganized regional distribution map.

9. The intelligent control method for worker behavior based on multi-modal sign recognition according to claim 1, characterized in that, The dynamic configuration of the safety policy for the operation area includes: Map the regional identification to the region of the classification features of the job behavior pattern; Control the execution actions of the safety policy according to the spatio-temporal distribution characteristics of the behavior pattern, including the activation of protection equipment, the adjustment of the operation path, and the configuration operation of the alarm level; Dynamically allocate protection resources to the corresponding spatial regions according to the spatial distribution of the behavior pattern and the preset safety policy rules; The control of the execution actions of the safety policy includes: When the behavior pattern reaches the preset risk level threshold in the target area, trigger the linkage command of the protection equipment in the adjacent area; Dynamically combine the boundaries of the safety area according to the operation path policy to generate an alarm level vector; Adjust the sensor parameters of the monitoring nodes in the target area based on the alarm level vector.

10. An intelligent control system for workers' behavior based on multi-modal physical sign recognition is used to implement an intelligent control method for workers' behavior based on multi-modal physical sign recognition as described in any one of claims 1 to 9, and is characterized in that, The system includes a plurality of multi-modal sensor arrays deployed in the operation area at a set interval, and a behavior analysis server connected to the multi-modal sensor arrays; wherein, the multi-modal sensor arrays are used to collect the physical sign state data and environmental interaction data of the operators in the area to generate a behavior feature dataset; the behavior analysis server is used to receive the real-time physical sign fusion data of the plurality of multi-modal sensor arrays, construct a physical sign fusion matrix, process the physical sign fusion matrix, extract pattern features in combination with the behavior feature dataset, and predict the behavior pattern according to the physiological state change gradient and the action trajectory information, output the spatio-temporal distribution characteristics of the job behavior pattern through the behavior association model, update the behavior feature dataset according to the spatio-temporal distribution characteristics to determine the classification features of the job behavior pattern, and dynamically configure the safety policy for the operation area according to the classification features.

Images