A method for identifying illegal actions of mine personnel based on constructive federated learning
By constructively federated learning to jointly train network models in multiple mining areas, and using Openpose to extract miner skeleton nodes and perform multi-dimensional feature analysis, the problem of difficult identification of illegal behavior by miners in mining areas is solved, thereby improving regulatory efficiency and safety.
Patent Information
- Application Number
- CN202210393676.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-14
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2042-04-14
AI Technical Summary
In existing technologies, it is difficult to effectively identify violations by miners through artificial intelligence, especially in the absence of sufficient labeled data, resulting in low regulatory efficiency and a high accident rate.
By employing a constructivist federated learning approach, video data is acquired by installing cameras on belt conveyors in multiple mining areas. Openpose is used to extract miner skeleton nodes, and multidimensional feature analysis is combined to construct a local network model. Adaptive parameter aggregation is then performed in a trusted third-party aggregation cloud to form an efficient violation behavior identification model.
This approach has improved the accuracy and efficiency of identifying violations while ensuring data privacy, reduced the accident rate in mining areas, and alleviated the burden on monitoring personnel.
Smart Images

Figure CN115147750B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of illegal behavior recognition of workers in mining areas, and in particular to a method for identifying illegal actions of mine personnel based on constructive federated learning. Background Art
[0002] my country is a country with vast land and abundant resources. Its mineral resources rank among the top in the world and its potential is enormous. There are countless mined and unmined mining areas, and countless amounts of various ores are mined each year, which continuously inject energy and materials into my country's development.
[0003] Safety has always been a big problem in ore mining and development. Various mining accidents are shocking and pose great danger to the lives of miners and the safety of equipment. Among them, the illegal behavior of workers during operation is a huge hidden danger. The traditional method to eliminate workers' illegal behavior is usually to stare at the monitoring screen manually. This kind of supervision purely relying on monitoring personnel has many disadvantages, such as: long-term high-intensity monitoring by monitoring personnel is prone to visual fatigue, too many monitoring areas to take care of all, etc. If artificial intelligence monitoring methods are used, the supervision effect can be greatly improved, accidents can be prevented, and the safety of workers and equipment can be guaranteed. However, the current artificial intelligence monitoring methods usually require a large amount of manually labeled data to train the recognition network model. It is usually difficult to train a network model with good recognition performance based solely on data in a certain mining area. Summary of the invention
[0004] The present invention aims to solve the defects of the prior art in the above-mentioned problems and provides a method for identifying illegal actions of mine personnel based on constructive federated learning. Since the mines participating in constructive federated learning have similar equipment, similar industrial processes, similar actions during mining operations, and similar characteristics, a network model with better recognition performance can be jointly trained while ensuring the data privacy of each mine, thereby replacing manual monitoring, improving supervision efficiency, and reducing the accident rate.
[0005] In view of the above technical problems, the present invention provides a method for identifying illegal actions of mine personnel based on constructive federated learning, which includes the following steps:
[0006] Step 1: Obtain historical videos of miners’ behaviors in the belt area of each mining area by installing RGB cameras above the belt conveyors of more than two mining areas, use human skeleton extraction tools to analyze and extract feature data of miners’ behaviors in the belt area from the video stream data, manually annotate the behavior categories and organize them into a local miners’ behavior database of each mining area;
[0007] As a human skeleton extraction tool, Openpose can extract the skeleton nodes of miners in video stream data. According to the extracted skeleton nodes, key frames can be filtered and multi-dimensional behavior features of miners can be extracted. The multi-dimensional features include the relative distance feature value of miners' limbs, the angle feature value of miners' limbs, the inter-frame velocity feature value of miners' limbs, and the inter-frame angular velocity feature value of miners' limbs. The details are as follows:
[0008] Openpose is used to extract the miner's skeleton nodes. 15 skeleton nodes are artificially selected from the extracted miner's skeleton nodes, and they form 14 rigid bodies that can represent the miner's body. The relative distance eigenvalue of the miner's limbs, that is, the length of the 14 rigid bodies of the miner's body, is to calculate the relative distance of the skeleton nodes at both ends of the 14 rigid bodies. For example, the length of a rigid body from the right shoulder to the center of the shoulder is taken as an example. Let the coordinates of the skeleton node at the center of the shoulder be p2(x2,y2), and the coordinates of the skeleton node of the right shoulder be p4(x4,y4). Then the length d2 of the second rigid body is:
[0009] d2=|p2(x2,y2)-p2(x4,y4)|
[0010] where p i (x i ,y i ) represents the two-dimensional coordinates of skeleton node i.
[0011] In order to eliminate the difference in individual skeleton size, the above relative distances are normalized, that is, the relative distances between all skeleton nodes are divided by the relative distance from the center point of the shoulder to the center point of the hip, that is:
[0012]
[0013] Where h represents the relative distance from the center point of the shoulder to the center point of the hip.
[0014] For the nth frame of the miner's behavior image, the miner's limb relative distance feature value L n It is expressed as:
[0015] L n ={d n,1 ,d n,2 ,...,d n,14 |n=1,2,...,N}
[0016] Where n represents the nth frame image in the total length N of the miner behavior video, d n,i Represents the relative distance feature value of the limbs of the i-th miner in the n-th frame image.
[0017] The eigenvalue of the miner's limb angle is the eigenvalue of the angle formed between the miner's limbs. We artificially select 14 key angles of the miner's human skeleton and take the right arm as an example to calculate the eigenvalue of the miner's limb angle θ3. Let the right shoulder skeleton node coordinates be p4 (x4, y4), the right elbow skeleton node coordinates be p5 (x5, y5), and the right hand skeleton node coordinates be p6 (x6, y6). Then the eigenvalue of the miner's limb angle θ3 is:
[0018]
[0019] where p i (x i ,y i ) represents the two-dimensional coordinates of skeleton node i.
[0020] For the nth frame of the miner's behavior image, the miner's limb angle feature value θ n It is expressed as:
[0021] θ n ={θ n,1 ,θ n,2 ,...,θ n,14 |n=1,2,...,N}
[0022] Where n represents the nth frame image in the total length N of the miner behavior video, θ n,i Represents the eigenvalue of the limb angle of the i-th miner in the n-th frame image.
[0023] Use cosine similarity to extract key frames from the video data stream to reduce resource consumption and speed up the recognition of the model. For the two frames of miners’ limb angle feature values:
[0024] θ n-1 (θ n-1,1 ,θ n-1,2 ,...,θ n-1,14 ),θ n (θ n,1 ,θ n,2 ,...,θ n,14 )
[0025] Where n-1 and n represent two adjacent frames in the video, θ n-1,i ,θ n,i They represent the i-th angle eigenvalue in the n-1-th frame and the i-th angle eigenvalue in the n-th frame respectively.
[0026] The similarity between two frames is calculated as follows:
[0027]
[0028] Where cos(θ) represents the similarity between two frames. The closer the value is to 1, the more similar the two frames are. When it exceeds the set threshold, the similar frames are discarded. The threshold is generally obtained through experimental testing.
[0029] The inter-frame miner limb velocity feature value is the average speed of the miner's same skeleton node moving between two frames. Let n-1, n represent the miner's behavior images in two adjacent frames, then the inter-frame displacement S of skeleton node i is n,i for:
[0030] S n,i =|p n,i (x n,i ,y n,i )-p n-1,i (x n-1,i ,y n-1,i )|
[0031] where p n,i (x n,i ,y n,i ) represents the two-dimensional coordinates of the i-th skeleton node in the n-th frame.
[0032] Let the time interval between two frames be t, then the inter-frame speed V of the miner skeleton node i in the nth frame n,i for:
[0033]
[0034] The inter-frame velocity characteristic value of the miner's body in the nth frame can be expressed as:
[0035] V n = {V n,1 ,V n,2 ,...,V n,15 |n=1,2,...,N}
[0036] Where n represents the nth frame image in the total length N of the miner behavior video, V n,i Represents the eigenvalue of the miner's limb velocity in the i-th frame of the n-th frame image.
[0037] The inter-frame angular velocity characteristic value of the miner's limbs is the average speed of the change of the same key angle between two frames. Let n-1, n represent the miner's behavior images in two adjacent frames, then the inter-frame change of the key angle i is Ω n,i for:
[0038] Ω n,i =θ n,i -θ n-1,i
[0039] where θ n,i Represents the eigenvalue of the limb angle of the i-th miner in the n-th frame.
[0040] Let the time interval between two frames be t, then the inter-frame angular velocity ω of the miner's limb angle i in the nth frame is n,i for:
[0041]
[0042] The angular velocity characteristic value of the miner's body between frames of the nth frame can be expressed as:
[0043] ω n ={ω n,1 ,ω n,2 ,...,ω n,14 |n=1,2,...,N}
[0044] Where n represents the nth frame image in the total length N of the miner behavior video, ω n,i Represents the angular velocity feature value of the miner's limbs in the i-th frame of the n-th frame image.
[0045] The multi-dimensional feature combination derives the miner behavior feature x in the nth frame n ,Right now:
[0046] x n ={L n ,θ n ,V n ,ω n ,T n |n=1,2,...,N}
[0047] ={L n,1 ,L n,2 ,...,L n,14 ,θ n,1 ,θ n,2 ,...,θ n,14 ,V n,1 ,V n,2 ,...,V n,15 ,ω n,1 ,ω n,2 ,...,ω n,14 ,T n |n=1,2,...,N}
[0048] Where N represents the total number of miner behavior image frames, L n,i represents the relative distance feature value of the limbs of the i-th miner in the n-th frame, θ n,i represents the eigenvalue of the limb angle of the i-th miner in the n-th frame, V n,i represents the eigenvalue of the miner’s limb velocity between the i-th frames in the n-th frame, ω n,i represents the angular velocity characteristic value of the miner’s limbs between the i-th frames in the n-th frame, T n Indicates the miner behavior category label of the nth frame.
[0049] Assume that there are N frames of miner behavior videos with behavior category labels in the violation identification terminal k, then the local miner behavior database X k It can be expressed as:
[0050]
[0051] in Represents the multi-dimensional features of the i-th frame of the miner behavior image at the violation identification terminal k.
[0052] Step 2: Set up a constructive federated learning aggregation cloud at a trusted third party. Each mining area acts as a violation identification terminal and participates in constructive federated learning. When constructing a local network model at the violation identification terminal, the local miner behavior database of each mining area is used as the feature database. Starting from a zero network, hidden nodes are gradually added. For each new node constructed in the local network model, two or more groups of hidden nodes with different parameters are randomly configured. Then, based on the principle of maximizing the discriminant value, the optimal hidden node is selected, and the optimal hidden node and the current variance of the local network model of the violation identification terminal are transmitted to the aggregation cloud.
[0053] In step 2, a randomly configured neural network is used as the local network model of each violation identification terminal, and k represents the kth violation identification terminal among all violation identification terminals participating in constructive federated learning. When the violation identification terminal k starts to participate in constructive federated learning, the local network model learning parameters are initialized, including: the maximum number of random configurations T max , expected recognition error ε k 、The random configuration range of hidden layer parameters Υ={λ min :Δλ:λ max}、Learning parameter r k , activation function g(.), initial residual e0=T k , where λ min is the lower bound of the distribution interval of the random parameter, λ max is the upper limit of the random parameter allocation interval, Δλ is the random parameter allocation interval increment parameter, T is the actual category label of the miner behavior feature data in the local belt area of the violation identification terminal k, and ε k represents the expected tolerance set by the violation identification terminal k, r k represents the learning parameters set by the violation identification terminal k, and the maximum number of random configurations T max , the hidden layer parameter random configuration range Υ and the activation function g(.) are set the same for all violation identification terminals.
[0054] In step 2, when the illegal identification terminal k constructs a new hidden layer node L, in the symmetric interval Υ={-λ min :λ min Random configuration T maxGroup different parameters and Hidden layer nodes, calculate the output matrix of the current hidden layer nodes of the local network model
[0055]
[0056] The superscript T represents the transpose of a matrix or a vector. represents the M-th dimension input feature of the local network model of the violation identification terminal k;
[0057] Set a non-negative real number sequence L is the total number of hidden layer nodes in the current local network model, r k represents the learning parameters of the violation identification terminal k;
[0058] The randomly configured T max The parameters of the group hidden layer nodes are respectively brought into the constraint discriminant:
[0059]
[0060]
[0061] In the formula, Q represents the dimension of the network model output, and the symbol · represents the inner product of the vector. represents the discrimination value corresponding to the qth output of the local network model when the number of hidden layer nodes in the violation identification terminal k is L, represents the node discrimination value randomly configured by the current violation identification terminal k at the Lth iteration, It represents the qth output error of the local network model when the local network model of the illegal identification terminal k is the L-1th hidden layer node; if the randomly configured hidden layer nodes all make the constraint discriminant is less than zero, then the update random configuration interval is Υ={-(λ min +Δλ):λ min +Δλ}, reconfigure T max Substitute the group hidden layer node parameters into the constraint discriminant calculation until the random configuration makes the constraint discriminant greater than zero;
[0062] If there is only one set of randomly configured hidden layer node parameters that makes the constrained discriminant greater than zero, then this set of parameters is the optimal hidden layer node parameters; if there are more than two sets of randomly configured hidden layer node parameters that make the constrained discriminant greater than zero, then the set of parameters that makes the constrained discriminant take the largest value is selected as the optimal hidden layer node parameters.
[0063] According to the selected local network model, the optimal hidden layer node parameters Calculate the current hidden layer output matrix of the local network model And the output weight matrix The calculation formula is as follows:
[0064]
[0065] in represents the hidden layer output matrix of the L-1 hidden layer nodes before the Lth hidden layer node, represents the parameters of the L-1 hidden nodes before the Lth hidden node determined by constructive federated learning, represents the hidden layer output matrix of the Lth hidden layer node, Represents the optimal hidden layer node selected according to the constraint discriminant.
[0066] The variance of the current local network model The calculation formula is as follows:
[0067]
[0068] Where T k Represents the true label of the behavior classification of the miner behavior image in the violation identification terminal k.
[0069] The selected optimal hidden layer node parameters and the current variance of the local network model are transmitted to the aggregation cloud.
[0070] Step 3: The aggregation cloud receives the optimal hidden layer nodes of all violation identification terminals, performs adaptive parameter aggregation, constructs an aggregated hidden layer node, and transmits the aggregated hidden layer node to each violation identification terminal;
[0071] In step 3, the following formula is used to construct the adaptive aggregation hidden layer nodes:
[0072]
[0073] in, represents the parameter of the aggregated hidden layer node L, K represents the total number of violation identification terminals participating in constructive federated learning, represents the optimal hidden node parameters of the Lth hidden node of the violation identification terminal k, represents the adaptive aggregation parameter of the Lth hidden layer node of the violation identification terminal k, It represents the variance of the Lth hidden layer node of the local network model of the illegal identification terminal k.
[0074] Step 4: Each violation identification terminal receives the aggregated hidden layer node, updates the newly added hidden layer node parameters of each violation identification terminal, and calculates the local network model output weight parameters to obtain the current local network model recognition error. According to the settings of each violation identification terminal, it is determined whether the current local network model recognition error meets the recognition accuracy requirements of the illegal actions of miners in the belt area in the violation identification terminal. If so, the violation identification terminal exits the constructed federated learning; otherwise, it returns to step 2 to continue to construct new hidden layer node parameters for the next round of federated learning.
[0075] In step 4, the hidden layer output of the Lth hidden layer node of the local network model is calculated using the following formula:
[0076]
[0077] in Represents the aggregate hidden layer node parameters determined according to constructive federated learning.
[0078] The local network model output weight matrix of the violation identification terminal is calculated using the following formula:
[0079]
[0080] The current hidden layer output matrix is The superscript + is used to find the generalized inverse of a matrix.
[0081] The output of the local network model of the violation identification terminal is calculated using the following formula
[0082]
[0083] Then the current local network model recognition error is obtained, and it is judged whether the local network model recognition error of the current violation identification terminal meets the expected recognition error ε of the illegal action of the miner behavior in the belt area of the terminal k If the requirements are met, the terminal exits the constructive federated learning; if not, it returns to step 2 and continues to construct new hidden layer node parameters for the next round of federated learning.
[0084] Step 5: After the violation identification terminal completes the constructive federated learning, it identifies the miner behavior category based on the real-time behavioral characteristics of the mining operations in the belt area.
[0085] Beneficial effects: The method for identifying illegal actions of mine personnel based on constructive federated learning of the present invention can not only effectively solve the problems of difficulty in real-time monitoring of illegal and dangerous behaviors of workers in mining areas, long-term high-intensity monitoring of monitoring personnel, visual fatigue and excessive burden, etc., but also the federated training improves the accuracy of the recognition model, reduces the accident rate in mining areas, is easy to use, and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS
[0086] Figure 1 This is a flow chart of a method for identifying illegal actions of mine personnel based on constructive federated learning in an embodiment of the present invention;
[0087] Figure 2 A schematic diagram of model training according to an embodiment of the present invention;
[0088] Figure 3 Schematic diagram of a miner's skeleton according to an embodiment of the present invention. DETAILED DESCRIPTION
[0089] The following is a further description of the embodiments of the present invention in conjunction with the accompanying drawings:
[0090] like Figure 1 , Figure 2 As shown, the present invention provides a method for identifying illegal actions of mine personnel based on constructive federated learning, which is specifically implemented as follows:
[0091] Step 1: Assume that there are 10 mining areas participating in constructive federated learning. Each mining area is equipped with a belt conveyor. The historical videos of miners' working behaviors in the belt areas of each mining area are obtained by installing RGB cameras above the belt conveyors of the 10 mining areas. The human skeleton extraction tool Openpose is used to filter key frames and extract multi-dimensional behavioral features of miners from the video stream data. The multi-dimensional features include the relative distance feature values of miners' limbs, the angle feature values of miners' limbs, the inter-frame limb velocity feature values of miners, and the inter-frame angular velocity feature values of miners' limbs. The behavior categories are manually labeled and organized into local miner behavior databases for each mining area.
[0092] like Figure 3 As shown, 15 miner skeleton nodes that can represent the miner's behavior are artificially selected, and the 15 skeleton nodes form 14 segments of rigid bodies that can represent the miner's body. 14 key angles of the miner's body are artificially selected. The 14 segments of the Euclidean distances of the skeleton nodes at both ends of the rigid body are calculated as the relative distance feature values of the miner's limbs. In order to eliminate the differences in the individual skeleton sizes of the miners, the above relative distances are normalized, that is, the relative position difference between all bone points is divided by the relative position difference from the center point of the shoulder to the center point of the hip.
[0093] The 14 angle eigenvalues formed between the 14 rigid bodies are calculated as the angle eigenvalues of the miner's limbs.
[0094] In order to reduce resource consumption and speed up model recognition, cosine similarity is used to extract key frames from video stream data, and only the key frames of miners' actions are retained. The similarity threshold is set to 0.9823, and the next frame of two adjacent frames exceeding the threshold is discarded.
[0095] The average speed of the same skeleton node of the miner between two frames is calculated as the inter-frame miner limb speed feature value. The spatial position of the miner skeleton node is known, and the time interval between two frames of the video stream data is fixed, so the time interval between the two frames can be known after extracting the key frame, so the speed of the miner skeleton node can be obtained by calculation.
[0096] The average speed of the miner's limb angle change between two frames is calculated as the inter-frame miner limb angular velocity characteristic value. The miner's limb angles in two adjacent frames are known, and the time interval between two frames of the video stream data is fixed. Therefore, the time interval between the two frames can be known after extracting the key frame, so the miner's limb angular velocity can be calculated.
[0097] Then the multi-dimensional feature combination of the miner behavior of the n-th frame miner behavior image can be expressed as x n ,Right now:
[0098] x n ={L n ,θ n ,V n ,ω n ,T n |n=1,2,...,N}
[0099] ={L n,1 ,L n,2 ,...,L n,14 ,θ n,1 ,θ n,2 ,...,θ n,14 ,V n,1 ,V n,2 ,...,V n,15 ,ω n,1 ,ω n,2 ,...,ω n,14 ,T n |n=1,2,...,N}
[0100] Where N represents the total number of miner behavior image frames, L n,i represents the relative distance feature value of the limbs of the i-th miner in the n-th frame, θ n,i represents the eigenvalue of the limb angle of the i-th miner in the n-th frame, V n,i represents the eigenvalue of the miner’s limb velocity between the i-th frames in the n-th frame, ω n,i represents the angular velocity characteristic value of the miner’s limbs between the i-th frames in the n-th frame, T n Indicates the miner behavior category label of the nth frame.
[0101] The mining belt area operation behavior feature database of the first mining area can be expressed as:
[0102]
[0103] Where N means there are N frames of miners' belt area behavior video frames in the first mining area.
[0104] Step 2: 10 mining areas are used as 10 violation identification terminals to participate in constructive federated learning. Taking the first violation identification terminal as an example, the initialization of the local network model learning parameters includes: the maximum number of random configurations T max =10, expected recognition error ε 1 =0.05, random configuration range of hidden layer parameters Υ = {1:1:10}, learning parameter r 1 = 0.9, activation function g(.) selects Sigmoid (S curve) function g(x) = 1 / (1+exp(-x)), initial residual e0 = T 1 .
[0105] Take the first terminal as an example to construct a local network model, start from zero network, gradually increase hidden layer nodes, when constructing the hidden layer nodes of the 100th local network model, according to the mining belt area operation behavior characteristic database of miners in this mining area, construct 10 groups of hidden layer nodes with different parameters that obey Gaussian distribution in the adjustable symmetric interval [-1,1], that is, input weights and bias Calculate the hidden layer output matrix
[0106]
[0107] The superscript T represents the transpose of a matrix or a vector. Represents the i-th dimension input feature of the local network model of the violation identification terminal 1.
[0108] set up L is the total number of hidden layer nodes in the current local network model, i.e. L = 100, r 1 =0.9 represents the learning parameter, is a sequence of non-negative real numbers;
[0109] Substitute the 10 randomly configured sets of hidden layer node parameters into the constraint discriminant respectively:
[0110]
[0111]
[0112] Where L=100 represents the number of hidden nodes in the current local network model, and Q=2 represents a two-classification task of whether the miner's behavior is illegal or not.
[0113] If the 10 randomly generated groups of hidden layer nodes all use the constraint discriminant If it is less than zero, the update configuration interval is Υ={-2:2}, and 10 sets of hidden layer node parameters are reconfigured and substituted into the discriminant calculation until the random configuration makes the constraint discriminant greater than zero.
[0114] If there is only one set of randomly configured hidden layer node parameters that makes the constrained discriminant greater than zero, then this set of parameters is the optimal hidden layer node parameters; if there are more than two sets of randomly configured hidden layer node parameters that make the constrained discriminant greater than zero, the set of hidden layer node parameters that makes the constrained discriminant take the largest value is selected as the optimal hidden layer node parameters.
[0115] According to the selected local network model, the optimal hidden layer node parameters Calculate the current hidden layer output matrix of the local network model And the output weight matrix The calculation formula is as follows:
[0116]
[0117] in represents the hidden layer output matrix of the 99 hidden layer nodes before the 100th hidden layer node in the violation identification terminal 1, represents the parameters of the 99 hidden nodes before the 100th hidden node determined by constructive federated learning, represents the hidden layer output matrix of the 100th hidden layer node in the violation identification terminal 1, It represents the 100th optimal hidden layer node parameter in the violation identification terminal 1 selected according to the constraint discriminant.
[0118] The variance of the current local network model in the violation identification terminal 1 The calculation formula is as follows:
[0119]
[0120] Where T 1 Represents the true behavior classification label of the miner behavior image in the violation identification terminal 1.
[0121] The violation identification terminal 1 transmits the optimal 100th hidden layer node parameter selected by the constraint discriminant and the current local network model variance to the aggregation cloud.
[0122] Step 3: The aggregation cloud receives the 100th optimal hidden layer node parameters and variances of the 10 violation identification terminals for parameter aggregation. The aggregation cloud constructs the aggregated hidden layer node parameters by weighted aggregation. The calculation formula is as follows:
[0123]
[0124] in, Represents the parameters of the 100th aggregate hidden layer node of the aggregate structure, represents the adaptive hidden layer node parameters of the 100th hidden layer node of the violation identification terminal k, represents the optimal aggregation parameter of the 100th hidden node of the violation identification terminal k, It represents the current variance of the local network model when the violation identification terminal k constructs the 100th hidden layer node.
[0125] The aggregation cloud transmits the constructed aggregation hidden layer node parameters to each violation identification terminal.
[0126] Step 4: The first violation identification terminal receives the aggregated hidden layer node, updates the parameters of the 100th newly added hidden layer node in its network model, and calculates the hidden layer output of the 100th hidden layer node in the local network model of the violation identification terminal 1 using the following formula:
[0127]
[0128] The following formula is used to calculate the local network model output weight matrix of the violation identification terminal 1:
[0129]
[0130] The current hidden layer output matrix is The superscript + is used to find the generalized inverse of a matrix.
[0131] The current output of the local network model of the violation identification terminal 1 is calculated using the following formula:
[0132]
[0133] Then the recognition error of the current network model is obtained. If the recognition error of the current network model is less than the expected recognition error ε 1 , the terminal exits the constructive federated learning; if it is greater than the model's expected recognition error ε 1 , then return to step 2 and continue to construct new hidden layer node parameters for the next round of federated learning.
[0134] Step 5: The violation identification terminal network completed through constructive federated learning can distinguish the miners' behavior categories according to the real-time behavioral characteristics of mining operations in the belt area.
[0135] Taking the violation identification terminal in the first mining area as an example, after exiting the constructed federated learning, the local network model is fixed. When in use, the human skeleton extraction tool openpose extracts the skeleton nodes in the real-time behavior video of the miners in the belt area, and sorts out the real-time multi-dimensional behavior characteristics of the miners in the belt area based on the skeleton nodes. The local network model in the violation identification terminal takes the real-time multi-dimensional behavior characteristics of the miners in the belt area as input, and calculates and outputs the current behavior category of the miners in the belt area.
[0136] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for identifying illegal actions of mine personnel based on constructive federated learning, characterized in that: The following steps are involved: Step 1: Obtain historical videos of miners’ behaviors in the belt area of each mining area by installing RGB cameras above the belt conveyors of more than two mining areas, use human skeleton extraction tools to analyze and extract feature data of miners’ behaviors in the belt area from the video stream data, annotate the behavior categories and organize them into a local miners’ behavior database of each mining area; Step 2: Set up a constructive federated learning aggregation cloud. Each mining area acts as a violation identification terminal and participates in constructive federated learning. When the violation identification terminal constructs a local network model, the local miner behavior database of each mining area is used as the feature database. Starting from a zero network, hidden nodes are gradually added. For each new node constructed in the local network model, two or more groups of hidden nodes with different parameters are randomly configured. Then, based on the principle of maximizing the discriminant value, the optimal hidden node is selected, and the optimal hidden node and the current variance of the local network model of the violation identification terminal are transmitted to the aggregation cloud. Step 3: The aggregation cloud receives the optimal hidden layer nodes of all violation identification terminals, performs adaptive parameter aggregation, constructs an aggregated hidden layer node, and transmits the aggregated hidden layer node to each violation identification terminal; Step 4: Each violation identification terminal receives the aggregated hidden layer node, updates the newly added hidden layer node parameters of each violation identification terminal, and calculates the local network model output weight parameters to obtain the current local network model recognition error. According to the settings of each violation identification terminal, it is determined whether the current local network model recognition error meets the recognition accuracy requirements of the illegal actions of miners in the belt area in the violation identification terminal. If so, the violation identification terminal exits the constructed federated learning. Otherwise, return to step 2 to continue constructing new hidden layer nodes and conduct the next round of federated learning; Step 5: After the violation identification terminal completes the constructive federated learning, it identifies the miner behavior category based on the real-time behavioral characteristics of the mining operations in the belt area.
2. The method according to claim 1, characterized in that In step 2, a randomly configured neural network is used as the local network model of each violation identification terminal, and k represents the kth violation identification terminal among all violation identification terminals participating in constructive federated learning. When the violation identification terminal k starts to participate in constructive federated learning, the local network model learning parameters are initialized, including: the maximum number of random configurations T max , expected tolerance ε k 、The random configuration range of hidden layer parameters Υ={λ min :Δλ:λ max }、Learning parameter r k , activation function g(.), initial residual e0=T k , where λ min is the lower bound of the distribution interval of the random parameter, λ max is the upper limit of the random parameter allocation interval, Δλ is the random parameter allocation interval increment parameter, T k is the actual category label of the local belt area miner behavior feature data of the violation identification terminal k, ε k represents the expected tolerance set by the violation identification terminal k, r k represents the learning parameters set by the violation identification terminal k, and the maximum number of random configurations T max , the hidden layer parameter random configuration range Υ and the activation function g(.) are set the same for all violation identification terminals.
3. According to the method of claim 2, in step 2, when the illegal identification terminal k constructs a new hidden layer node L, in the symmetric interval Υ={-λ min :λ min Random configuration T max Group different parameters and hidden layer nodes, calculate the output matrix of the current hidden layer nodes of the local network model The superscript T represents the transpose of a matrix or a vector. represents the M-th dimension input feature of the local network model of the violation identification terminal k; Set a non-negative real number sequence L is the total number of hidden layer nodes in the current local network model, r k represents the learning parameters of the violation identification terminal k; The randomly configured T max The group hidden layer node parameters are substituted into the following constraint discriminant: In the formula, Q represents the dimension of the network model output, and the symbol · represents the inner product of the vector. represents the discrimination value corresponding to the qth output of the local network model when the number of current hidden layer nodes in the violation identification terminal k is L, represents the node discrimination value randomly configured by the violation identification terminal k at the Lth iteration, It represents the qth output error of the local network model when the local network model of the illegal identification terminal k is the L-1th hidden layer node; if the randomly configured hidden layer nodes all make the constraint discriminant is less than zero, then the update random configuration interval is Υ={-(λ min +Δλ):λ min +Δλ}, reconfigure T max Substitute the group hidden layer node parameters into the constraint discriminant calculation until the random configuration makes the constraint discriminant greater than zero; If there is only one set of randomly configured hidden layer node parameters that makes the constraint discriminant greater than zero, then this set of parameters is the optimal hidden layer node parameters; if there are more than two sets of randomly configured hidden layer node parameters that make the constraint discriminant greater than zero, then the set of parameters that makes the constraint discriminant value the largest is selected as the optimal hidden layer node parameters; According to the selected local network model, the optimal hidden layer node parameters Calculate the current hidden layer output matrix of the local network model And the output weight matrix The calculation formula is as follows: in represents the hidden layer output matrix of the L-1 hidden layer nodes before the Lth hidden layer node, represents the parameters of the L-1 hidden nodes before the Lth hidden node determined by constructive federated learning, represents the hidden layer output matrix of the Lth hidden layer node, represents the optimal hidden layer node selected according to the constraint discriminant; The variance of the current local network model The calculation formula is as follows: Where T k Represents the true label of the behavior classification of the miner behavior image in the violation identification terminal k.
4. The method according to claim 3, characterized in that In step 3, the following formula is used to construct the adaptive aggregation hidden layer nodes: in, represents the parameter of the aggregated hidden layer node L, K represents the total number of violation identification terminals participating in constructive federated learning, represents the optimal hidden node parameters of the Lth hidden node of the violation identification terminal k, represents the adaptive aggregation parameter of the Lth hidden layer node of the violation identification terminal k, It represents the variance of the Lth hidden layer node of the local network model of the illegal identification terminal k.
5. The method according to claim 4, characterized in that In step 4, the hidden layer output of the Lth hidden layer node of the local network model is calculated using the following formula: in Represents the aggregate hidden layer node parameters determined by constructive federated learning; The local network model output weight matrix of the violation identification terminal is calculated using the following formula: The current hidden layer output matrix is The superscript + is used to find the generalized inverse of a matrix; The output of the local network model of the violation identification terminal is calculated using the following formula
Citation Information
Patent Citations
Method for recognizing human knee motion postures based on extreme learning machine
AU2020103811A4
Behavior modal identification method of random configuration network for dynamically updating output weight
CN112132096A
Cited By
Mine violation identification system, method, electronic device and readable storage medium
CN122867238A
