Miner behavior monitoring method and device

Abstract

The invention discloses a miner behavior monitoring method and device, and the method comprises the steps: collecting multi-source data in a miner operation environment through a preset sensor, and carrying out the data cleaning of the multi-source data; fusing the cleaned multi-source data to obtain fused data; and finally, target detection is performed on the fused data to extract miner behaviors, and the miner behaviors are input into the interaction model to obtain miner behavior states, so that the miner behavior states can be comprehensively and accurately monitored, and the miner behavior safety is more effectively guaranteed.

Description

Technical Field

[0001] The invention belongs to the technical field of underground mine safety, and in particular relates to a method and a device for monitoring miner behavior. Background Art

[0002] In mining production, the safety and production efficiency of miners have always been the core concerns of the industry's development, especially the safety of miners, which is the top priority in mining production. However, in actual mining operations, due to the particularity of the environment, such as insufficient light, excessive dust and loud noise, the existing miner behavior monitoring solutions are difficult to provide comprehensive and accurate data support in this environment.

[0003] Therefore, how to comprehensively and accurately monitor the behavior of miners when working underground in a mine to ensure the safety of the miners is a technical problem to be solved by those skilled in the art. Summary of the invention

[0004] The purpose of the present invention is to solve the technical problem that the data is incomplete and the accuracy is low when monitoring the miners' behavior in the prior art.

[0005] To achieve the above technical objectives, on the one hand, the present invention provides a method for monitoring miner behavior, the method comprising: Collect multi-source data in a mining operation environment through preset sensors, and perform data cleaning on the multi-source data; Fusing the cleaned multi-source data to obtain fused data; Target detection is performed on the fused data to extract miner behavior, and the miner behavior is input into the interaction model to obtain the miner behavior state.

[0006] Furthermore, the multi-source data is fused through the following formula: , In the formula, for h The fused data corresponding to the time instant, n is the total number of sensors, for h Moment i The weights of the preset sensors, For the i The measured values ​​of the preset sensors.

[0007] Furthermore, the miner behavior is extracted through the following formula: , , Where Y is the set of fusion data corresponding to different moments, is the predicted value of the miner's behavior state, softmax is the normalized exponential function, and is the weight coefficient, is the activation function, and is the bias term, The predicted value of the miner's behavior state is obtained by dividing the multi-source data into groups according to the time, where S is the miner's behavior, , , is the fusion weight coefficient, f is the mapping from the fusion data at the corresponding moment to the predicted value of the miner's behavior state, N , M , C is the corresponding number of detected targets.

[0008] Furthermore, the interaction model is specifically: , In the formula, is the next hidden state, is the activation function, is the normalized attention weight, is the adjacency matrix after adding the self-loop, is the current hidden state, is the current weight matrix, The input data structure required to convert the miner behavior S into an interaction model.

[0009] Furthermore, the method further comprises: Performing a weighted assessment of the miner's behavior status; The final state after weighted evaluation is output.

[0010] Furthermore, weighted evaluation is performed using the following formula: , Where Result is the final state, m is the total number of underground personnel behavior types, is the weight coefficient corresponding to different behaviors, n is the total number of modes, is the modal fuzzy weight, is the modal fusion result.

[0011] On the other hand, the present invention also provides a miner behavior monitoring device, the device comprising: A collection module, used to collect multi-source data in a mining operation environment through preset sensors, and perform data cleaning on the multi-source data; A fusion module is used to fuse the cleaned multi-source data to obtain fused data; The determination module is used to perform target detection on the fused data to extract the miner's behavior, and input the miner's behavior into the interaction model to obtain the miner's behavior state.

[0012] A miner behavior monitoring method and device provided by the present invention, compared with the prior art, the method first collects multi-source data in a mining working environment through a preset sensor, and cleans the multi-source data; then fuses the cleaned multi-source data to obtain fused data; finally, target detection is performed on the fused data to extract the miner behavior, and the miner behavior is input into an interactive model to obtain the miner behavior status, which can comprehensively and accurately monitor the miner's behavior status, thereby more effectively ensuring the miner's behavior safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 The figure is a flow chart of a miner behavior monitoring method provided in an embodiment of this specification; Figure 2 Shown is a schematic diagram of the structure of a miner behavior monitoring device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0016] like Figure 1The flowchart of the miner behavior monitoring method provided by the embodiment of this specification is shown. Although this specification provides the method operation steps or device structures shown in the following embodiments or drawings, the method or device may include more or fewer operation steps or module units after partial merger based on routine or no creative labor. In the steps or structures where there is no necessary causal relationship logically, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure shown in the embodiments or drawings of this specification. When the method or module structure is applied in actual devices, servers or terminal products, it can be executed sequentially or in parallel according to the method or module structure shown in the embodiments or drawings (for example, in the environment of parallel processors or multi-threaded processing, or even in the implementation environment of distributed processing and server clusters).

[0017] The miner behavior monitoring method provided in the present embodiment can be applied to terminal devices such as clients and servers. Figure 1 As shown, the method specifically comprises the following steps: Step S101: collect multi-source data in a mining environment through preset sensors, and clean the multi-source data.

[0018] Specifically, the preset sensors include visual sensors, infrared sensors, acceleration sensors, etc. When collecting multi-source data, these preset sensors are used to collect multimodal data, that is, multi-source data, in the mining working environment, including images, sounds, and physiological parameters. After the collection is completed, these data will be cleaned. The data cleaning process includes removing noise, removing outliers, and removing erroneous data to ensure data quality and converting data from different sources into a unified format to facilitate subsequent fusion processing.

[0019] Step S102: Fusing the cleaned multi-source data to obtain fused data.

[0020] Specifically, multimodal data fusion involves integrating and fusing information from different sensors or data sources to obtain more comprehensive, accurate and rich data expressions.

[0021] In the embodiment of the present application, the multi-source data is fused specifically by the following formula: , In the formula, for h The fused data corresponding to the time instant, n is the total number of sensors, for h Moment i The weights of the preset sensors, For the iThe measured values ​​of the preset sensors, for h Moment n Data collected by sensors.

[0022] The distance dynamic model of the target distance is expressed as: , where is the system attenuation factor, is the noise disturbance under the current state k, is the distance measurement value of the current state, is the distance measurement value of the next state. The final output result of the distance dynamic model is recorded as .

[0023] For ranging sensors such as radar and infrared, the measurement model is usually expressed as: , where is the target true state at state k (a vector containing target information), is the observation matrix at the current moment, is the measurement noise. The final output of the sensor detection model is recorded as .

[0024] Specifically, multimodal intermediate fusion involves converting different modal data into high-dimensional feature expressions and then fusing them in the intermediate layer of the model. Assume there are two modal features: and . Then the fused features are: , is the fusion feature vector. In the formula, the target distance is used as the modal feature input , infrared ranging as modal feature input , and They respectively represent the final results of the measurement models mentioned above (target distance, infrared ranging, etc.).

[0025] Subsequent processing uses a fully connected layer for transformation: .in, is the weight matrix of the fully connected layer, is the bias term, is the output vector of the fully connected layer, is the ReLU activation function.

[0026] Step S103: Perform target detection on the fused data to extract miner behavior, and input the miner behavior into the interaction model to obtain the miner behavior state.

[0027] Specifically, the target detection formula is used to perform target detection on the fused data to extract the miner behavior. Specifically, the miner behavior is extracted through the following formula: , , Where Y is the set of fusion data corresponding to different moments, is the predicted value of the miner's behavior state, softmax is the normalized exponential function, and is the weight coefficient, is the activation function, and is the bias term, The predicted value of the miner's behavior state is obtained by dividing the multi-source data into groups according to the time, where S is the miner's behavior, , , is the fusion weight coefficient, f is the mapping from the fusion data at the corresponding moment to the predicted value of the miner's behavior state, N , M , C is the corresponding number of detected targets.

[0028] The above formula can quickly and accurately detect targets in the image, such as underground workers, through center point prediction and size regression. The algorithm can identify key information such as the position and posture of the miners.

[0029] The target detection formula is used to measure the difference between the model prediction and the true value, and optimize the model parameters by minimizing the loss function. In practical applications, the above formula has achieved good results in target detection tasks, especially when dealing with dense scenes and occluded scenes. This has a good optimization effect on the underground working environment of our system.

[0030] Among them, the first item is the target box regression loss, Use L1 loss; the second term center point prediction loss, Use binary cross entropy loss; the third category prediction loss, Use cross entropy loss.

[0031] In addition, the interaction model is a model established for the interaction relationship between miners. By analyzing the relative positions and movement trajectories between miners, the interaction relationship between miners, such as collaboration and conflict, can be inferred. This interaction relationship modeling helps to improve the accuracy and reliability of miner behavior identification.

[0032] The graph convolutional network updates the representation of a node by aggregating the information of neighboring nodes. The formula is as follows: , In the formula, is the next hidden state, is the activation function, D is the degree matrix, is the normalized adjacency matrix, is the current hidden state, is the current weight matrix.

[0033] Based on the above formula, the present application scheme uses the attention mechanism to assign different importance weights to different neighbor nodes. First, the attention score is determined by the following formula: , In the formula, is the attention score, To calculate the function, and are adjacent nodes, For relative position.

[0034] The attention score is then normalized by weight to obtain the importance weight. The weight normalization formula is as follows: , In the formula, is the importance weight, is the attention score, is the set of all neighbors of i, k are all neighbors of i, is the attention score of the neighboring node.

[0035] The attention weights are combined with the aggregation process of the graph convolutional network to take into account the importance differences of neighboring nodes when calculating new node representations. Specifically, the present invention multiplies the importance weights with the adjacency matrix to obtain a weighted adjacency matrix, which is then used in the aggregation process of the graph convolutional network, and finally the following interaction model is obtained: , In the formula, is the next hidden state, is the activation function, is the normalized attention weight, is the adjacency matrix after adding the self-loop, is the current hidden state, is the current weight matrix, The input data structure required to convert the miner behavior S into an interaction model.

[0036] In the embodiment of the present application, when the miner behavior status is finally output, the miner behavior status can also be weighted evaluated as needed to obtain a more accurate output result. More specifically, the weighted evaluation is performed through the following formula: , Where Result is the final state, m is the total number of underground personnel behavior types, is the weight coefficient corresponding to different behaviors, n is the total number of modes, is the modal fuzzy weight, is the modal fusion result.

[0037] Based on the above-mentioned miner behavior monitoring method, one or more embodiments of this specification also provide a platform and terminal for monitoring miner behavior. The platform or terminal may include devices, software, modules, plug-ins, servers, clients, etc. that use the methods described in the embodiments of this specification and are combined with necessary implementation hardware devices. Based on the same innovative concept, the system in one or more embodiments provided in the embodiments of this specification is as described in the following embodiments. Since the implementation scheme and method for solving the problem of the system are similar, the implementation of the specific system in the embodiments of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. The terms "unit" or "module" used below can be a combination of software and / or hardware that implements predetermined functions. Although the system described in the following embodiments is preferably implemented in software, hardware and a combination of software and hardware are also possible and conceived.

[0038] Specifically, Figure 2 Schematic diagram of the module structure of an embodiment of the miner behavior monitoring device provided in this specification. Figure 2 As shown, the miner behavior monitoring device provided in this specification includes: The acquisition module 201 is used to collect multi-source data in the mining environment through preset sensors and perform data cleaning on the multi-source data; A fusion module 202 is used to fuse the cleaned multi-source data to obtain fused data; The determination module 203 is used to perform target detection on the fused data to extract the miner's behavior, and input the miner's behavior into the interaction model to obtain the miner's behavior state.

[0039] It should be noted that the above-mentioned system may also include other implementation modes according to the description of the corresponding method embodiment. The specific implementation modes may refer to the description of the above-mentioned corresponding method embodiment, which will not be described one by one here.

[0040] The present application also provides an electronic device, including: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the method provided in the above embodiment.

[0041] The electronic device provided in the embodiment of the present application stores the executable instructions of the processor through the memory. When the processor executes the executable instructions, it can first collect multi-source data in the mining working environment through the preset sensors, and clean the multi-source data; then fuse the cleaned multi-source data to obtain fused data; finally, perform target detection on the fused data to extract the miner's behavior, and input the miner's behavior into the interactive model to obtain the miner's behavior status, which can comprehensively and accurately monitor the miner's behavior status, thereby more effectively ensuring the miner's behavior safety.

[0042] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0043] The methods or devices described in the above embodiments provided in this specification can implement business logic through computer programs and record them on storage media, and the storage media can be read and executed by computers to achieve the effects of the solutions described in the embodiments of this specification, such as: Collect multi-source data in a mining operation environment through preset sensors, and perform data cleaning on the multi-source data; Fusing the cleaned multi-source data to obtain fused data; Target detection is performed on the fused data to extract miner behavior, and the miner behavior is input into the interaction model to obtain the miner behavior state.

[0044] The storage medium may include a physical device for storing information, which is usually a medium that digitizes information and then stores it in an electrical, magnetic or optical manner. The storage medium may include: a device that stores information in an electrical energy manner, such as various memories, such as RAM, ROM, etc.; a device that stores information in a magnetic energy manner, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information in an optical manner, such as a CD or a DVD. Of course, there are other readable storage media, such as quantum memory, graphene memory, etc.

[0045] The embodiments of this specification are not limited to complying with industry communication standards, standard computer resource data update and data storage rules, or the situations described in one or more embodiments of this specification. Certain industry standards or slightly modified implementation plans based on the implementation described in the custom method or embodiment can also achieve the same, equivalent or similar, or predictable implementation effects after deformation of the above-mentioned embodiments. The embodiments obtained by using these modified or deformed data acquisition, storage, judgment, processing methods, etc. can still fall within the scope of the optional implementation plans of the embodiments of this specification.

[0046] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a computer-readable medium such as a microprocessor or processor and a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, ATMEL AT91SAM, MICROCHIP PIC18F26K20, and SILICONE LABS C8051F320. The memory controller can also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the devices for implementing various functions can be considered as both software modules for implementing the method and structures within the hardware component.

[0047] The device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. There may be other divisions in actual implementation, such as multiple units or plug-ins may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interface, indirect coupling or communication connection of the device or unit, which may be electrical, mechanical or other forms.

[0048] These computer program instructions may also be loaded onto a computer or other programmable resource data updating device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions executed on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0049] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. In the description of this specification, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0050] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A method for monitoring miner behavior, characterized in that: The method comprises: Collect multi-source data in a mining operation environment through preset sensors, and perform data cleaning on the multi-source data; Fusing the cleaned multi-source data to obtain fused data; Target detection is performed on the fused data to extract miner behavior, and the miner behavior is input into the interaction model to obtain the miner behavior state.

2. The miner behavior monitoring method according to claim 1, characterized in that: Specifically, the multi-source data is fused through the following formula: ， In the formula, for h The fused data corresponding to the time instant, n is the total number of sensors, for h Moment i The weights of the preset sensors, For the i The measured values ​​of the preset sensors.

3. The miner behavior monitoring method according to claim 1, characterized in that: Specifically, the miner behavior is extracted through the following formula: ， ， Where Y is the set of fusion data corresponding to different moments, is the predicted value of the miner's behavior state, softmax is the normalized exponential function, and is the weight coefficient, is the activation function, and is the bias term, The predicted value of the miner's behavior state is obtained by dividing the multi-source data into groups according to the time, where S is the miner's behavior, , , is the fusion weight coefficient, f is the mapping from the fusion data at the corresponding moment to the predicted value of the miner's behavior state, N , M , C is the corresponding number of detected targets.

4. The method for monitoring miner behavior according to claim 1, characterized in that: The interaction model is specifically: ， In the formula, is the next hidden state, is the activation function, is the normalized attention weight, is the adjacency matrix after adding the self-loop, is the current hidden state, is the current weight matrix, The input data structure required to convert the miner behavior S into an interaction model.

5. The method for monitoring miner behavior according to claim 1, characterized in that: The method further comprises: Performing a weighted assessment of the miner's behavior status; The final state after weighted evaluation is output.

6. The method for monitoring miner behavior as claimed in claim 5, characterized in that: The specific weighted evaluation is carried out through the following formula: ， In the formula, Result is the final state, m is the total number of underground personnel behavior types, is the weight coefficient corresponding to different behaviors, n is the total number of modes, is the modal fuzzy weight, is the modal fusion result.

7. A miner behavior monitoring device, characterized in that: The device comprises: A collection module, used to collect multi-source data in a mining operation environment through preset sensors, and perform data cleaning on the multi-source data; A fusion module is used to fuse the cleaned multi-source data to obtain fused data; The determination module is used to perform target detection on the fused data to extract the miner's behavior, and input the miner's behavior into the interaction model to obtain the miner's behavior state.

Images