Deep reinforcement learning-based scraper conveyor chain breakage identification method

Abstract

The invention relates to a scraper conveyor broken chain identification method based on deep reinforcement learning, and belongs to the technical field of scraper conveyor intelligent identification. According to the invention, a human brain cognitive mechanism is imitated, and an intelligent camera is used as a target detection carrier of a chain scission identification system. The method comprises the following steps: firstly, acquiring an offline sample of a broken chain fault state of the scraper conveyor from an external environment through a camera, inputting the offline sample into a deep belief network for learning and training, and storing knowledge obtained by training at the same time; secondly, the intelligent camera obtains the current state from the actual application environment and executes related decisions to act on the scraper conveying equipment, so that the external environment is affected, meanwhile, awards from the environment are obtained to serve as reinforcement learning feedback information, and autonomous learning is completed; and finally, when the camera recognizes the chain breakage state of the chain, a decision system of the camera outputs a shutdown decision. The method not only can learn off-line samples, but also can carry out on-line intensive training through an actual application environment, and can adapt to different working face environments.

Description

Technical Field

[0001] The present invention relates to a method for identifying chain breakage of a scraper conveyor by deep reinforcement learning, belonging to the technical field of intelligent sensing and identification of scraper conveyors. Background Art

[0002] With the continuous development of the intelligent technology of the transportation equipment in the fully mechanized coal mining face underground, the chain breakage detection technology of the scraper conveyor has also been popularized and applied. The scraper conveyor is the only transportation equipment in the fully mechanized coal mining face, and the chain is the key operating component for the scraper conveyor to transport coal, and its state directly affects the safe, efficient and stable operation of the equipment in the fully mechanized coal mining face. The realization of the chain breakage detection technology of the scraper conveyor can effectively avoid the accumulation of the chain in the chain breakage state, reduce the wear of the scraper and related components, improve the reliability of the equipment operation, enhance the production safety and efficiency of coal, and thus further improve the intelligent level of the scraper conveyor.

[0003] In the prior art, there is a chain breakage monitoring system and method for a mine scraper conveyor. The system is composed of a chain breakage monitoring device, a monitoring box and a combined switch. Among them, the chain breakage monitoring device includes a middle trough insert plate and a sensor assembly. When a moving scraper approaches the sensor assembly, the sensor outputs a low level, otherwise it outputs a high level. The monitoring box monitors the output signal of the sensor in real time, calculates the adjacent scraper spacing difference through the high and low level periods, and when the spacing difference is greater than the protection threshold, the monitoring box realizes the chain breakage stop protection of the scraper conveyor by controlling the combined switch. The main disadvantages of this method are that the scraper of the scraper conveyor is a continuously moving component, and the sensor installed in the middle trough of the scraper conveyor is easily affected by the movement of the scraper and causes wear and damage, thereby affecting the accuracy and reliability of the chain breakage monitoring system.

[0004] The present invention proposes a method for identifying chain breakage of a scraper conveyor by deep reinforcement learning. This method uses an intelligent camera as the target detection carrier of the chain breakage identification system. First, the intelligent camera obtains the offline samples of the chain breakage fault state of the scraper conveyor from the external environment and inputs them into the deep belief network for learning and training, and at the same time stores the knowledge obtained from the training; secondly, the intelligent camera obtains the current state from the application environment and executes relevant decisions to act on the scraper conveyor equipment, thereby affecting the external environment, and at the same time obtains the reward from the environment as the reinforcement learning feedback information to complete autonomous learning. This method can not only obtain accurate identification results in a short time, but also has the ability of online reinforcement training, can adapt to different scraper conveyor environments in the underground coal mine working face, and effectively improves the intelligent chain breakage identification technology level of the scraper conveyor. Summary of the Invention

[0005] The object of the present invention is to overcome the deficiencies existing in the prior art and design a chain break identification method for scraper conveyors based on deep reinforcement learning. The present invention mainly aims at the problems of poor detection accuracy, low stability, poor adaptability and reliability in complex scenarios of the existing chain break detection technology for scraper conveyors, and proposes a chain break identification method for scraper conveyors based on deep reinforcement learning.

[0006] The object of the present invention is achieved as follows: A chain break identification method for scraper conveyors based on deep reinforcement learning, which imitates the cognitive mechanism of the human brain and uses an intelligent camera as the target detection carrier of the chain break identification system. First, the intelligent camera obtains offline samples of the chain break fault state of the scraper conveyor from the external environment and inputs them into the deep belief network for learning and training, and at the same time stores the knowledge obtained from the training; secondly, the intelligent camera obtains the current state from the application environment and executes relevant decisions to act on the scraper conveyor equipment, thereby affecting the external environment, and at the same time obtains the reward from the environment as the reinforcement learning feedback information to complete autonomous learning; finally, when the intelligent camera identifies the chain break state, the network decision system of the intelligent camera will output a shutdown decision.

[0007] The training model of this method is based on the deep belief network, simulates the cognitive mechanism of the human brain, and combines the classical reinforcement learning Q-learning algorithm to design a six-tuple model: { S , A , f , Q , γ , κ}: (1) S represents the set of environmental states, s t ∈ S represents the external environmental state information at the t moment; (2) A represents the set of behavior decisions of the intelligent camera, a t ∈ A represents the behavior decision information of the camera at the t moment; (3) f : f ( s t+1 | s t , a t ) represents the state transition function of the reinforcement learning; (4) Q : Q ( st , a t )= E ( Q t+1 | s t , a t ) describes the state of the external environment of the system at t the moment s t Due to the execution of the intelligent camera decision a t after which the state of the external environment transfers to s t+1 the reward function at that time; (5) γ ∈(0, 1) represents the trade-off factor; (6) κ ∈(0, 1) represents the learning factor.

[0008] In the reinforcement learning stage of this method, the Markov decision process is used to iterate the reward value:: γ ; Based on the deep belief network, this method adds an online reinforcement learning training module, which can not only learn offline sample information but also has the ability of online reinforcement training, can adapt to different shearer conveyor environments in coal mine working faces, improves the reliability and generalization of the chain break monitoring system, and further enhances the intelligent chain break identification technology level of the shearer conveyor. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present invention will be further described in detail below in conjunction with the drawings and specific implementation manners: Figure 1 is the learning and training process of the present invention Figure 2 is the learning and training framework of the present invention In the figure, 1 is an intelligent camera, 2 is the external environment, 3 is an offline sample, 4 is a deep belief network, 5 is the state of the intelligent camera, 6 is the behavior decision of the intelligent camera, and 7 is the reward value. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following describes the structure and calculation process of the present invention in conjunction with specific implementation manners.

[0011] A shearer conveyor chain break identification method based on deep reinforcement learning according to the present invention, its learning and training framework is as Figure 1. The present invention imitates the cognitive mechanism of the human brain and uses an intelligent camera as the target detection carrier of the chain break recognition system. First, the intelligent camera obtains offline samples of the chain break fault state of the scraper conveyor from the external environment and inputs them into the deep belief network for learning and training, and at the same time stores the knowledge obtained from the training; secondly, the intelligent camera obtains the current state from the application environment and executes relevant decisions to act on the scraper conveyor equipment, thereby affecting the external environment, and at the same time obtains the reward from the environment as the reinforcement learning feedback information to complete autonomous learning; finally, when the intelligent camera recognizes the chain break state, the network decision-making system of the intelligent camera will output a shutdown decision.

[0012] This method trains the model based on the deep belief network, simulates the cognitive mechanism of the human brain, and combines the classical reinforcement learning Q-learning algorithm to design a six-tuple model: { S , A , f , Q , γ , κ}: (1) S represents the set of environmental states, s t ∈ S represents the external environmental state information at time t ; (2) A represents the set of behavioral decisions of the intelligent camera, a t ∈ A represents the behavioral decision information of the camera at time t ; (3) f : f ( s t+1 | s t , a t ) represents the state transition function of reinforcement learning; (4) Q : Q ( s t , a t ) = E ( Q t+1 | s t , a t ) describes the external environmental state t at time s tDue to the execution of the intelligent camera decision a t After that, the external environmental state transfers to s t+1 The reward function at this time; (5) γ ∈(0, 1) represents the compromise factor; (6) κ ∈(0, 1) represents the learning factor.

[0013] In the reinforcement learning stage of this method, the Markov decision process is used to iterate the reward value:: γ 。

Claims

1. A method for identifying chain breakage of scraper conveyors based on deep reinforcement learning, characterized in that: The intelligent camera (1) is used as the target detection carrier of the chain break identification system. First, the intelligent camera (1) obtains offline samples of the chain break fault state of the scraper conveyor from the external environment (2) (3) and inputs them into the deep belief network (4) for learning and training, and the knowledge obtained from the training is stored at the same time; Secondly, the smart camera (1) obtains the current state (5) from the application environment and executes relevant behavioral decisions (6) to act on the scraper conveyor equipment, thereby affecting the external environment. At the same time, it obtains the reward value (7) from the environment as reinforcement learning feedback information to complete autonomous learning. Finally, when the smart camera recognizes that the chain is broken, the network decision system of the smart camera (1) will output a shutdown decision.

2. The chain-breaking identification method for scraper conveyors based on deep reinforcement learning according to claim 1, wherein: The training model of the present invention is based on a deep belief network (4), simulates the cognitive mechanism of the human brain, and combines classical reinforcement learning Q- learning to design a six-tuple model: { S , A , f , Q , γ , κ}: (1) S represents the set of environmental states s t ∈ S represents the external environmental state information at t moment; (2) A Represents the set of behavior decisions of the intelligent camera, a t ∈ A Indicates the behavior decision information of the camera at t the moment; (3) f : f ( s t+1 | s t , a t ) represents the state transition function of reinforcement learning; (4) Q : Q ( s t , a t ) = E ( Q t+1 | s t , a t ) describes the reward information when the external environmental state of the system at the t moment transfers to the s t due to the execution of the intelligent camera decision a t and then the external environmental state transfers to the s t+1 ; (5) γ ∈(0, 1) represents the compromise factor; (6) κ ∈(0, 1) represents the learning factor.

3. The chain breakage recognition method for scraper conveyors based on deep reinforcement learning according to claim 1, characterized in that: In the reinforcement learning stage of the present invention, the Markov decision process is used to iterate the reward value: γ .

Images