A device safety approach control system based on vision guidance

Abstract

The present invention belongs to the technical field of coal mining, and specifically relates to a device safety approach control system based on visual guidance, which aims to solve the problem that traditional coal mining equipment requires manual operation and monitoring, has low production efficiency and high labor intensity. The present invention includes a roadheader interaction module, which is used to collect the position information of the roadheader and the coal quantity information in the shuttle car receiving hopper, send it to the shuttle car interaction module, and also control the movement of the roadheader; the shuttle car interaction module is used to obtain the position information of the roadheader, the electromagnetic change signal and the position information of the crusher, control the travel direction and travel position of the shuttle car; and collect the coal quantity information in the shuttle car receiving hopper and the coal quantity of the crusher, and control the movement of the shuttle car scraper chain; the crusher interaction module is used to obtain the coal quantity information on the crusher conveyor belt, and control the movement of the crusher receiving hopper and conveyor belt. The present invention can improve work efficiency and reduce labor intensity.

Description

Technical Field

[0001] The invention belongs to the technical field of coal mining, and in particular relates to a device safety approach control system based on vision guidance. Background Art

[0002] In recent years, with the rapid development of social economy and the advancement of science and technology, the coal mining industry has been constantly pursuing efficient, safe and sustainable development. In order to improve the production efficiency of coal mining equipment, reduce production costs and protect the lives of miners, the intelligent transformation of coal mining equipment has become an important issue in the current coal mining industry.

[0003] First of all, the background of the intelligent transformation of coal mine equipment lies in the pursuit of production efficiency. Coal mining is a complex and dangerous process. Traditional coal mine equipment requires manual operation and monitoring, and there are problems such as low production efficiency and high labor intensity. After the introduction of intelligent technology, coal mine equipment can realize functions such as automatic control, intelligent diagnosis and fault prevention, improve production efficiency, reduce labor costs, and obtain data and information in real time to help management make better decisions and scheduling.

[0004] Secondly, the background of the intelligent transformation of coal mine equipment is to strengthen the safety of miners. The working environment of coal mines is complex and harsh, and accidents and disasters are prone to occur. By introducing intelligent technology, remote monitoring and operation of coal mine equipment can be realized, reducing the time and frequency of personnel exposure to dangerous environments and reducing the risk of accidents. Smart sensors can monitor the working status and environmental parameters of coal mine equipment in real time, such as temperature, humidity and gas concentration. Once an abnormal situation is found, timely measures can be taken to ensure the safety of miners.

[0005] In addition, the background of the intelligent transformation of coal mining equipment is to improve resource utilization efficiency and environmental sustainable development. As an important energy resource, the mining and utilization of coal has a certain impact on the environment. Through intelligent transformation, it is possible to achieve refined management and optimized configuration of coal mining equipment, improve resource utilization efficiency, reduce energy consumption and waste emissions, promote the development of the coal mining industry in a green and low-carbon direction, and achieve sustainable utilization of resources.

[0006] Based on this, the present invention proposes a device safety approach control system based on vision guidance. Summary of the invention

[0007] In order to solve at least one of the above technical problems existing in the prior art, the present invention provides a device safety approach control system based on vision guidance.

[0008] The present invention is implemented by the following technical solutions: a vision-guided equipment safety approach control system, including a tunneling machine interaction module, a shuttle car interaction module, a crusher interaction module and an image acquisition module;

[0009] The tunneling machine interaction module is used to collect the position information of the tunneling machine and the coal quantity information in the receiving hopper of the shuttle car, and send it to the shuttle car interaction module. The tunneling machine interaction module is also used to control the movement of the cutting head of the tunneling machine and the loading mechanism of the tunneling machine;

[0010] The shuttle car interaction module is used to obtain the position information of the roadheader, the electromagnetic change signal emitted by the roadheader interaction module and the crusher interaction module, and the position information of the crusher, and control the travel direction and travel position of the shuttle car; the shuttle car interaction module is also used to obtain the coal quantity information in the shuttle car receiving hopper and the crusher coal quantity, and control the movement of the shuttle car scraper chain;

[0011] The crusher interaction module is used to obtain information about the amount of coal on the crusher conveyor belt and control the movement of the crusher conveyor belt;

[0012] The image acquisition module is used to obtain image data of the tunnel boring machine and the crusher, calculate the time it takes for the shuttle car to run to the tunnel boring machine and the crusher, and control the stopping and moving of the shuttle car in combination with the shuttle car interaction module.

[0013] Preferably, the tunnel boring machine interaction module includes a tunnel boring machine control module, a first tunnel boring machine information acquisition module, a first tunnel boring machine information transmission module and a second tunnel boring machine information transmission module;

[0014] The first roadheader information collection module is installed in the middle of the roadheader body, and is used to collect coal quantity information in the receiving hopper of the shuttle car and send it to the roadheader control module, and the roadheader control module is used to control the movement of the cutting head of the roadheader and the loading mechanism of the roadheader;

[0015] The first roadheader information transmission module is installed on the tail of the roadheader of the roadheader, and is used to collect the position information of the roadheader and send it to the shuttle car interaction module;

[0016] The second roadheader information transmission module is installed on the tail of the roadheader, and is used to generate a first electromagnetic change signal when the distance between the second roadheader information transmission module and the first shuttle car information collection module in the shuttle car interaction module is less than a preset distance threshold;

[0017] The tunnel boring machine control module is installed in the middle of the machine body. When the first electromagnetic change signal is received, the tunnel boring machine control module is used to control the cutting machine head to start coal mining, and at the same time, the tunnel boring machine loading mechanism starts to operate.

[0018] Preferably, the shuttle car interaction module includes a first shuttle car information collection module, a third shuttle car information collection module, a fourth shuttle car information collection module and a shuttle car travel control module;

[0019] The first shuttle car information acquisition module is installed on the head of the shuttle car, and is used to generate a first electromagnetic change signal with the second roadheader information transmission module, and send a stop signal to the shuttle car travel control module. The shuttle car travel control module is used to control the shuttle car to stop according to the stop signal and the image detection module. The shuttle car travel control module is installed in the middle of the body of the shuttle car;

[0020] The third shuttle car information collection module is installed on both sides of the shuttle car in the length direction of the body, and is used to collect the position information of the roadheader sent by the first roadheader information transmission module, and send it to the shuttle car travel control module, and the shuttle car travel control module controls the travel direction of the shuttle car based on the position information; the third shuttle car information collection module is also used to collect the coal quantity information on the shuttle car, when the coal quantity information on the shuttle car exceeds the first preset weight threshold, the shuttle car travel control module controls the shuttle car scraper chain to transport coal backwards;

[0021] When the coal quantity information on the shuttle car reaches the full load threshold, the third shuttle car information collection module sends a stop command to the tunneling machine control module, and sends a command to the shuttle car travel control module to move toward the crusher, and the tunneling machine control module controls the cutting machine head and the tunneling machine belt to stop working according to the stop command;

[0022] The fourth shuttle car information collection module is installed on the rear of the shuttle car. When the distance between the fourth shuttle car information collection module and the first crusher information transmission module in the crusher interaction module is less than a preset distance threshold, a second electromagnetic change signal is generated, and the second electromagnetic change signal is sent to the shuttle car travel control module. The shuttle car travel control module issues a coal unloading instruction according to the second electromagnetic change signal and the image detection module, and controls the shuttle car to stop moving.

[0023] Preferably, the crusher comprises a first crusher information transmission module, a second crusher information transmission module, a first crusher information collection module and a crusher control module;

[0024] The first crusher information transmission module is installed on the head of the crusher, and the first crusher information transmission module is used to generate the second electromagnetic change signal with the fourth shuttle car information collection module;

[0025] The second crusher information transmission module is installed on the body of the crusher, and is used to send the position information of the crusher to the third shuttle car information collection module. The third shuttle car information collection module sends the position information of the crusher to the shuttle car travel control module and controls the travel direction of the shuttle car.

[0026] The first crusher information collection module is installed on the body of the crusher. The first crusher information collection module is used to collect coal quantity information on the conveyor belt of the crusher. When the coal quantity information on the conveyor belt exceeds a second preset weight threshold, a coal transportation instruction is sent to the crusher control module. The crusher control module controls the movement of the conveyor belt of the crusher according to the coal transportation instruction. The crusher control module is installed on the body of the crusher.

[0027] Preferably, the third shuttle car information collection module is also used to monitor the coal accumulation information of the crusher, and when the coal accumulation information exceeds a preset accumulation threshold, a stop command is issued to the shuttle car travel control module, and the shuttle car travel control module controls the shuttle car scraper chain to stop transmitting coal to the crusher according to the stop command;

[0028] When the third shuttle car information collection module issues a stop command and again monitors that the coal accumulation information is less than the preset accumulation threshold, the shuttle car travel control module controls the shuttle car scraper chain to operate and continue to transport coal to the crusher.

[0029] Preferably, when the third shuttle car information collection module detects that the shuttle car is empty, the shuttle car travel control module issues a shuttle car scraper chain stop command, and at the same time issues a shuttle car travel command to the shuttle car travel control module to control the shuttle car to move away from the crusher and move toward the tunnel boring machine.

[0030] Preferably, after the first crusher information collection module detects that its own hopper and the conveyor belt are empty, it sends a stop operation instruction to the crusher control module, and the crusher is in a temporary dormant state, waiting for the shuttle car to approach next time to start collaborative work.

[0031] Preferably, the shuttle vehicle interaction module further includes a second shuttle vehicle information collection module;

[0032] The second shuttle car information collection module is installed on both sides of the shuttle car body in the width direction. The second shuttle car information collection module is used to obtain the interval distance between the shuttle car body and the coal walls on both sides of the tunnel, and send it to the shuttle car travel control module. The shuttle car travel control module controls the travel direction of the shuttle car based on the interval distance.

[0033] Preferably, the image detection module is installed on the front and rear of the shuttle car, and the image detection module is used to calculate the detection frame of each object in the acquired image and perform fine-tuning to obtain the final detection frame, and identify whether the object in the final detection frame is a tunnel boring machine and a crusher; if so, based on the set distance data and the speed data of the shuttle car at this moment, the time data of the shuttle car running to the tunnel boring machine and the crusher is calculated, and the shuttle car travel control module controls the shuttle car to stop moving in combination with the time data, the stop signal and the second electromagnetic change signal.

[0034] Preferably, the method for obtaining the final detection frame is:

[0035] Combined with the gradient function, the position loss of the detection box is calculated with the goal of minimizing the deviation between the predicted detection output by the deep convolutional neural network and the corresponding real detection box;

[0036] According to the calculated position loss of the detection frame, the position adjustment amount of the detection frame is calculated, and the position of the detection frame is adjusted to obtain the final detection frame;

[0037] The position loss of the detection frame is calculated as follows:

[0038]

[0039]

[0040] Among them, F (A,B) is the position loss of the detection box, A is the predicted detection box output by the deep convolutional neural network, area(A) is the area of ​​detection box A, B is the corresponding real detection box, area(B) is the area of ​​detection box B, ∩ represents the intersection, ∪ represents the union, d0 represents the Euclidean distance between the center points of detection box A and detection box B, d c is the distance of the diagonal of the detection box B, ω * is the penalty factor.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] (1) The present invention can achieve that the shuttle car can maintain a safe distance from both sides of the tunnel during its movement to avoid collision accidents; when the shuttle car is moving, it can collect information from the interactive device to achieve long-distance centering; when the shuttle car is close to the equipment, it can automatically lock the position through the proximity switch; when the shuttle car and the equipment are locked, the information collection device is used to control the equipment to start working; the third shuttle car information collection module can detect the amount of coal in the shuttle car, and can also receive signals from other information transmission modules to achieve interactive functions;

[0043] (2) The present invention can realize automatic operation of the equipment belt through the first shuttle car information collection module and the first crusher information collection module; when the coal quantity reaches a fixed value, each information collection and each control module are linked to enable the specific equipment to perform corresponding actions; when the shuttle car is moving, the tunnel boring machine and the crusher can be put on standby to reduce energy consumption; when the shuttle car approaches, the equipment can automatically start the operation mode.

[0044] (3) The interaction between the above modules has brought about significant improvements in the following aspects:

[0045] Improve efficiency: This system can quickly load and unload coal, greatly reducing the time and physical labor of manual operation, thereby significantly improving work efficiency. Reduce labor costs: The automatic loading and unloading function reduces the dependence on manpower. Coal mining enterprises can reduce the number of workers in the loading and unloading process and save labor costs. Improve safety: Automated operation can reduce safety accidents caused by human factors, such as misoperation, excessive fatigue, etc., and improve the safety of the workplace. Accurately control the load: This system usually accurately controls the amount of loaded coal, avoiding overloading or underloading, which is conducive to maintaining the stability of transportation and the service life of equipment. Enhance transportation continuity: The automatic loading and unloading capability of the shuttle car makes the coal transportation process more continuous and smooth, reducing pauses and waiting time, and improving the operating efficiency of the entire transportation system. Reduce material loss: Since the automatic loading and unloading process is more accurate and controllable, it can reduce the damage and scattering of coal during the loading and unloading process, reducing material loss. Energy saving and environmental protection: By improving transportation efficiency and reducing ineffective operations, the automatic loading and unloading shuttle car helps to reduce energy consumption and emissions, which is in line with the concept of green and environmentally friendly mining. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 is a schematic diagram of a tunnel boring machine interactive system based on a vision-guided equipment safety approach control system of the present invention;

[0048] Figure 2 is a schematic diagram of a shuttle vehicle interaction system of a device safety approach control system based on vision guidance of the present invention;

[0049] Figure 3 is a schematic diagram of a crusher interaction system of a vision-guided equipment safety approach control system of the present invention;

[0050] Figure 4 It is a schematic diagram of the overall structure of a device safety approach control system based on vision guidance of the present invention;

[0051] Figure 5 It is a schematic diagram of the coal loading process in a vision-guided equipment safety approach control system of the present invention;

[0052] Figure 6 It is a schematic diagram of the coal unloading process in a vision-guided equipment safety approach control system of the present invention.

[0053] In the figure: 1-cutter head; 2-borrowing machine tail; 3-borrowing machine control module; 4-first boring machine information acquisition module; 5-first boring machine information transmission module; 6-second boring machine information transmission module; 7-first image detection module; 8-second image detection module; 9-shuttle car travel control module; 10-first shuttle car information acquisition module; 11-second shuttle car information acquisition module; 12-third shuttle car information acquisition module; 13-fourth shuttle car information acquisition module; 14-crusher control module; 15-first crusher information acquisition module; 16-first crusher information transmission module; 17-second crusher information transmission module. DETAILED DESCRIPTION

[0054] In conjunction with the drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0055] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size should fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention. It should be noted that in this specification, relational terms such as first and second are only used to distinguish one entity from several other entities, and do not necessarily require or imply any actual relationship or order between these entities.

[0056] like Figures 1 to 6 As shown, the present invention provides a vision-guided equipment safety approach control system, including a tunneling machine interaction module, a shuttle car interaction module, a crusher interaction module and an image acquisition module;

[0057] The tunneling machine interaction module is used to collect the position information of the tunneling machine and the coal quantity information in the receiving hopper of the shuttle car, and send it to the shuttle car interaction module. The tunneling machine interaction module is also used to control the movement of the cutting head of the tunneling machine and the loading mechanism of the tunneling machine;

[0058] The shuttle car interaction module is used to obtain the position information of the roadheader, the electromagnetic change signal emitted by the roadheader interaction module and the crusher interaction module, and the position information of the crusher, and control the travel direction and travel position of the shuttle car; the shuttle car interaction module is also used to obtain the coal quantity information in the shuttle car receiving hopper and the crusher coal quantity, and control the movement of the shuttle car scraper chain;

[0059] The crusher interaction module is used to obtain information about the amount of coal on the crusher conveyor belt and control the movement of the crusher conveyor belt;

[0060] The image acquisition module is used to obtain image data of the tunnel boring machine and the crusher, calculate the time it takes for the shuttle car to run to the tunnel boring machine and the crusher, and control the stopping and moving of the shuttle car in combination with the shuttle car interaction module.

[0061] For further explanation of the present invention, see Figure 1 , the tunnel boring machine interaction module includes a tunnel boring machine control module, a first tunnel boring machine information acquisition module, a first tunnel boring machine information transmission module and a second tunnel boring machine information transmission module;

[0062] The first roadheader information acquisition module is installed in the middle part of the roadheader body, and is used to collect information on the amount of coal in the shuttle car's receiving hopper and send it to the roadheader control module. The roadheader control module is used to control the movement of the cutting head and the roadheader loading mechanism of the roadheader; specifically, when the amount of coal inside the shuttle car has not reached full load, and the amount of coal in the shuttle car's receiving hopper is too much, the roadheader control module obtains this information, and controls the cutting head and the roadheader loading mechanism to stop running, and no longer transports coal to the shuttle car's receiving hopper, so as to avoid overflow after excessive accumulation of coal in the shuttle car's receiving hopper.

[0063] The first roadheader information transmission module is installed on the tail of the roadheader of the roadheader, and is used to collect the position information of the roadheader and send it to the shuttle car interaction module;

[0064] The second roadheader information transmission module is installed on the tail of the roadheader, and is used to generate a first electromagnetic change signal when the distance between the second roadheader information transmission module and the first shuttle car information collection module in the shuttle car interaction module is less than a preset distance threshold;

[0065] The tunnel boring machine control module is installed in the middle of the machine body. When the first electromagnetic change signal is received, the tunnel boring machine control module is used to control the cutting machine head to start coal mining, and at the same time, the tunnel boring machine loading mechanism starts to operate.

[0066] For further explanation of the present invention, see Figure 2 , the shuttle car interaction module includes a first shuttle car information collection module, a third shuttle car information collection module, a fourth shuttle car information collection module and a shuttle car travel control module;

[0067] The first shuttle car information acquisition module is installed on the head of the shuttle car, and is used to generate a first electromagnetic change signal with the second roadheader information transmission module, and send a stop signal to the shuttle car travel control module. The shuttle car travel control module is used to control the shuttle car to stop according to the stop signal and the image detection module. The shuttle car travel control module is installed in the middle of the body of the shuttle car;

[0068] The third shuttle car information collection module is installed on both sides of the shuttle car in the length direction of the body, and is used to collect the position information of the roadheader sent by the first roadheader information transmission module, and send it to the shuttle car travel control module, and the shuttle car travel control module controls the travel direction of the shuttle car based on the position information; the third shuttle car information collection module is also used to collect the coal quantity information on the shuttle car, when the coal quantity information on the shuttle car exceeds the first preset weight threshold, the shuttle car travel control module controls the shuttle car scraper chain to transport coal backwards;

[0069] When the coal quantity information on the shuttle car reaches the full load threshold, the third shuttle car information collection module sends a stop command to the tunneling machine control module, and sends a command to the shuttle car travel control module to move toward the crusher, and the tunneling machine control module controls the cutting machine head and the tunneling machine belt to stop working according to the stop command;

[0070] The fourth shuttle car information collection module is installed on the rear of the shuttle car. When the distance between the fourth shuttle car information collection module and the first crusher information transmission module in the crusher interaction module is less than a preset distance threshold, a second electromagnetic change signal is generated, and the second electromagnetic change signal is sent to the shuttle car travel control module. The shuttle car travel control module issues a coal unloading instruction according to the second electromagnetic change signal and the image detection module, and controls the shuttle car to stop moving.

[0071] For further explanation of the present invention, see Figure 3The shuttle car interaction module also includes a second shuttle car information collection module; the second shuttle car information collection module is installed on both sides of the shuttle car body in the width direction, and the second shuttle car information collection module is used to obtain the interval distance between the shuttle car body and the coal walls on both sides of the tunnel, and send it to the shuttle car travel control module. The shuttle car travel control module controls the travel direction of the shuttle car based on the interval distance.

[0072] As a further explanation of the present invention, the first information transmission module of the first tunnel boring machine is installed on a side of the tail of the tunnel boring machine close to the first tunnel boring machine information collection module.

[0073] The above content is the function and connection relationship of each module in the process of the shuttle car approaching the tunneling machine to load coal, see Figures 1 to 4 , the specific workflow is as follows:

[0074] When the shuttle car approaches the tunnel boring machine from a long distance, the third shuttle car information collection module located on the top of the shuttle car is responsible for collecting information sent by the first tunnel boring machine information transmission module at the tail of the tunnel boring machine, and transmits the information to the shuttle car travel control module. The shuttle car travel control module controls the travel direction of the shuttle car, and continuously adjusts the forward head of the shuttle car and the tail of the anchor digger to be in a centered state; at the same time, the second shuttle car information collection modules located on both sides of the shuttle car body collect the distance between the equipment itself and the coal walls on both sides of the tunnel, and transmit the distance information to the shuttle car travel control module. The shuttle car travel control module continuously adjusts the travel direction of the equipment to ensure that the shuttle car always maintains a safe distance from the coal wall during the travel process.

[0075] When the shuttle car receiving hopper and the tail of the anchor boring machine gradually overlap to a specific position, the first shuttle car information collection module located at the shuttle car receiving hopper and the second tunnel boring machine information transmission module at the tail of the tunnel boring machine are close to each other to generate electromagnetic signal changes, and send the signal to the shuttle car travel control module. After receiving the signal, the shuttle car travel control module controls the shuttle car to stop moving; at the same time, the second tunnel boring machine information collection module at the tail of the tunnel boring machine sends the signal to the tunnel boring machine control module. After receiving the signal, the tunnel boring machine control module controls the cutting head of the tunnel boring machine to start coal mining, and at the same time, the tunnel boring machine belt operation starts.

[0076] The roadheader control module controls the belt to load coal into the shuttle car receiving hopper. The third shuttle car information collection module located on the top of the shuttle car collects the amount of coal. When a specific threshold is reached, the shuttle car travel control module controls the shuttle car scraper chain to transport coal backwards. When the third shuttle car information collection module detects that the shuttle car is fully loaded with coal, it sends a stop conveying command to the roadheader. At the same time, the shuttle car moves away from the roadheader toward the crusher. After receiving the stop conveying command sent by the shuttle car, the roadheader control module issues a stop command, at which time the roadheader conveyor belt and cutting machine head stop working.

[0077] When the shuttle car moves toward the crusher to unload coal, the connection relationship between the modules is described as follows:

[0078] For further explanation of the present invention, see Figure 3 , Figure 4 , the crusher includes a first crusher information transmission module, a second crusher information transmission module, a first crusher information collection module and a crusher control module;

[0079] The first crusher information transmission module is installed on the head of the crusher, and the first crusher information transmission module is used to generate the second electromagnetic change signal with the fourth shuttle car information collection module;

[0080] The second crusher information transmission module is installed on the body of the crusher, and is used to send the position information of the crusher to the third shuttle car information collection module. The third shuttle car information collection module sends the position information of the crusher to the shuttle car travel control module and controls the travel direction of the shuttle car.

[0081] The first crusher information collection module is installed on the body of the crusher. The first crusher information collection module is used to collect coal quantity information on the conveyor belt of the crusher. When the coal quantity information on the conveyor belt exceeds a second preset weight threshold, a coal transportation instruction is sent to the crusher control module. The crusher control module controls the movement of the conveyor belt of the crusher according to the coal transportation instruction. The crusher control module is installed on the body of the crusher.

[0082] For further explanation of the present invention, see Figure 4 The third shuttle car information collection module is also used to monitor the coal accumulation information of the crusher. When the coal accumulation information exceeds a preset accumulation threshold, a stop command is issued to the shuttle car travel control module. The shuttle car travel control module controls the shuttle car scraper chain to stop transmitting coal to the crusher according to the stop command.

[0083] For further explanation of the present invention, see Figure 4 When the third shuttle car information collection module issues a stop command and again detects that the coal accumulation information is less than the preset accumulation threshold, the shuttle car travel control module controls the shuttle car scraper chain to operate and continue to transport coal to the crusher.

[0084] For further explanation of the present invention, see Figure 4 When the third shuttle car information collection module detects that the shuttle car is empty, the shuttle car travel control module issues a shuttle car scraper chain stop command, and at the same time issues a shuttle car travel command to the shuttle car travel control module to control the shuttle car to move away from the crusher and move toward the tunnel boring machine.

[0085] As a further explanation of the present invention, after the first crusher information collection module detects that its own hopper and the conveyor belt are empty, it sends a stop operation instruction to the crusher control module, and the crusher is in a temporary dormant state, waiting for the shuttle car to approach next time to start collaborative work.

[0086] When unloading coal, see Figure 6 , the specific workflow is:

[0087] When the shuttle car moves toward the crusher, the third shuttle car information collection module located on the top of the shuttle car collects the position information sent by the second crusher information transmission module of the crusher head, and transmits the position information to the shuttle car travel control module. The shuttle car travel control module controls the travel direction of the shuttle car, and continuously adjusts the shuttle car and the crusher to be in a centered state; at the same time, the second shuttle car information collection modules located on both sides of the shuttle car body collect the distance between the shuttle car itself and the coal walls on both sides of the tunnel, and transmit the distance information to the shuttle car travel control module; the shuttle car travel control module continuously adjusts the travel direction of the shuttle car to ensure that the shuttle car always maintains a safe distance from the coal wall when moving toward the crusher.

[0088] When the shuttle car receiving hopper and the crusher head gradually overlap to a specific position, the fourth shuttle car information collection module located in the shuttle car rear hopper and the first crusher information transmission module of the crusher head are close to each other to generate electromagnetic signal changes, and send the signal to the shuttle car travel control module. After receiving the signal, the shuttle car travel control module controls the shuttle car to stop moving; at the same time, the shuttle car travel control module issues a coal unloading command, and the shuttle car scraper chain transports coal to the crusher. The first crusher information collection module is responsible for monitoring the amount of coal on the crusher conveyor belt. When the coal amount reaches a certain amount, it issues a coal transportation command. After receiving the command, the crusher control module controls the crusher to work and transport coal to the conveyor belt to prevent excessive accumulation of coal and cause crusher failure. When the third shuttle car information collection module located in the shuttle car detects that there is too much coal accumulation in the crusher, the shuttle car travel control module issues a stop command, and the shuttle car scraper chain stops transporting coal to the crusher; when the third shuttle car information collection module of the shuttle car detects that there is no more coal accumulation in the crusher, the shuttle car travel control module controls the shuttle car scraper chain to operate and continue to transport coal to the crusher.

[0089] When the shuttle car's third shuttle car information collection module detects that the shuttle car is empty, the shuttle car travel control module issues a belt stop command, and the belt stops running. At the same time, the shuttle car travel control module issues an equipment travel command, and the shuttle car moves away from the crusher and moves toward the anchor miner; at the same time, the shuttle car sends a signal to the crusher that it is about to move away. After the crusher information collection module detects that its own hopper and belt are empty, the control module issues a stop operation command, and the crusher is temporarily dormant, waiting for the shuttle car to approach next time to start working together.

[0090] The coordinated operation of the tunnel boring machine, shuttle car and crusher enables vision-based safe equipment approach and automated loading, transportation and unloading of coal, which is of great significance in the intelligent production of the tunneling working face.

[0091] As a further explanation of the present invention, the image detection module is installed on the front and rear of the shuttle car. The image detection module is used to calculate the detection frame of each object in the acquired image and perform fine-tuning to obtain a final detection frame, and identify whether the object in the final detection frame is a tunnel boring machine and a crusher; if so, based on the set distance data and the speed data of the shuttle car at this moment, the time data of the shuttle car running to the tunnel boring machine and the crusher is calculated, and the shuttle car travel control module controls the shuttle car to stop moving in combination with the time data, the stop signal and the second electromagnetic change signal.

[0092] Among them, "the image detection module is installed on the front and rear of the shuttle car", specifically, the image detection module includes a first image detection module and a second image detection module, the first image detection module is installed at the front of the shuttle car, used to detect the tunneling machine, and the second image detection module is installed at the front of the shuttle car, used to detect the crusher

[0093] Wherein, the image detection module may be a camera.

[0094] Among them, "the time data of the shuttle car running to the tunnel boring machine and the crusher is calculated based on the set distance data and the speed data of the shuttle car at this moment", specifically:

[0095] The distance that the camera can capture is fixed. The distance between the tunnel boring machine and the crusher that the camera can capture is acquired in advance as preset distance data, and the time data is obtained according to the speed-time calculation formula.

[0096] In the present invention, "the shuttle car travel control module controls the shuttle car to stop traveling in combination with the time data, the stop signal and the second electromagnetic change signal", specifically: when the shuttle car travels toward the tunnel boring machine, the first image detection module starts timing when it obtains the image of the tunnel boring machine, and when the timing time coincides with the time data, it is determined whether the stop signal is obtained;

[0097] If yes, the shuttle is controlled to stop moving;

[0098] If not, the shuttle is also controlled to stop moving, and the real-time acquired image data is uploaded to the background receiving end, and the inspection personnel will determine whether it is necessary to stop moving.

[0099] Similarly, when the shuttle car moves toward the crusher, the second image detection module starts timing when acquiring the image of the crusher, and when the timing time coincides with the time data, it is determined whether the second electromagnetic change signal is acquired;

[0100] If yes, the shuttle is controlled to stop moving;

[0101] If not, the shuttle is also controlled to stop moving, and the real-time acquired image data is uploaded to the background receiving end, and the inspection personnel will determine whether it is necessary to stop moving.

[0102] Through the above content, the present invention can realize the two-way combination of visual detection and sensor detection, achieve the accuracy of target acquisition, and improve the safety of operation between devices.

[0103] Among them, in this embodiment, "the image detection module is used to calculate the detection frame of each object in the acquired image and perform fine-tuning", before that, the image detection module uses a plurality of set Gaussian filters to perform high-frequency extraction and photometric fusion on each image in the video image acquired by the image detection module to obtain a multi-scale enhanced image;

[0104] Performing weighted fusion on each scale enhanced image of the multi-scale enhanced map to obtain an enhanced image corresponding to each image in the video image; extracting high-level features and low-level features of each image in the video image from the enhanced image through a feature extraction network;

[0105] A Gaussian pyramid and a Laplacian pyramid of the high-level features are established, and fused with the corresponding low-level features in the feature channel dimension to obtain a fused feature map; a detection frame of each object is obtained on the fused feature map and fine-tuned.

[0106] The multi-scale enhancement map is:

[0107]

[0108] Among them, h k and k is the high-frequency component and corresponding enhancement map obtained by high-frequency extraction and photometric fusion at the kth scale, k represents the number of scales, W k is the Gaussian operator corresponding to the kth scale, a is a preset hyperparameter lower than the average value of image pixels,

[0109] The enhanced image corresponding to each image in the video image is:

[0110]

[0111] Where Y is the enhanced image corresponding to an image in the video image; y i It is the enhanced image obtained by high-frequency extraction and photometric fusion at the i-th scale.

[0112] As a further explanation of the present invention, the final detection frame is specifically:

[0113] Combined with the gradient function, the position loss of the detection box is calculated with the goal of minimizing the deviation between the predicted detection output by the deep convolutional neural network and the corresponding real detection box;

[0114] According to the calculated position loss of the detection frame, the position adjustment amount of the detection frame is calculated, and the position of the detection frame is adjusted to obtain the final detection frame;

[0115] The position loss of the detection frame is calculated as follows:

[0116]

[0117]

[0118] Among them, F (A,B) is the position loss of the detection box, A is the predicted detection box output by the deep convolutional neural network, area(A) is the area of ​​detection box A, B is the corresponding real detection box, area(B) is the area of ​​detection box B, ∩ represents the intersection, ∪ represents the union, d0 represents the Euclidean distance between the center points of detection box A and detection box B, d c is the distance of the diagonal of the detection box B, ω * is the penalty factor.

[0119] Among them, this embodiment also filters the predicted detection box output by the deep convolutional neural network to retain the best predicted detection box, and the specific method is as follows:

[0120] Arrange all prediction boxes of the same target in descending order according to confidence, use the DIoU-NMS algorithm to filter the target boxes, filter out redundant prediction boxes, and retain the best prediction boxes:

[0121]

[0122]

[0123]

[0124] b 1 and b 2 They represent the center points of the pre-selected candidate box 1 and the candidate box 2, ε represents the NMS threshold, M represents the predicted detection box with the highest score, and B p represents the p-th predicted detection box, S pRepresents the confidence of the p-th predicted detection box, R DIoU is the penalty factor of DIoU-NMS, ρ represents the Euclidean distance between the two detection boxes, c represents the diagonal distance between the minimum closure area of ​​the pre-selected candidate box 1 and the candidate box 2; IoU is the absolute value of the ratio of the intersection and union of the center points of the pre-selected candidate box 1 and the candidate box 2.

[0125] Among them, the penalty factor ω * Based on the parameter that measures the aspect ratio of the detection frame A and the detection frame B, the following is obtained:

[0126]

[0127]

[0128] Among them, ν is a parameter that measures the aspect ratio of the detection box A and the detection box B, and w gt and h gt Corresponding to the width and height of the real detection box B, w p and h p They correspond to the width and height of the predicted detection box A respectively.

[0129] It should be noted that the above embodiment provides a device safety approach control system based on visual guidance, and only uses the division of the above functional modules as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0130] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A device safety approach control system based on vision guidance, characterized in that: It includes a tunnel boring machine interaction module, a shuttle car interaction module, a crusher interaction module and an image acquisition module; The tunnel boring machine interaction module is used to collect the position information of the tunnel boring machine and the coal quantity information in the receiving hopper of the shuttle car, and send it to the shuttle car interaction module. The tunnel boring machine interaction module is also used to control the movement of the cutting head of the tunnel boring machine and the tunnel boring machine loading mechanism; the tunnel boring machine interaction module includes a tunnel boring machine control module, a first tunnel boring machine information collection module, a first tunnel boring machine information transmission module and a second tunnel boring machine information transmission module; the first tunnel boring machine information collection module is installed in the middle of the body of the tunnel boring machine, and is used to collect the coal quantity information in the receiving hopper of the shuttle car, and send it to the tunnel boring machine control module, and the tunnel boring machine control module is used to control the cutting head and tunnel boring machine of the tunnel boring machine The first roadheader information transmission module is installed on the tail of the roadheader, and is used to collect the position information of the roadheader and send it to the shuttle car interaction module; the second roadheader information transmission module is installed on the tail of the roadheader, and is used to generate a first electromagnetic change signal when the distance between the second roadheader information transmission module and the first shuttle car information collection module in the shuttle car interaction module is less than a preset distance threshold; the roadheader control module is installed in the middle of the fuselage of the roadheader, and the roadheader control module is used to control the cutter head to start coal mining when receiving the first electromagnetic change signal, and at the same time, the roadheader loading mechanism is started; The shuttle car interaction module is used to obtain the position information of the roadheader, the electromagnetic change signal emitted by the roadheader interaction module and the crusher interaction module, and the position information of the crusher, and control the travel direction and travel position of the shuttle car; the shuttle car interaction module is also used to obtain the coal quantity information in the shuttle car receiving hopper and the crusher coal quantity, and control the movement of the shuttle car scraper chain; The crusher interaction module is used to obtain information about the amount of coal on the crusher conveyor belt and control the movement of the crusher conveyor belt; The image acquisition module is used to obtain image data of the tunnel boring machine and the crusher, calculate the time it takes for the shuttle car to run to the tunnel boring machine and the crusher, and control the stopping and moving of the shuttle car in combination with the shuttle car interaction module.

2. According to claim 1, a vision-guided equipment safety access control system is characterized in that: The shuttle car interaction module includes a first shuttle car information collection module, a third shuttle car information collection module, a fourth shuttle car information collection module and a shuttle car travel control module; The first shuttle car information acquisition module is installed on the head of the shuttle car, and is used to generate a first electromagnetic change signal between the second roadheader information transmission module and send a stop signal to the shuttle car travel control module. The shuttle car travel control module is used to control the shuttle car to stop according to the stop signal and the image detection module. The shuttle car travel control module is installed in the middle of the body of the shuttle car; The third shuttle car information collection module is installed on both sides of the shuttle car in the length direction of the body, and is used to collect the position information of the roadheader sent by the first roadheader information transmission module, and send it to the shuttle car travel control module, and the shuttle car travel control module controls the travel direction of the shuttle car based on the position information; the third shuttle car information collection module is also used to collect the coal quantity information on the shuttle car, when the coal quantity information on the shuttle car exceeds the first preset weight threshold, the shuttle car travel control module controls the shuttle car scraper chain to transport coal backwards; When the coal quantity information on the shuttle car reaches the full load threshold, the third shuttle car information collection module sends a stop command to the tunneling machine control module, and sends a command to the shuttle car travel control module to move toward the crusher, and the tunneling machine control module controls the cutting machine head and the tunneling machine belt to stop working according to the stop command; The fourth shuttle car information collection module is installed on the rear of the shuttle car. When the distance between the fourth shuttle car information collection module and the first crusher information transmission module in the crusher interaction module is less than a preset distance threshold, a second electromagnetic change signal is generated, and the second electromagnetic change signal is sent to the shuttle car travel control module. The shuttle car travel control module issues a coal unloading instruction according to the second electromagnetic change signal and the image detection module, and controls the shuttle car to stop moving.

3. The device safety access control system based on vision guidance according to claim 2 is characterized in that: The crusher includes a first crusher information transmission module, a second crusher information transmission module, a first crusher information collection module and a crusher control module; The first crusher information transmission module is installed on the head of the crusher, and the first crusher information transmission module is used to generate the second electromagnetic change signal with the fourth shuttle car information collection module; The second crusher information transmission module is installed on the body of the crusher, and is used to send the position information of the crusher to the third shuttle car information collection module. The third shuttle car information collection module sends the position information of the crusher to the shuttle car travel control module and controls the travel direction of the shuttle car. The first crusher information collection module is installed on the body of the crusher. The first crusher information collection module is used to collect coal quantity information on the conveyor belt of the crusher. When the coal quantity information on the conveyor belt exceeds a second preset weight threshold, a coal transportation instruction is sent to the crusher control module. The crusher control module controls the movement of the conveyor belt of the crusher according to the coal transportation instruction. The crusher control module is installed on the body of the crusher.

4. The device safety access control system based on vision guidance according to claim 3 is characterized in that: The third shuttle car information collection module is also used to monitor the coal accumulation information of the crusher. When the coal accumulation information exceeds a preset accumulation threshold, a stop command is issued to the shuttle car travel control module. The shuttle car travel control module controls the shuttle car scraper chain to stop transmitting coal to the crusher according to the stop command. When the third shuttle car information collection module issues a stop command and again monitors that the coal accumulation information is less than the preset accumulation threshold, the shuttle car travel control module controls the shuttle car scraper chain to operate and continue to transport coal to the crusher.

5. The device safety access control system based on vision guidance according to claim 4 is characterized in that: When the third shuttle car information collection module detects that the shuttle car is unloaded, the shuttle car travel control module issues a shuttle car scraper chain stop command, and at the same time issues a shuttle car travel command to the shuttle car travel control module to control the shuttle car to move away from the crusher and toward the tunnel boring machine.

6. The device safety access control system based on vision guidance according to claim 5, characterized in that: After the first crusher information collection module detects that its own hopper and the conveyor belt are empty, it sends a stop operation command to the crusher control module, and the crusher is in a temporary dormant state, waiting for the shuttle car to approach next time to start collaborative work.

7. The device safety access control system based on vision guidance according to claim 6 is characterized in that: The shuttle vehicle interaction module also includes a second shuttle vehicle information collection module; The second shuttle car information collection module is installed on both sides of the shuttle car body in the width direction. The second shuttle car information collection module is used to obtain the interval distance between the shuttle car body and the coal walls on both sides of the tunnel, and send it to the shuttle car travel control module. The shuttle car travel control module controls the travel direction of the shuttle car based on the interval distance.

8. The device safety access control system based on vision guidance according to claim 7 is characterized in that: The image detection module is installed on the front and rear of the shuttle car. The image detection module is used to calculate the detection frame of each object in the acquired image and perform fine-tuning to obtain a final detection frame, and identify whether the object in the final detection frame is a tunnel boring machine and a crusher; if so, based on the set distance data and the speed data of the shuttle car at this moment, the time data of the shuttle car running to the tunnel boring machine and the crusher is calculated, and the shuttle car travel control module controls the shuttle car to stop moving in combination with the time data, the stop signal and the second electromagnetic change signal.

9. The device safety access control system based on vision guidance according to claim 8, characterized in that: The method for obtaining the final detection frame is: Combined with the gradient function, the position loss of the detection box is calculated with the goal of minimizing the deviation between the predicted detection output by the deep convolutional neural network and the corresponding real detection box; According to the calculated position loss of the detection frame, the position adjustment amount of the detection frame is calculated, and the position of the detection frame is adjusted to obtain the final detection frame; The position loss of the detection frame is calculated as follows: Among them, F (A,B) is the position loss of the detection box, A is the predicted detection box output by the deep convolutional neural network, area(A) is the area of ​​detection box A, B is the corresponding real detection box, area(B) is the area of ​​detection box B, ∩ represents the intersection, ∪ represents the union, d0 represents the Euclidean distance between the center points of detection box A and detection box B, d c is the distance of the diagonal of the detection box B, ω * is the penalty factor.

Images