Collision detection method for an excavation and anchoring apparatus under depth estimation

Abstract

The application discloses a collision detection method of a digging anchor device under depth estimation, left and right event cameras are respectively installed on the left and right sides of the body of the digging anchor device; the left and right event cameras are used to record the light intensity changes in the respective target environment in real time, and the collision between the underground digging anchor devices is determined; due to many interference factors, errors and misjudgments may exist in a single determination criterion, therefore, the depth criterion and the distance criterion are adopted for comprehensive determination, that is, the depth criterion and the distance criterion are selected according to the depth information and the distance of the object for collision determination, the depth and distance information in the real-time depth image are detected according to the defined depth criterion and distance criterion, and the iteration method is used to determine whether a collision event occurs, the iteration method traverses all event data, and the distance, depth information and collision criterion are used for judgment, so that the technical problem of low accuracy of the detection result of the existing collision detection method of the digging anchor device is solved.

Description

Technical Field

[0001] This invention belongs to the field of underground coal mining and relates to collision detection methods, specifically a collision detection method for tunneling and anchoring equipment under depth estimation. Background Technology

[0002] Due to the unique working environment in coal mines, characterized by high levels of dust, high temperatures, and high humidity, the working conditions are extremely harsh. Roadheader equipment is a crucial piece of equipment used in underground coal mine working faces to support the walls of mine roadways and facilitate roadway excavation. To ensure the safety and efficiency of mining operations, depth estimation and collision detection of roadheader equipment are necessary. Traditional depth estimation and collision detection methods typically use sensors such as ordinary cameras and lidar, which rely heavily on the subjective judgment and observation of workers, easily leading to inconsistencies and inaccuracies in the detection results. Furthermore, these sensors face challenges in the mining environment, such as unstable lighting and high dust levels, which can also result in low accuracy and unstable results.

[0003] With the development of event cameras, they have become a new type of sensor due to their advantages such as high temporal resolution, low power consumption, and resistance to motion blur. Unlike traditional cameras, event cameras output events only when pixel values ​​change, rather than periodically outputting image frames. This allows event cameras to respond quickly to dynamic changes in a scene and exhibit excellent performance under complex conditions such as low light and high-speed motion.

[0004] Therefore, there is an urgent need for a collision detection method for tunneling and anchoring equipment based on event cameras, which can fully utilize the characteristics of event cameras to achieve efficient and accurate depth estimation and collision detection of tunneling and anchoring equipment in mining environments. Event stream data acquired through event cameras can provide rich temporal information, balancing real-time performance and accuracy. At the same time, the low power consumption of event cameras can meet the requirements of long-term monitoring. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a collision detection method for tunneling and anchoring equipment under depth estimation, so as to solve the technical problem that the detection results of existing collision detection methods for tunneling and anchoring equipment are not accurate.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A collision detection method for excavating and anchoring equipment under depth estimation specifically includes the following steps:

[0008] Step 1: Install the left event camera and the right event camera on the left and right sides of the tunneling and anchoring equipment, respectively;

[0009] Step 2: Based on the running trajectory of the tunneling and anchoring equipment, use two event cameras (left and right) to record the changes in light intensity in their respective target environments in real time, and obtain the left event stream data and the right event data stream.

[0010] The target environment is an underground fully mechanized mining face;

[0011] Step 3: Preprocess the left event stream data and right event data stream obtained in Step 2 respectively;

[0012] Step four: Perform feature extraction on the preprocessed left event stream data and right event data stream respectively to obtain the timestamp t(i), the time interval Δt between events, and the pixel coordinates (x, y) corresponding to the left event stream data. i ,y i ), and the timestamp t(r) corresponding to the right event stream data, the time interval Δt between events, and the pixel coordinates (x) r ,y r The quicksort algorithm is used to sort the events in the left and right event streams respectively.

[0013] Where: i is the sequence number of the event in the left event stream data obtained by the left event camera; r is the sequence number of the event in the right event stream data obtained by the right event camera;

[0014] Step 5: Match the events in the left and right event streams one by one to obtain multiple event pairs. Compare them to find the best event pair, retain the timestamp and pixel coordinates of the best event pair, and use it as the stereo matching result.

[0015] Step six: Calculate the depth value of each pixel in the optimal event pair obtained in step five, to obtain the depth image corresponding to the optimal event pair, and the depth value Z of the depth image. i ;

[0016] Step 7: Based on the optimal event obtained in Step 6, the corresponding depth image depth value Z is calculated. i To determine whether there is a collision between the excavation and anchoring equipment.

[0017] This invention also includes the following technical features:

[0018] Step five specifically includes the following sub-steps:

[0019] 5.1: Determine the time matching window. Select an event 'a' within the time matching window of the left event stream data obtained in step two, and search for the corresponding event 'b' in the time matching window of the right event stream data.

[0020] The time matching window includes at least one event, and its size is less than Δt;

[0021] 5.2: The matching degree M(x) between event a and event b is calculated using formula (1). i ,y i ,t i );

[0022]

[0023] in:

[0024] (x i ,y i () represents the pixel coordinates of event a;

[0025] (t i () is the timestamp of event a;

[0026] (x r ,y r () represents the pixel coordinates of event b;

[0027] (t r () is the timestamp of event b;

[0028] Size the time-matching window;

[0029] 5.3 Repeat steps 5.1 to 5.2 i times to obtain the matching degree of corresponding events in i event pairs; compare to obtain the matching degree M(x). i ,y i ,t i The highest-ranking event pair is considered the best event pair.

[0030] 5.4, ​​Save the matching degree M(x) i ,y i ,t i The timestamp and pixel coordinates of the highest event pair are used as the stereo matching result.

[0031] Step six specifically includes the following sub-steps:

[0032] Step 6.1, calculate the disparity d(x,y) of each pixel in the best event pair using formula (2);

[0033] d(x,y)=(x i ,y i )-(x r ,y r (2)

[0034] in:

[0035] (x i ,y i ) and (x r ,y r) represent the pixel coordinates of the events from the two event cameras, respectively;

[0036] Step 6.2: Based on the disparity d(x,y) obtained in step 6.1, calculate the depth value Z(x,y) of each pixel in the best event pair using formula (3);

[0037]

[0038] in:

[0039] f is the focal length of the event camera;

[0040] B is the baseline distance of the event camera;

[0041] Step 6.3: Fill the corresponding pixel with the depth value Z(x,y) obtained in Step 6.2 to generate a depth image. The depth value of this depth image is Z. i .

[0042] Step seven specifically includes the following steps:

[0043] Step 7.1, calculate the depth difference d between the two tunneling and anchoring devices using formula (4). Z Determine d Z If the value is greater than 0, then a collision may occur, proceed to step 6.2; otherwise, it is assumed that no collision has occurred, and continue to check whether a collision will occur at the next timestamp.

[0044] d Z =|Z i -Z i-1 | (4)

[0045] in:

[0046] Z i This indicates that the two excavating and anchoring devices were at timestamp (t). i The depth value on );

[0047] Z i-1 This indicates that the two excavating and anchoring devices were at timestamp (t). i-1 The depth value on );

[0048] Step 7.2: Calculate the minimum distance d between the two excavation and anchoring devices using formula (5), and determine whether the minimum distance d is less than the threshold distance L. If it is, a collision occurs; otherwise, no collision occurs.

[0049]

[0050] in:

[0051] (x i y iThis indicates that the two tunneling and anchoring devices were at timestamp (t). i Pixel coordinates on )

[0052] (x i-1 y i-1 This indicates that the two tunneling and anchoring devices were at timestamp (t). i-1 ) pixel coordinates.

[0053] The event camera used is a DAVIS event camera.

[0054] In step three, the preprocessing includes noise reduction and Gaussian filtering.

[0055] Compared with the prior art, the beneficial technical effects of this invention are:

[0056] In this invention, for collision detection between underground tunneling and anchoring equipment, due to numerous interfering factors, a single judgment criterion may have errors and misjudgments. Therefore, a comprehensive judgment is adopted using depth and distance criteria. That is, the depth and distance criteria are selected based on the depth information and distance of the object to determine the collision. The depth and distance information in the real-time depth image are detected according to the defined depth and distance criteria, and an iterative method is used to determine whether a collision event has occurred. The iterative method traverses all event pairs of data and makes judgments based on distance, depth information, and collision criteria, thus solving the technical problem of low accuracy of existing collision detection methods for tunneling and anchoring equipment. Attached Figure Description

[0057] Figure 1 This is an overall flowchart of the present invention;

[0058] Figure 2 The flowcharts for steps five and six are as follows;

[0059] Figure 3 This is the flowchart for step seven;

[0060] Figure 4 Depth images of the safe operating status of the tunneling and anchoring equipment;

[0061] Figure 5 Image of the threshold critical depth for the excavation and anchoring equipment;

[0062] Figure 6 This is a depth image for the threshold alarm of the tunneling and anchoring equipment.

[0063] The specific content of the present invention will be further explained in detail below with reference to the embodiments. Detailed Implementation

[0064] It should be noted that, unless otherwise specified, all components in this invention are those known in the art.

[0065] The following are specific embodiments of the present invention. It should be noted that the present invention is not limited to the following specific embodiments. All equivalent modifications made based on the technical solutions of this application fall within the protection scope of the present invention.

[0066] This invention provides a collision detection method for tunneling and anchoring equipment under depth estimation, specifically including the following steps:

[0067] Step 1: Install the left event camera and the right event camera on the left and right sides of the tunneling and anchoring equipment, respectively;

[0068] Step 2: Based on the running trajectory of the tunneling and anchoring equipment, use two event cameras (left and right) to record the changes in light intensity in their respective target environments in real time, and obtain the left event stream data and the right event data stream.

[0069] The target environment is an underground fully mechanized mining face;

[0070] Step 3: Preprocess the left event stream data and right event data stream obtained in Step 2 respectively;

[0071] Step four: Perform feature extraction on the preprocessed left event stream data and right event data stream respectively to obtain the timestamp t(i), the time interval Δt between events, and the pixel coordinates (x, y) corresponding to the left event stream data. i ,y i ), and the timestamp t(r) corresponding to the right event stream data, the time interval Δt between events, and the pixel coordinates (x) r ,y r The quicksort algorithm is used to sort the events in the left and right event streams respectively.

[0072] Where: i is the sequence number of the event in the left event stream data obtained by the left event camera; r is the sequence number of the event in the right event stream data obtained by the right event camera;

[0073] Step 5: Match the events in the left and right event streams one by one to obtain multiple event pairs. Compare them to find the best event pair, retain the timestamp and pixel coordinates of the best event pair, and use it as the stereo matching result.

[0074] Step six: Calculate the depth value of each pixel in the optimal event pair obtained in step five, to obtain the depth image corresponding to the optimal event pair, and the depth value Z of the depth image. i ;

[0075] Step 7: Based on the optimal event obtained in Step 6, the corresponding depth image depth value Z is calculated. i To determine whether there is a collision between the excavation and anchoring equipment.

[0076] In the above solution, for collision determination between underground tunneling and anchoring equipment, due to numerous interfering factors, a single determination criterion may have errors and misjudgments. Therefore, a comprehensive determination using depth and distance criteria is adopted. That is, the depth and distance criteria are selected based on the depth information and distance of the object to determine the collision. The depth and distance information in the real-time depth image are detected according to the defined depth and distance criteria, and an iterative method is used to determine whether a collision event has occurred. The iterative method solves the technical problem of low accuracy of the detection results of existing tunneling and anchoring equipment collision detection methods by traversing all event pairs of data and making judgments based on distance, depth information and collision criteria.

[0077] Specifically, step five includes the following sub-steps:

[0078] 5.1: Determine the time matching window. Select an event 'a' within the time matching window of the left event stream data obtained in step two, and search for the corresponding event 'b' in the time matching window of the right event stream data.

[0079] The time matching window must include at least one event, and its size must be less than Δt;

[0080] 5.2: The matching degree M(x) between event a and event b is calculated using formula (1). i ,y i ,t i );

[0081]

[0082] in:

[0083] (x i ,y i () represents the pixel coordinates of event a;

[0084] (t i () is the timestamp of event a;

[0085] (x r ,y r () represents the pixel coordinates of event b;

[0086] (t r () is the timestamp of event b;

[0087] Size the time-matching window;

[0088] 5.3 Repeat steps 5.1 to 5.2 i times to obtain the matching degree of corresponding events in i event pairs; compare to obtain the matching degree M(x). i ,y i ,t i The highest-ranking event pair is considered the best event pair.

[0089] 5.4, ​​Save the matching degree M(x) i ,y i ,t i The timestamp and pixel coordinates of the highest event pair are used as the stereo matching result.

[0090] Specifically, step six includes the following sub-steps:

[0091] Step 6.1, calculate the disparity d(x,y) of each pixel in the best event pair using formula (2);

[0092] d(x,y)=(x i ,y i )-(x r ,y r (2)

[0093] in:

[0094] (x i ,y i ) and (x r ,y r ) represent the pixel coordinates of the events from the two event cameras, respectively;

[0095] Step 6.2: Based on the disparity d(x,y) obtained in step 6.1, calculate the depth value Z(x,y) of each pixel in the best event pair using formula (3);

[0096]

[0097] in:

[0098] f is the focal length of the event camera;

[0099] B is the baseline distance of the event camera;

[0100] Step 6.3: Fill the corresponding pixel with the depth value Z(x,y) obtained in Step 6.2 to generate a depth image. The depth value of this depth image is Z. i .

[0101] Specifically, step seven includes the following steps:

[0102] Step 7.1, calculate the depth difference d between the two tunneling and anchoring devices using formula (4). Z Determine d Z If the value is greater than 0, then a collision may occur, proceed to step 6.2; otherwise, it is assumed that no collision has occurred, and continue to check whether a collision will occur at the next timestamp.

[0103] d Z =|Z i -Zi-1 | (4)

[0104] in:

[0105] Z i This indicates that the two excavating and anchoring devices were at timestamp (t). i The depth value on );

[0106] Z i-1 This indicates that the two excavating and anchoring devices were at timestamp (t). i-1 The depth value on );

[0107] Step 7.2: Calculate the minimum distance d between the two excavation and anchoring devices using formula (5), and determine whether the minimum distance d is less than the threshold distance L. If it is, a collision occurs; otherwise, no collision occurs.

[0108]

[0109] in:

[0110] (x i y i This indicates that the two tunneling and anchoring devices were at timestamp (t). i Pixel coordinates on )

[0111] (x i-1 y i-1 This indicates that the two tunneling and anchoring devices were at timestamp (t). i-1 ) pixel coordinates.

[0112] Specifically, the event camera used is a DAVIS event camera.

[0113] Preferably, compared to the traditional DVS event camera, the DAVIS event camera combines a frame-based camera (not synchronized with the event camera). The pixels generated by exposure can provide rich texture information that is missing in the event data. In high-dust working environments, the latter can better acquire edge information of the excavation and anchoring equipment.

[0114] Specifically, in step three, preprocessing includes noise reduction and Gaussian filtering.

Claims

1. A collision detection method for tunneling and anchoring equipment under depth estimation, characterized in that, Specifically, the following steps are included: Step 1: Install the left event camera and the right event camera on the left and right sides of the tunneling and anchoring equipment, respectively; Step 2: Based on the running trajectory of the tunneling and anchoring equipment, use two event cameras (left and right) to record the changes in light intensity in their respective target environments in real time, and obtain left event stream data and right event stream data; The target environment is an underground fully mechanized mining face; Step 3: Preprocess the left event stream data and right event stream data obtained in Step 2 respectively; Step four: Perform feature extraction on the preprocessed left and right event stream data respectively to obtain the timestamps corresponding to the left event stream data. Time interval between events and pixel coordinates and the timestamps corresponding to the right event stream data. Time interval between events and pixel coordinates The quicksort algorithm is used to sort the events in the left and right event streams respectively. Where: i is the sequence number of the event in the left event stream data obtained by the left event camera; r is the sequence number of the event in the right event stream data obtained by the right event camera; Step 5: Match the events in the left and right event streams one by one to obtain multiple event pairs. Compare them to find the best event pair, retain the timestamp and pixel coordinates of the best event pair, and use it as the stereo matching result. Step 5.1: Determine the time matching window. Select an event 'a' within the time matching window of the left event stream data obtained in Step 2, and search for the corresponding event 'b' in the time matching window of the right event stream data. The time matching window includes at least one event, and its size is less than [a certain value]. ; Step 5.2: Calculate the matching degree between event a and event b using formula (1). ; （1） in: Here are the pixel coordinates of event a; The timestamp of event a; Here are the pixel coordinates of event b; The timestamp of event b; Size the time-matching window; Step 5.3: Repeat steps 5.1-5.2 multiple times to obtain the matching degree of corresponding events in multiple event pairs; compare them to obtain the matching degree. The highest-ranking event pair is considered the best event pair. Step 5.4, save the matching score The timestamp and pixel coordinates of the highest event pair are used as the stereo matching result; Step six: Calculate the depth value of each pixel in the optimal event pair obtained in step five, to obtain the depth image corresponding to the optimal event pair, and the depth value of the depth image. ; Step 6.1: Calculate the disparity of each pixel in the optimal event pair using formula (2). ; （2） in: and These represent the pixel coordinates of events from the two event cameras, respectively. Step 6.2, according to the disparity obtained in step 6.1 , the depth value of each pixel in the optimal event pair is calculated using formula (3) ; （3） in: It is the focal length of the event camera; It is the baseline distance of the event camera; Step 6.3: Calculate the depth value of each pixel obtained in Step 6.

2. The corresponding pixels are filled in to generate a depth image, the depth value of which is... ; Step 7: Based on the optimal event pair obtained in Step 6, calculate the corresponding depth image depth value. To determine whether there is a collision between the excavation and anchoring equipment; Step 7.1: Calculate the depth difference between the two tunneling and anchoring devices using formula (4). ,judge If the value is greater than 0, then a collision may occur, proceed to step 7.2; otherwise, it is assumed that no collision has occurred, and continue to check whether a collision will occur at the next timestamp. （4） in: Indicates the timestamps of the two excavating and anchoring devices. Depth value on; Indicates the timestamps of the two excavating and anchoring devices. Depth value on; Step 7.2, calculate the minimum distance between the two tunneling and anchoring devices using formula (5). Determine the minimum distance If the distance is less than the threshold distance L, a collision occurs; otherwise, no collision occurs. （5） in: Indicates the timestamps of the two excavating and anchoring devices. pixel coordinates on; Indicates the timestamps of the two excavating and anchoring devices. The pixel coordinates on the screen.

2. The collision detection method for tunneling and anchoring equipment under depth estimation as described in claim 1, characterized in that, The event camera used is a DAVIS event camera.

3. The collision detection method for tunneling and anchoring equipment under depth estimation as described in claim 1, characterized in that, In step three, the preprocessing includes noise reduction.

Images