Mining image acquisition system and method

Abstract

The invention relates to the technical field of computer vision, and particularly discloses a mining image acquisition system and method, and the system comprises a non-polarization beam splitting cubic mirror which is used for carrying out the beam splitting of incident light of a mining scene, and obtaining first light and second light; the event camera is used for receiving the first light and generating an asynchronous event stream based on the first light; the microcomputer is used for receiving the asynchronous event stream, determining the moving speed according to the asynchronous event stream, and determining the exposure time according to the moving speed; the three-channel color imaging camera is used for receiving the second light and collecting mining scene information carried in the second light according to the exposure time so as to generate a color image; and the microcomputer is also used for carrying out image optimization processing on the color image to obtain a target image of the mining scene. Based on the above scheme, the stability and definition of mining scene image acquisition can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, specifically to a mining image acquisition system and method. Background Technology

[0002] Mining image acquisition primarily relies on Red-Green-Blue (RGB) industrial cameras, employing fixed exposure modes. However, the complex underground environment of coal mines, coupled with the high-speed movement and vibrations of tunneling machines, easily leads to motion blur in images. Furthermore, unstable lighting conditions in mines further degrade image quality, impacting the accuracy of subsequent visual positioning algorithms. Event cameras, as a novel sensor, offer advantages such as low latency and high temporal resolution, enabling accurate capture of information related to high-speed moving objects; however, they cannot directly output complete image data. While some solutions combine event cameras with RGB industrial cameras, they fail to achieve effective synergy, struggling to simultaneously address motion blur and image integrity requirements, thus failing to provide stable and clear image data for mining visual positioning. Therefore, a high-quality mining image acquisition solution adaptable to the complex underground environment and high-speed movement scenarios is urgently needed. Summary of the Invention

[0003] To address the issues of poor stability and clarity in current mining scene image acquisition, this application provides the following technical solution: In a first aspect, embodiments of this application provide a mining image acquisition system, comprising: A non-polarizing beam-splitting cubic mirror is used to split incident light rays in mining scenarios to obtain a first ray and a second ray. An event camera is used to receive the first ray of light and generate an asynchronous event stream based on the first ray of light. A microcomputer is used to receive asynchronous event streams, determine the movement speed based on the asynchronous event streams, and determine the exposure time based on the movement speed; the movement speed characterizes the relative movement speed of the object being photographed in the mining scene relative to the mining image acquisition system; A three-channel color imaging camera is used to receive the second light beam and acquire the mining scene information carried in the second light beam according to the exposure time to generate a color image; The microcomputer is also used to perform image optimization processing on color images to obtain target images of mining scenes; The event camera and the three-channel color imaging camera are respectively connected to the two light output ports of the non-polarizing beam splitter cube mirror, and the microcomputer is connected to the event camera and the three-channel color imaging camera respectively.

[0004] In some embodiments of this application, the mining image acquisition system further includes: The support base is used to support the event camera, the unpolarized beam splitter cube mirror, and the three-channel color imaging camera, and to level the event camera, the unpolarized beam splitter cube mirror, and the three-channel color imaging camera so that the field of view of the event camera and the three-channel color imaging camera are the same.

[0005] In some embodiments of this application, the mining image acquisition system further includes: The power module is used to provide operating power to the components in the mining image acquisition system.

[0006] In some embodiments of this application, the mining image acquisition system further includes: Explosion-proof boxes are used to provide enclosed spaces to prevent gases in mining environments from entering the explosion-proof boxes. The non-polarizing beam splitter cube mirror, event camera, microcomputer, three-channel color imaging camera, and power module are housed in an explosion-proof enclosure.

[0007] In some embodiments of this application, the explosion-proof box is provided with a camera window, and the camera window is inlaid with explosion-proof glass; The entrance of the non-polarizing beam-splitting cubic mirror faces the camera window so that it can receive incident light through the camera window.

[0008] In some embodiments of this application, the microcomputer is also used to determine the exposure time based on the moving speed and a preset mapping relationship; The preset mapping relationship includes the mapping relationship between different movement speeds and exposure times.

[0009] In some embodiments of this application, the microcomputer is further configured to generate an edge image based on an asynchronous event stream, determine target feature points in the edge image, convert the pixel-level optical flow vector of the target feature points into a moving velocity vector, and determine the maximum value in the moving velocity vector as the moving velocity. Among them, target feature points represent feature points that reflect the edge or contour of an object.

[0010] In some embodiments of this application, the microcomputer is also used to perform denoising processing on the color image to obtain a denoised image; and to adjust the image parameters of the denoised image to a preset range to obtain a target image.

[0011] Secondly, embodiments of this application provide a mining image acquisition method, applied to a mining image acquisition system. The mining image acquisition system includes a non-polarizing beam-splitting cubic mirror, an event camera, a microcomputer, and a three-channel color camera. The event camera and the three-channel color imaging camera are respectively connected to two light-emitting ports of the non-polarizing beam-splitting cubic mirror, and the microcomputer is connected to both the event camera and the three-channel color imaging camera. The method includes: The incident light beam in the mining scene is split by a non-polarizing beam-splitting cubic mirror to obtain the first ray and the second ray; The event camera receives the first light beam, generates an asynchronous event stream based on the first light beam, and transmits it to the microcomputer. The movement speed is determined by a microcomputer and an asynchronous event stream, and the exposure time is determined based on the movement speed; the movement speed characterizes the relative movement speed of the object being photographed within the mining scene relative to the mining image acquisition system; The second light beam is received by a three-channel color imaging camera, and the mining scene information carried in the second light beam is collected according to the exposure time to generate a color image. The target image of the mining scene is obtained by performing image optimization processing on the color image using a microcomputer.

[0012] In some embodiments of this application, the movement speed is determined by a microcomputer and an asynchronous event stream, including: Edge images are generated using a microcomputer based on an asynchronous event stream. Identify target feature points in an edge image; where target feature points represent feature points that reflect the edge or contour of an object; The pixel-level optical flow vector of the target feature point is converted into a moving velocity vector, and the maximum value in the moving velocity vector is determined as the moving velocity.

[0013] Therefore, this application utilizes the low-latency edge information provided by the event camera to adjust the exposure time in real time, which can quickly respond to the motion status of relevant machines in the mining scene, eliminate motion blur under high-speed motion, thereby improving image quality and adapting well to the complex environmental conditions in underground coal mines. Furthermore, by using a microcomputer to optimize image quality, a mining image with both clarity and integrity under high-speed motion scenes is finally obtained, which greatly improves the stability and quality of mining image acquisition. Attached Figure Description

[0014] To more intuitively illustrate the prior art and this application, several exemplary figures are provided below. It should be understood that the specific shapes and structures shown in the figures should not generally be regarded as limiting conditions for implementing this application; for example, based on the technical concept disclosed in this application and the exemplary figures, those skilled in the art are able to easily make conventional adjustments or further optimizations to the addition / reduction / classification, specific shapes, positional relationships, connection methods, size ratios, etc. of certain units (components).

[0015] Figure 1 Schematic diagram of the composition structure of the mining image acquisition system provided in the embodiments of this application Figure 1 ; Figure 2Schematic diagram of the composition structure of the mining image acquisition system provided in the embodiments of this application Figure 2 ; Figure 3 Schematic diagram of the composition structure of the mining image acquisition system provided in the embodiments of this application Figure 3 ; Figure 4 A schematic diagram of the implementation process of the mining image acquisition method provided in the embodiments of this application. Figure 1 ; Figure 5 A schematic diagram of light beam splitting provided in an embodiment of this application; Figure 6 A schematic diagram of the implementation process of the mining image acquisition method provided in the embodiments of this application. Figure 2 .

[0016] Figure label: Mining image acquisition system 0, non-polarizing beam splitter cube mirror 1, event camera 2, microcomputer 3, three-channel color imaging camera 4, support base 5, power supply module 6; Explosion-proof box cover 71, explosion-proof box body 72, explosion-proof glass 73, base plate 8. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. Any combination of different embodiments is possible.

[0018] In the description of this application: unless otherwise stated, "a plurality of" means two or more. The terms "first," "second," "third," etc., in this application are intended to distinguish the objects referred to and do not have any special meaning in terms of technical connotation (e.g., they should not be construed as an emphasis on importance or order). Expressions such as "including," "comprising," and "having" also mean "not limited to" (certain units, components, materials, steps, etc.).

[0019] Traditional mining image acquisition systems typically use RGB industrial cameras for direct image capture. However, traditional RGB industrial cameras capture images with fixed exposure times, which leads to motion blur, especially in high-speed scenes, significantly degrading image quality and affecting the accuracy of subsequent visual positioning algorithms. Furthermore, the working environment in coal mines is complex, with extremely unstable lighting conditions. Therefore, acquiring clear images under high-speed and low-light conditions has always been a technical challenge for mining image acquisition systems.

[0020] Event cameras, as a novel type of sensor, possess extremely low latency and high temporal resolution, enabling them to accurately capture high-speed moving objects. However, event cameras themselves can only provide edge information and cannot directly provide complete image data. Therefore, how to combine event cameras with traditional RGB industrial cameras, leveraging the advantages of both to solve motion blur and improve image quality, is a current research hotspot. Currently, some solutions combining event cameras and RGB industrial cameras exist, but they fail to achieve effective synergy between the two, making it difficult to simultaneously address motion blur and image integrity requirements, and thus unable to provide stable and clear image data for mine vision positioning.

[0021] To address the aforementioned problems in mining image acquisition, this application provides a mining image acquisition system and method. Through the collaborative work of an event camera and an RGB industrial camera, the system can effectively solve the motion blur problem in images during vibration and high-speed movement of underground tunneling machines in coal mines. It automatically optimizes camera exposure time at different movement speeds to improve image quality, thereby providing accurate and stable image data for subsequent visual positioning.

[0022] This embodiment of the mining image acquisition method is based on a mining image acquisition system; therefore, the relevant content will not be elaborated upon in this embodiment.

[0023] This application provides a mining image acquisition system, such as... Figure 1 As shown, the mining image acquisition system 0 may include a non-polarizing beam splitter cube mirror 1, an event camera 2, a microcomputer 3, and a three-channel color imaging camera 4.

[0024] In the embodiments of this application, the non-polarizing beam-splitting cubic mirror 1 can be used to split the incident light beam in a mining scene to obtain a first light beam and a second light beam.

[0025] In the embodiments of this application, the event camera 2 can be used to receive a first light ray and generate an asynchronous event stream based on the first light ray.

[0026] In embodiments of this application, the asynchronous event stream may include the spatial coordinates of the object's edge, a timestamp, and the polarity of brightness changes; the asynchronous event stream can be used to record the running information of objects in a mining scene.

[0027] In the embodiments of this application, the microcomputer 3 can be used to receive an asynchronous event stream, determine the moving speed based on the asynchronous event stream, and determine the exposure time based on the moving speed; the moving speed characterizes the relative moving speed of the object being photographed in the mining scene relative to the mining image acquisition system.

[0028] In the embodiments of this application, the mining image acquisition system can be installed in a tunneling machine, which can move and operate within the mine tunnel.

[0029] Understandably, the objects being photographed can be those within the mine shaft, such as rock walls or equipment.

[0030] The three-channel color imaging camera 4 can be used to receive the second light and acquire the mining scene information carried in the second light according to the exposure time to generate a color image.

[0031] For example, a three-channel color imaging camera could be an RGB industrial camera from a certain brand.

[0032] In the embodiments of this application, the microcomputer 3 can also be used to perform image optimization processing on the color image to obtain the target image of the mining scene.

[0033] In the embodiments of this application, the event camera and the three-channel color imaging camera are respectively connected to the two light output ports of the non-polarizing beam splitter cube mirror, and the microcomputer is connected to the event camera and the three-channel color imaging camera respectively.

[0034] For example, a non-polarizing beam-splitting cubic mirror splits the incident light from the mine into two beams. The first beam is transmitted along the original optical path to an event camera, generating an asynchronous event stream, which is then transmitted to a microcomputer. The microcomputer extracts the relative motion velocity from the event stream using a sparse optical flow method and determines the exposure time to be 5ms accordingly. A three-channel color imaging camera receives the second beam, acquires a color image of the mining scene with an exposure time of 5ms, transmits it to the microcomputer, and then performs image optimization processing to obtain the target image.

[0035] In the embodiments of this application, the distribution of light from the same source is achieved by using a non-polarizing beam-splitting cubic mirror, ensuring that the fields of view of the two cameras are consistent, thus solving the problem of field of view deviation in the current related combination schemes; the combination of motion information captured by the event camera and complete image acquisition by the color camera, along with dynamic exposure adjustment, not only eliminates motion blur caused by high-speed motion, but also ensures image integrity, providing high-quality data for visual positioning.

[0036] In some embodiments of this application, such as Figure 2 As shown, the mining image acquisition system may also include a support base 5.

[0037] In the embodiments of this application, the support base 5 can be used to support the event camera, the non-polarizing beam splitter, and the three-channel color imaging camera, and to level the event camera, the non-polarizing beam splitter, and the three-channel color imaging camera so that the field of view of the event camera and the three-channel color imaging camera are the same.

[0038] In the embodiments of this application, leveling refers to adjusting the height by means of the adjustable feet of the support base, and calibrating with a level to make the optical axes of the two cameras parallel.

[0039] In the embodiments of this application, the event camera, the three-channel color imaging camera, and the non-polarizing beam splitter cube mirror are all fixed on the support base. By adjusting the height of the support base legs, it is ensured that the lens centers of the event camera and the three-channel color imaging camera are aligned with the light output center of the non-polarizing beam splitter cube mirror.

[0040] In the embodiments of this application, the support base can avoid field of view deviation caused by installation tilt, ensuring that the motion information captured by the event camera corresponds one-to-one with the image data collected by the color camera, providing a structural basis for the precise linkage of subsequent movement speed calculation and exposure adjustment.

[0041] In some embodiments of this application, such as Figure 3 As shown, the mining image acquisition system may also include a power supply module 6.

[0042] In the embodiments of this application, the power module 6 can be used to provide operating power to the components in the mining image acquisition system.

[0043] In the embodiments of this application, the working power supply is adapted to the power supply requirements of each component, has overvoltage and overcurrent protection functions, and meets the intrinsic safety standards for mining applications.

[0044] In the embodiments of this application, the power module can ensure stable power supply for the mining image acquisition system during long-term continuous operation underground, avoid image acquisition interruption or data loss due to abnormal power supply, and improve the reliability of the mining image acquisition system.

[0045] In some embodiments of this application, the mining image acquisition system may further include an explosion-proof enclosure; wherein, as Figure 3 As shown, the explosion-proof box may include an explosion-proof box cover 71 and an explosion-proof box body 72.

[0046] In embodiments of this application, an explosion-proof box can be used to provide a closed space so that gases in a mining scenario cannot enter the explosion-proof box.

[0047] In some embodiments of this application, the explosion-proof enclosure is mainly used to block methane gas in the mine tunnel to improve the safety of the mining image acquisition system; the explosion-proof enclosure cover can be fixedly connected to the explosion-proof enclosure body by bolts, for example, such as Figure 3 As shown, the explosion-proof box cover can be fixedly connected to the explosion-proof box body using 6 bolts.

[0048] The non-polarizing beam splitter cube mirror, event camera, microcomputer, three-channel color imaging camera, and power module are housed in an explosion-proof enclosure.

[0049] Understandably, the enclosed space is achieved through the sealing structure of the explosion-proof box, which can prevent flammable and explosive gases such as methane from entering.

[0050] In the embodiments of this application, the non-polarizing beam splitter cube mirror, event camera, microcomputer, three-channel color imaging camera and power module are fixed in an explosion-proof box. The box is sealed with bolts to form a closed space. After testing, the sealing performance of the box meets the explosion-proof requirements for mining, and the gas leakage rate is zero.

[0051] In the embodiments of this application, the explosion-proof box can be adapted to the special flammable and explosive environment of underground mines, isolate dangerous gases from electrical components, avoid safety accidents, and ensure the safe use of the mining image acquisition system underground.

[0052] In the embodiments of this application, such as Figure 3 As shown, the explosion-proof box may be equipped with a camera window, and the camera window is inlaid with explosion-proof glass 73; the light inlet of the non-polarizing beam splitter faces the camera window so as to receive incident light through the camera window.

[0053] In some embodiments of this application, the size of the camera window can be matched with the entrance port of the non-polarizing beam splitter.

[0054] In some embodiments of this application, the explosion-proof glass has both explosion-proof performance and high light transmittance, thus balancing safety protection and light transmission.

[0055] For example, a camera window with a diameter of 50mm is opened on the front of the explosion-proof box, and explosion-proof glass with a thickness of 10mm is inlaid. The light inlet of the non-polarizing beam splitter is aligned with the center of the window to ensure that the incident light can pass through the glass smoothly and enter the beam splitter.

[0056] In the embodiments of this application, by setting up a camera window and explosion-proof glass, the incident light in the mining scene can be transmitted normally to the beam splitter while maintaining the sealing of the explosion-proof box. This does not affect the image acquisition effect and continues the explosion-proof protection performance, achieving a balance between safety and practicality.

[0057] In some embodiments of this application, such as Figure 3 As shown, the non-polarizing beam splitter cube mirror, event camera, microcomputer, three-channel color imaging camera and power module can be fixed on the base plate 8, and then the base plate is fixed in the explosion-proof box.

[0058] In some embodiments of this application, the microcomputer 3 can also be used to determine the exposure time based on the moving speed and a preset mapping relationship.

[0059] In some embodiments of this application, the preset mapping relationship includes the mapping relationship between different moving speeds and exposure times.

[0060] In some embodiments of this application, the preset mapping relationship can be a negative correlation based on the common movement speed range of underground tunneling machines (e.g., 0~5m / s), that is, the moving speed is negatively correlated with the exposure time; the preset mapping relationship can be stored in the exposure control program of the microcomputer and can be modified as needed.

[0061] In the embodiments of this application, a precise correlation is established between moving speed and exposure time to avoid the subjectivity and lag of manual settings, thereby realizing the automated and intelligent adjustment of exposure parameters and ensuring that clear images can be acquired under different motion states.

[0062] In some embodiments of this application, the microcomputer 3 can also be used to generate an edge image based on an asynchronous event stream, determine target feature points in the edge image, convert the pixel-level optical flow vector of the target feature points into a moving velocity vector, and determine the maximum value in the moving velocity vector as the moving velocity; wherein, the target feature points represent feature points that reflect the edge or contour of an object.

[0063] In the embodiments of this application, the edge image can be generated by a microcomputer according to the cumulative asynchronous event stream of a preset time window, which can highlight the edge contour of the object.

[0064] In the embodiments of this application, the pixel-level optical flow vector represents the pixel displacement change of the target feature point within a continuous time window, and can be converted into the actual relative movement speed, i.e., the movement speed vector.

[0065] For example, the microcomputer accumulates the event stream in a 10ms time window to generate an edge image, filters out 10 target feature points on the edge of the rock wall, calculates the pixel-level optical flow vector of each point, and converts it into actual moving speed vectors of 1.2m / s, 1.5m / s, 2.0m / s, etc., with the maximum value being 2.0m / s, so 2.0m / s is taken as the final moving speed.

[0066] In the embodiments of this application, the maximum relative movement speed is used as the basis for exposure adjustment to ensure that it can cope with the most extreme motion scenes and completely eliminate motion blur; the selection of target feature points focuses on the object edge, improves the accuracy of speed calculation, and avoids environmental noise interference.

[0067] In some embodiments of this application, the microcomputer 3 can also be used to denoise a color image to obtain a denoised image; and to adjust the image parameters of the denoised image to a preset range to obtain a target image.

[0068] In the embodiments of this application, the denoising process can employ a median filtering algorithm to remove motion noise and environmental interference noise from the image.

[0069] In the embodiments of this application, image parameters may include brightness, contrast, etc., and the preset range can be set according to the visual positioning requirements in mining scenarios.

[0070] For example, a microcomputer can perform 3×3 window mid-range filtering on a color image to remove noise generated by motion; adjust the brightness of the denoised image to 50~200 (grayscale value) and the contrast to a preset range of 1.2~1.8 to obtain the target image.

[0071] In the embodiments of this application, noise reduction processing reduces the impact of environmental interference and motion noise on image quality, parameter adjustment makes the visual effect of the image uniform, avoids the recognition deviation of the visual positioning algorithm due to differences in brightness and contrast, and improves the applicability of image data.

[0072] This application provides a mining image acquisition system that uses a non-polarizing beam-splitting cubic mirror to split incident light rays from the same source in a mining scene, enabling an event camera and a three-channel color imaging camera to obtain a consistent field of view. This leverages the low-latency motion capture advantage of the event camera while utilizing the complete image output capability of the color camera, overcoming the functional limitations of single-camera or non-coordinated solutions. It also provides a foundation for subsequent linkage between speed calculation and exposure adjustment. A closed-loop mechanism is established for asynchronous event flow, motion speed calculation, and adaptive exposure time adjustment. The maximum relative motion speed of objects within the mine tunnel relative to the system is extracted using the sparse optical flow method, and the exposure time is dynamically optimized based on a preset mapping relationship. Compared to related technologies where mining cameras use a fixed exposure mode, which is unsuitable for high-speed movement or vibration scenarios of tunneling machines, this application completely overcomes the limitations of fixed exposure, achieving real-time matching of exposure parameters and motion state, effectively eliminating the effects of high-speed movement. Image blurring is addressed to improve image clarity in complex underground movement scenarios. Furthermore, this application integrates core components such as a beam splitter, two types of cameras, a microcomputer, and a power module into an explosion-proof enclosure. Combined with the leveling function of the support base and the light-transmitting design of the explosion-proof glass, it forms an integrated structure adapted to the underground mining environment, achieving a balance between functional practicality and environmental adaptability. This avoids the impact of hazardous gases in the mine on the equipment and ensures that the acquisition accuracy is unaffected by the installation environment. Moreover, after color image acquisition, noise reduction processing and image parameter standardization adjustments output target images that meet the requirements of visual positioning. Compared to current related technologies that only focus on the image acquisition stage and lack targeted post-processing optimization, where acquired images are easily affected by underground noise and lighting fluctuations, this application can provide high-quality, highly consistent image data for subsequent visual positioning algorithms, avoiding positioning deviations caused by image quality issues and improving the accuracy and stability of tunneling machine visual navigation.

[0073] Based on the above embodiments, in another embodiment of this application, a mining image acquisition method is provided, applied to a mining image acquisition system. The mining image acquisition system includes a non-polarizing beam-splitting cubic mirror, an event camera, a microcomputer, and a three-channel color camera; the event camera and the three-channel color imaging camera are respectively connected to two light outlets of the non-polarizing beam-splitting cubic mirror, and the microcomputer is connected to both the event camera and the three-channel color imaging camera; as shown... Figure 4 As shown, the mining image acquisition method of the mining image acquisition system may include the following steps: Step 101: The incident light beam from the mining scene is split into a first ray and a second ray by using a non-polarizing beam-splitting cubic mirror.

[0074] In the embodiments of this application, the mining image acquisition system can split the incident light rays of the mining scene into a first ray and a second ray by using a non-polarizing beam-splitting cubic mirror.

[0075] It should be noted that, in the embodiments of this application, the mining image acquisition system can be installed in the tunneling machine. When the tunneling machine is working in the mine tunnel, the mining image acquisition system can acquire images of the mine tunnel in real time.

[0076] Step 102: Receive the first light ray through the event camera, generate an asynchronous event stream based on the first light ray, and transmit it to the microcomputer.

[0077] In the embodiments of this application, the mining image acquisition system can split the incident light rays of the mining scene into a first light ray and a second light ray by using a non-polarizing beam splitter cube mirror. Then, the first light ray is received by an event camera, and an asynchronous event stream is generated based on the first light ray and transmitted to a microcomputer.

[0078] Step 103: Determine the moving speed using a microcomputer and asynchronous event stream, and determine the exposure time based on the moving speed; the moving speed characterizes the relative moving speed of the object being photographed in the mining scene relative to the mining image acquisition system.

[0079] In the embodiments of this application, the mining image acquisition system can receive a first light ray through an event camera, generate an asynchronous event stream based on the first light ray and transmit it to a microcomputer, then determine the moving speed through the microcomputer and the asynchronous event stream, and determine the exposure time based on the moving speed; the moving speed characterizes the relative moving speed of the object being photographed in the mining scene relative to the mining image acquisition system.

[0080] In some embodiments of this application, when the mining image acquisition system determines the moving speed through a microcomputer and an asynchronous event stream, it generates an edge image based on the asynchronous event stream using the microcomputer; determines target feature points in the edge image; wherein the target feature points represent feature points reflecting the edge or contour of an object; converts the pixel-level optical flow vector of the target feature points into a moving speed vector, and determines the maximum value in the moving speed vector as the moving speed.

[0081] In the embodiments of this application, the moving speed can provide core data support for the accurate determination of subsequent exposure time and is a key link in eliminating motion blur.

[0082] In some embodiments of this application, when the mining image acquisition system determines the exposure time based on the moving speed, it can determine the exposure time based on the moving speed and a preset mapping relationship; wherein, the preset mapping relationship includes the mapping relationship between different moving speeds and exposure times.

[0083] Step 104: Receive the second light beam through a three-channel color imaging camera, and collect the mining scene information carried in the second light beam according to the exposure time to generate a color image.

[0084] In the embodiments of this application, the mining image acquisition system can split the incident light rays of the mining scene by a non-polarizing beam splitter to obtain a first light ray and a second light ray. Then, a three-channel color imaging camera receives the second light ray and acquires the mining scene information carried in the second light ray according to the exposure time to generate a color image.

[0085] Step 105: Perform image optimization processing on the color image using a microcomputer to obtain the target image of the mining scene.

[0086] In the embodiments of this application, the mining image acquisition system can receive the second light through a three-channel color imaging camera and acquire the mining scene information carried in the second light according to the exposure time to generate a color image. Then, the color image is optimized by a microcomputer to obtain the target image of the mining scene.

[0087] For example, when the underground tunneling machine is operating, the non-polarized beam-splitting cubic mirror in the mining image acquisition system receives the incident light and splits it into a first light and a second light; the event camera receives the first light and generates an asynchronous event stream that is transmitted to the microcomputer; the microcomputer determines the maximum relative movement speed to be 3.5 m / s based on the asynchronous event stream and calls a preset mapping relationship to determine the exposure time to be 5 ms; the three-channel color imaging camera receives the second light, acquires a color image with an exposure time of 5 ms, and transmits it to the microcomputer; the microcomputer denoises the image and adjusts the parameters to a preset range, and outputs the target image for visual positioning.

[0088] In the embodiments of this application, the entire mining image acquisition process realizes a fully automated closed loop of "light beam splitting, motion information capture, speed calculation, exposure adjustment, image acquisition, and optimization processing", which requires no manual intervention, is suitable for continuous operation scenarios of tunneling machines, and ensures both image clarity and acquisition efficiency.

[0089] In the embodiments of this application, when the mining image acquisition system performs image optimization processing on the color image using a microcomputer to obtain the target image of the mining scene, it can also perform denoising processing on the color image using a microcomputer to obtain a denoised image; and adjust the image parameters of the denoised image to a preset range to complete the image optimization processing and obtain the target image.

[0090] Based on the above embodiments, in another embodiment of this application, a non-uniform exposure imaging system based on the combination of an event camera and an RGB industrial camera is provided, namely a mining image acquisition system, and a method for operating the system, which is suitable for image acquisition and visual positioning when installed on a tunneling machine in a coal mine.

[0091] Traditional mining image acquisition systems typically use RGB industrial cameras for direct image capture. However, these cameras capture images with fixed exposure times, leading to motion blur, especially in high-speed scenes, which significantly degrades image quality and affects the accuracy of subsequent visual positioning algorithms. Furthermore, the complex working environment in coal mines and extremely unstable lighting conditions make acquiring clear images under high-speed and low-light conditions a persistent technical challenge for mining image acquisition systems. Event cameras, as a novel type of sensor, offer extremely low latency and high temporal resolution, enabling precise capture of fast-moving objects. However, event cameras only provide edge information and not complete image data. Therefore, combining event cameras with traditional RGB industrial cameras, leveraging their respective advantages to solve motion blur and improve image quality, is a current research hotspot in this field.

[0092] In the embodiments of this application, by working together with an event camera and a traditional RGB industrial camera, the motion blur problem of images during the vibration and high-speed movement of underground tunneling machines in coal mines can be solved, and the camera exposure time can be automatically optimized at different movement speeds to improve image quality, thereby providing accurate and stable image data for subsequent visual positioning.

[0093] As mentioned above Figure 2As shown, the event camera and RGB industrial camera in the mining image acquisition system can be connected by a non-polarizing beam splitter cube mirror. The support base is responsible for leveling the cameras to ensure that the two cameras have the same field of view. Then the two cameras are connected to a microcomputer. The development board contains camera control programs and image processing algorithms to achieve non-uniform exposure imaging acquisition.

[0094] like Figure 5 As shown, the lenses of two cameras are connected perpendicularly to the two light outlets of the unpolarized beam-splitting cube mirror 1. Light enters the unpolarized beam-splitting cube mirror; 50% of the light continues along the original optical path into the lens of the event camera 2 for imaging, while the other 50% is reflected by a reflective grid in the cube mirror and enters the lens of the RGB industrial camera (three-channel color imaging camera 4) along a direction perpendicular to the original optical path for imaging. The two cameras have the same field of view.

[0095] For example, such as Figure 6 As shown, the workflow of the mining image acquisition system includes: Step 201, tunneling machine movement; Step 202, event camera generating event stream; Step 203, calculating the movement vector using the sparse optical flow method; Step 204, extracting the maximum movement velocity vector; Step 205, real-time control of the RGB camera's exposure time; by adjusting the RGB camera's exposure time in real time, motion blur can be eliminated; Step 206, image acquisition; Step 207, image denoising; by denoising and related processing the acquired images, image quality uniformity can be ensured, and the images are ultimately used for visual positioning. Through the above steps, real-time, clear image acquisition within the mine tunnel can be achieved, ensuring the accuracy and clarity of images in high-speed motion environments, thereby effectively improving the visual positioning accuracy of the tunneling machine operating within the mine tunnel.

[0096] Furthermore, as mentioned above Figure 3 As shown, for the actual installation of a mining image acquisition system on a coal mine underground tunneling machine, an event camera, an RGB industrial camera, a power module, a non-polarizing beam splitter cube mirror, and a microcomputer can together constitute a non-uniform exposure imaging system. The non-uniform exposure imaging system can be fixed to a base plate, and then the base plate is fixed to an explosion-proof box with bolts. The explosion-proof box has a camera window, which is inlaid with explosion-proof glass. The cover of the explosion-proof box is fixed with 6 bolts, so that the box forms a closed space to prevent gas from entering and achieve the explosion-proof effect.

[0097] In some embodiments of this application, the connection of each component in the mining image acquisition system is first ensured to be normal, including the connection of the event camera and the RGB industrial camera through a non-polarizing beam-splitting cubic mirror to ensure that both capture images within the same field of view. When the event camera is triggered, the moving velocity vector of the object in the image is calculated based on the sparse optical flow method, and the exposure time of the RGB industrial camera is adjusted in real time according to the magnitude of the maximum velocity vector. Then, the acquired image is denoised and parameter-processed using a microcomputer to optimize the image quality. The processed image can be used for subsequent visual positioning algorithms to achieve precise navigation of the tunneling machine.

[0098] In summary, the mining image acquisition system of this application eliminates motion blur under high-speed movement by dynamically adjusting the exposure time, ensuring the acquisition of high-quality images; the event camera provides low-latency edge information, adjusts the exposure time in real time, and quickly responds to the movement status of the tunneling machine, effectively reducing latency; the mining image acquisition system can automatically optimize the exposure time of the RGB camera in real time according to different movement speeds, adapting to the complex environment of underground coal mines; by optimizing image quality, it provides high-quality image data, which helps to improve the accuracy and stability of subsequent visual positioning algorithms.

[0099] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment.

[0100] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0101] This application is described with reference to schematic and / or block diagrams illustrating the implementation of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each step and / or block in the schematic and / or block diagrams, as well as combinations thereof, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more steps of the schematic and / or one or more blocks of the block diagrams.

[0102] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the implementation flow diagram. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0104] The above embodiments are merely preferred embodiments provided to fully illustrate this application, and the scope of protection of this application is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on this application are all within the scope of protection of this application.

Claims

1. A mining image acquisition system, characterized in that, include: A non-polarizing beam-splitting cubic mirror is used to split incident light rays in mining scenarios to obtain a first ray and a second ray. An event camera is used to receive the first light ray and generate an asynchronous event stream based on the first light ray; A microcomputer is configured to receive the asynchronous event stream, determine the movement speed based on the asynchronous event stream, and determine the exposure time based on the movement speed; the movement speed represents the relative movement speed of the object being photographed within the mining scene relative to the mining image acquisition system. A three-channel color imaging camera is used to receive the second light and acquire the mining scene information carried in the second light according to the exposure time to generate a color image; The microcomputer is also used to perform image optimization processing on the color image to obtain the target image of the mining scene; The event camera and the three-channel color imaging camera are respectively connected to the two light output ports of the non-polarizing beam splitter cube mirror, and the microcomputer is connected to the event camera and the three-channel color imaging camera respectively.

2. The mining image acquisition system according to claim 1, characterized in that, Also includes: A support base is used to support the event camera, the non-polarizing beam splitter, and the three-channel color imaging camera, and to level the event camera, the non-polarizing beam splitter, and the three-channel color imaging camera so that the field of view of the event camera and the three-channel color imaging camera are the same.

3. The mining image acquisition system according to claim 1, characterized in that, Also includes: A power module is used to provide operating power to the components in the mining image acquisition system.

4. The mining image acquisition system according to claim 3, characterized in that, Also includes: An explosion-proof box is used to provide a closed space to prevent gases in the mining scenario from entering the explosion-proof box. The non-polarizing beam splitter cube mirror, the event camera, the microcomputer, the three-channel color imaging camera, and the power module are housed inside the explosion-proof enclosure.

5. The mining image acquisition system according to claim 4, characterized in that, The explosion-proof box is equipped with a camera window, and the camera window is fitted with explosion-proof glass. The light inlet of the non-polarizing beam-splitting cubic mirror faces the camera window so as to receive incident light through the camera window.

6. The mining image acquisition system according to any one of claims 1 to 5, characterized in that, The microcomputer is also used to determine the exposure time based on the moving speed and a preset mapping relationship; The preset mapping relationship includes the mapping relationship between different moving speeds and exposure times.

7. The mining image acquisition system according to claim 6, characterized in that, The microcomputer is also configured to generate an edge image based on the asynchronous event stream, determine target feature points in the edge image, convert the pixel-level optical flow vector of the target feature points into a moving velocity vector, and determine the maximum value in the moving velocity vector as the moving velocity; The target feature points represent feature points that reflect the edge or contour of an object.

8. The mining image acquisition system according to claim 1, characterized in that, The microcomputer is also used to perform denoising processing on the color image to obtain a denoised image; and to adjust the image parameters of the denoised image to a preset range to obtain the target image.

9. A method for acquiring images in mining applications, characterized in that, An image acquisition system for mining applications is described, comprising a non-polarizing beam-splitting cubic mirror, an event camera, a microcomputer, and a three-channel color camera. The event camera and the three-channel color imaging camera are respectively connected to two light-emitting ports of the non-polarizing beam-splitting cubic mirror, and the microcomputer is connected to both the event camera and the three-channel color imaging camera. The method includes: The incident light beam in the mining scene is split by the non-polarizing beam-splitting cubic mirror to obtain the first light beam and the second light beam; The event camera receives the first light beam, generates an asynchronous event stream based on the first light beam, and transmits it to the microcomputer. The movement speed is determined by the microcomputer and the asynchronous event stream, and the exposure time is determined based on the movement speed; the movement speed represents the relative movement speed of the object being photographed within the mining scene relative to the mining image acquisition system. The three-channel color imaging camera receives the second light and collects the mining scene information carried in the second light according to the exposure time to generate a color image; The target image of the mining scene is obtained by performing image optimization processing on the color image using the microcomputer.

10. The mining image acquisition method according to claim 9, characterized in that, The process of determining the movement speed using the microcomputer and the asynchronous event stream includes: The microcomputer generates an edge image based on the asynchronous event stream; Target feature points are determined in the edge image; wherein, the target feature points represent feature points that reflect the edge or contour of an object; The pixel-level optical flow vector of the target feature point is converted into a moving velocity vector, and the maximum value in the moving velocity vector is determined as the moving velocity.

Images