Mine safety production monitoring method based on multi-view optical system

Abstract

The invention discloses a mine safety production monitoring method based on a multi-view optical system, and relates to the technical field of image processing, and the method comprises the steps: obtaining an original mine image through the multi-view optical system; obtaining internal parameters and external parameters of the multi-view optical system as parameter information of the multi-view optical system; acquiring a mine image by using the original mine image according to the parameter information of the multi-view optical system; and performing pipeline splicing processing according to the mine image to obtain a mine safety production monitoring result. The method not only solves the problems of camera distortion and monitoring dead angles in a traditional monitoring scheme, but also improves the quality and real-time performance of the monitored image through an intelligent image processing technology and NPU rectangular data processing, and has remarkable technical advantages and application prospects. And the safety and the working efficiency of mine production can be greatly improved by adopting a multi-view optical system.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a mine safety production monitoring method based on a multi-view optical system. Background Art

[0002] In mine production, due to the complex environment and the needs of production operations, it is necessary to understand the overall working status of the environment in production operations in real time to ensure safe production in the mine. Traditionally, two wide-angle cameras are used to monitor the environment at different angles, but this method has problems such as large camera distortion and inability to fully display the mine production environment, which brings inconvenience to the monitoring and maintenance of the staff.

[0003] Therefore, there is an urgent need for a mine safety production monitoring method based on a multi-eye optical system to solve the camera distortion and monitoring blind spot problems existing in traditional monitoring methods. Summary of the invention

[0004] The purpose of the present invention is to propose a mine safety production monitoring method based on a multi-eye optical system to solve the camera distortion and monitoring blind spot problems existing in traditional monitoring solutions, and to improve the safety and work efficiency of mine production.

[0005] To achieve the above object, the present invention provides a mine safety production monitoring method based on a multi-view optical system, comprising the following steps:

[0006] Use multi-eye optical system to obtain original mine images;

[0007] Acquire internal parameters and external parameters of the multi-view optical system as parameter information of the multi-view optical system;

[0008] Acquiring a mine image using the original mine image according to parameter information of the multi-vision optical system;

[0009] Pipeline splicing processing is performed based on the mine image to obtain mine production safety monitoring results.

[0010] Optionally, the multi-optical system includes several optical components, each of which is equipped with infrared and visible light sensors.

[0011] Optionally, obtaining the internal and external parameters of the multi-eye optical system includes:

[0012] Obtaining the focal length, principal point coordinates, pixel size ratio, radial distortion coefficient and tangential distortion coefficient of the multi-vision optical system as internal parameters of the multi-vision optical system;

[0013] A rotation matrix and a translation vector of the polynomial optical system are obtained as external parameters of the polynomial optical system.

[0014] Optionally, acquiring a mine image using the original mine image according to the parameter information of the multi-vision optical system includes:

[0015] Using the original mine image to obtain pixel coordinates of the initial mine image;

[0016] Acquiring the world coordinates of the initial mine image using the parameter information of the multi-vision optical system according to the pixel coordinates of the initial mine image;

[0017] The mine image is acquired according to the world coordinates of the initial mine image.

[0018] Optionally, using the original mine image to obtain pixel coordinates of the initial mine image includes:

[0019] Get denoising, auto exposure, auto white balance and color correction as ISP parameters;

[0020] Preprocessing the original mine image according to the ISP parameters to obtain the initial mine image;

[0021] The pixel coordinates of the initial mine image are acquired according to the initial mine image.

[0022] Optionally, acquiring the world coordinates of the initial mine image using the parameter information of the multi-vision optical system according to the pixel coordinates of the initial mine image includes:

[0023] Acquiring image coordinates of the initial mine image according to pixel coordinates of the initial mine image;

[0024] Based on the image coordinates of the initial image of the mine and the internal parameters of the multi-vision optical system, the NPU acceleration processing algorithm is used to perform distortion correction to obtain the target distortion-free image coordinates of the initial image of the mine;

[0025] Obtaining the camera coordinates of the initial image of the mine using a perspective projection matrix according to the target undistorted image coordinates of the initial image of the mine;

[0026] The world coordinates of the initial mine image are obtained according to the camera coordinates of the initial mine image by using a rigid body transformation matrix and external parameters of the polyhedral optical system.

[0027] Optionally, the undistorted image coordinates of the initial mine image are:

[0028]

[0029] Among them, (x distorted ,y distorted ) is the undistorted image coordinate of the initial image of the mine, (x, y) is the image coordinate of the initial image of the mine, is the radial distance, k1, k2, k3 are radial distortion coefficients, p 1 、p 2 is the tangential distortion coefficient.

[0030] Optionally, the NPU acceleration processing algorithm is:

[0031]

[0032] Among them, (x undistorted ,y undistorted ) is the undistorted image coordinates of the initial image of the mine after NPU acceleration processing, that is, the target undistorted image coordinates of the initial image of the mine, and f(·) represents the correction function processed by NPU acceleration.

[0033] Optionally, the world coordinates of the initial mine image are:

[0034]

[0035] in,

[0036] In the formula, (X w ,Y w ,Z w ) is the world coordinate of the initial image of the mine, (X c ,Y c ,Z c ) is the camera coordinate of the initial image of the mine, [R 3x4 ,T 3x1 ] is the rigid body transformation matrix, P 3x4 is the perspective projection matrix.

[0037] Optionally, performing pipeline splicing processing according to the mine image to obtain a mine safety production monitoring result includes:

[0038] Performing feature point detection and matching according to the mine image to obtain feature point pairs of the mine image;

[0039] Performing image registration according to the feature point pairs of the mine images to obtain an optimal transformation matrix between adjacent mine images;

[0040] According to the optimal transformation matrix between the adjacent mine images, the mine images are sequentially subjected to multi-band fusion and fade-in and fade-out to obtain a mine fusion image;

[0041] Using the mine fusion image, overall color smoothing and brightness adjustment are performed in sequence to obtain a mine fusion image to be spliced;

[0042] The mine fusion images to be spliced ​​are used for splicing processing to obtain a mine safety production panoramic image as the mine safety production monitoring result.

[0043] Compared with the closest prior art, the present invention has the following beneficial effects:

[0044] The multi-eye panoramic optical system of the present invention can realize all-round monitoring of the mine environment through the combination of 4 cameras, eliminate monitoring blind spots, and each camera supports infrared and visible light dual-mode monitoring, which can adapt to environmental monitoring needs under different lighting conditions. The present invention generates panoramic images in real time through image stitching algorithms, which is convenient for staff to monitor and maintain; combined with ISP 3A image processing technology, automatic exposure, automatic white balance and automatic focus are realized, and higher quality images are provided; the computing advantage of NPU for matrix data is utilized to significantly improve the efficiency and accuracy of image processing and stitching. The present invention not only solves the problems of camera distortion and monitoring blind spots in traditional monitoring solutions, but also improves the quality and real-time performance of monitoring images through intelligent image processing technology and NPU rectangular data processing. It has significant technical advantages and application prospects, and the multi-eye optical system adopted will greatly improve the safety and work efficiency of mine production. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 The present invention is a flowchart of a method for monitoring safe production in a mine based on a multi-vision optical system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in combination with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention and are not intended to limit the present invention.

[0049] Since the 4-eye panoramic camera needs to have a certain overlapping FOV area when doing panoramic stitching, the panoramic view (360+4*(15)) / 4=105 is used, which requires the horizontal FOV to be 105 degrees. The traditional multi-eye optical system uses a sensor that supports infrared high pixels. According to the configured lens, the correction algorithm is used to complete the image distortion processing, complete the overall signal processing and systematic processing. Most of the lenses matched in this way will have large distortion.

[0050] In addition, traditional mine production monitoring uses cameras to collect raw images, and then directly enters the image processing process to obtain mine production monitoring results. The normal image processing order is sensor capture (Sensor Capture) → Linux driver → obtain raw images → image processing → distortion processing pipeline → splicing pipeline processing. The raw images collected by traditional cameras usually have certain distortions, such as radial distortion and tangential distortion. Normal distortion processing is performed after image processing, which will consume some system resources. In this way, the effectiveness of image acquisition depends on the effectiveness of image processing distortion. For such a configuration, the current camera cannot exert its inherent performance as a front-end device. The present invention is optimized to solve the above technical problems. The optimized image processing sequence is: sensor→Linux driver→obtain original image→image processing→stitching pipeline processing. It combines the computing advantages of NPU to efficiently calculate rectangular data, optimizes the performance of image processing algorithms, and combines distortion with ISP expansion to complete image processing and correction functions, ensuring that the ISP parameters of all cameras are consistent, and eliminating the brightness and chromaticity differences between images through automatic exposure (AE) and automatic white balance (AWB) as much as possible, effectively avoiding the problem of dark corners and ensuring the unity and consistency of the stitched image.

[0051] like Figure 1 As shown, the embodiment of the present invention provides a mine safety production monitoring method based on a multi-view optical system, which uses multiple cameras to simultaneously acquire images under infrared and visible light conditions, and completes all-round monitoring of mine safety production through image correction and splicing algorithms, including the following steps:

[0052] S1, collect original mine images through multi-eye optical system;

[0053] The multi-eye optical system is used for all-round monitoring of mine safety production. The system includes four optical components, each of which uses a camera and is equipped with infrared and visible light sensors. It can simultaneously obtain original mine images in four directions. After correction and system algorithm processing, these images can achieve panoramic stitching of the mine environment, optimizing the monitoring and maintenance of personnel on the mine working face.

[0054] Specifically, the multi-view optical system uses four cameras to acquire images in four directions respectively. Each camera supports dual-mode acquisition of infrared and visible light to meet the monitoring needs under different lighting conditions. In this embodiment, an 800W sensor is selected according to actual conditions. In addition, any sensor with a power of more than 800W and capable of supporting wide dynamic output can be used.

[0055] S2, obtaining internal parameters and external parameters of each camera (optical component), that is, obtaining internal parameters and external parameters of the multi-view optical system as parameter information of the multi-view optical system;

[0056] S2-1. Obtain the internal parameters of the multi-view optical system to correct the perspective distortion and radial / tangential distortion in the image and ensure the image accuracy, especially in high-precision stitching applications where the internal parameters directly affect the geometry of the image;

[0057] The internal parameters of the camera are used to describe the internal characteristics of the camera and define how the camera maps points in the 3D world to the image plane. Specifically, they include the following parameters:

[0058] Focal length (f x ,f y ): focal length of the camera, usually in pixels, indicating the magnification of the image. In this embodiment, the focal length has been pre-calibrated to a standard value in the system;

[0059] Coordinates of the principal point (c x ,c y ): the center of the optical axis in the image, that is, the position of the center of the camera imaging system on the image plane. In this embodiment, the principal point coordinates are set at the center of the image by default;

[0060] Aspectratio: the ratio of horizontal and vertical pixels.

[0061] Radial distortion coefficients (k1, k2, k3): describe the radial distortion of the lens, especially the distortion of fisheye lenses;

[0062] Tangential distortion coefficient (p 1 ,p 2 ): Describes the tangential distortion of the lens, that is, the difference in the degree of distortion produced in different areas of the image.

[0063] S2-2, obtaining the internal parameters of the multi-camera system to determine the relative positions of the cameras in the three-dimensional space, and ensuring that the images in the multi-camera system can be aligned and stitched in the same space;

[0064] Among them, the external parameters are used to describe the position and direction of the camera in the world coordinate system and to define the posture in the three-dimensional space under the camera, including:

[0065] Rotation matrix (R): defines the rotation of the camera relative to the world coordinate system;

[0066] Translation vector (T): defines the displacement of the camera relative to the world coordinate system.

[0067] S3, acquiring a mine image using the original mine image according to parameter information of the multi-vision optical system;

[0068] S3-1. Preprocess the original mine image to obtain the pixel coordinates of the initial mine image. Specifically:

[0069] Based on the pixel-level operation of the original mine image itself, the original mine image is processed by denoising, white balancing, color correction, etc. In the image processing process, ISP parameter adjustment is integrated, including denoising NR, automatic exposure AE, automatic white balance AWB and color correction matrix CCM. The original mine image is preprocessed by using ISP parameters to obtain the initial mine image, and then the pixel coordinates of the initial mine image are obtained. The purpose is to optimize the visual quality of the image, eliminate the brightness and chromaticity differences between images, and avoid the problem of dark corners. The preprocessing specifically includes:

[0070] Noise Reduction: De-noise the original mine image to eliminate the noise caused by the sensor and the environment;

[0071] Auto Exposure: Automatically adjust the exposure time according to the ambient light to ensure that the original mine image has moderate brightness;

[0072] Automatic White Balance: Automatically adjusts the white balance of raw mine images to ensure color accuracy under different lighting conditions;

[0073] Color Correction Matrix: Perform color correction to make the colors in the original mine image closer to the actual scene.

[0074] S3-2, according to the pixel coordinates of the initial image of the mine, the parameter information of the multi-eye optical system is used to obtain the world coordinates of the initial image of the mine, that is, the initial image of the mine is subjected to distortion correction and coordinate transformation through NPU accelerated transformation to obtain the mine safety production image. Based on matrix operations such as distortion correction and coordinate transformation, efficient calculation is performed through NPU, which significantly improves the processing performance;

[0075] S3-2-1. Convert the pixel coordinates (u, v) of the initial image of the mine into the image coordinates (x, y) of the initial image of the mine.

[0076] S3-2-2. Based on the image coordinates (x, y) of the initial image of the mine and the internal parameters of the multi-eye optical system, the NPU acceleration processing algorithm is used to perform distortion correction to obtain the target distortion-free image coordinates of the initial image of the mine. Specifically:

[0077] Based on NPU accelerated transformation, the undistorted image coordinates (x distorted ,y distorted ):

[0078]

[0079] In the formula, is the radial distance, k1, k2, k3 are radial distortion coefficients, p 1 、p 2 is the tangential distortion coefficient;

[0080] NPU has highly parallel processing capabilities and can handle multiple matrix operation tasks at the same time. By utilizing the dedicated hardware acceleration function of NPU, the calculation time of distortion correction and coordinate transformation process can be significantly reduced. Especially for applications that require real-time processing, NPU acceleration can ensure the timeliness and responsiveness of image processing. Among them, the NPU acceleration processing algorithm is:

[0081]

[0082] In the formula, (x undistorted ,y undistorted ) is the undistorted image coordinates of the initial image of the mine after NPU acceleration processing, that is, the target undistorted image coordinates of the initial image of the mine, and f(·) represents the correction function processed by NPU acceleration.

[0083] S3-2-3, according to the target undistorted image coordinates (x undistorted ,y undistorted ) Using the perspective projection matrix P 3x4 Get the camera coordinates (X c ,Y c ,Z c );

[0084]

[0085] S3-2-4, according to the camera coordinates (X c ,Y c ,Z c ) Using the rigid body transformation matrix [R 3x4 ,T 3x1 ] and the external parameters of the multi-eye optical system to obtain the world coordinates of the initial image of the mine (X w ,Yw ,Z w ).

[0086]

[0087] S4. Pipeline splicing is performed based on the mine images to obtain the mine safety production monitoring results. Pipeline splicing is to splice images in four directions. The camera lens needs to be calibrated. Due to the particularity of the mine environment, all images need to be displayed in an equidistant cylinder when combined. The projection center is set and the images collected by the four modules are spliced. Specifically:

[0088] S4-1, feature point detection and matching: performing feature point detection and matching based on the mine image to obtain feature point pairs of the mine image;

[0089] S4-1-1. Feature point detection: Feature point detection is performed on mine images in four directions using feature detection algorithms (such as SIFT, SURF, ORB, etc.) to extract unique points. That is, feature point detection is performed on mine images to obtain feature points of mine images. These points are invariant in the image, that is, they can be consistently detected in different images.

[0090] S4-1-2. Feature point matching: Use feature matching algorithms (such as nearest neighbor search, FLANN, BFMatcher, etc.) to find corresponding feature point pairs between adjacent images, that is, match the feature points of mine images with the feature point pairs of mine safety production images. The purpose of matching is to find the overlapping parts between images for subsequent image registration and stitching.

[0091] S4-2, Image Registration: Image registration is performed based on the feature point pairs of the mine images to obtain the optimal transformation matrix between adjacent mine images, that is, the transformation matrix of adjacent images is calculated based on the feature point pairs of the mine images, and images from different cameras are mapped to the same coordinate system. Specifically:

[0092] According to the feature point pairs of the mine images, a geometric transformation (such as perspective transformation or affine transformation) is used to calculate the transformation matrix between adjacent images, that is, the transformation matrix between adjacent mine images, and the RANSAC (random sampling consistency) method is used to eliminate incorrect feature point matches, and an optimal transformation matrix (such as a homography matrix or an affine matrix) is calculated to align the images to the same spatial coordinates.

[0093] S4-3, image fusion: based on the optimal transformation matrix between adjacent mine images, multi-band fusion, fade-in and fade-out and other methods are used to perform seamless fusion to obtain a mine fusion image, that is, according to the optimal transformation matrix between adjacent mine images, multi-band fusion and fade-in and fade-out are performed on the mine images in sequence to obtain a mine fusion image;

[0094] S4-3-1. Multi-band fusion: Multi-band fusion is a common image fusion method. It uses the optimal transformation matrix between adjacent mine images to decompose the mine image into multiple frequency bands, and fuses each frequency band separately to obtain a mine multi-band fused image to achieve a smooth transition effect, especially at the image seams with large differences in brightness and color;

[0095] S4-3-2, Feathering: Based on the multi-band fusion image of the mine, the overlapping area is processed by weighted averaging to obtain the mine fusion image, so that the seam parts are gradually and smoothly merged together to avoid obvious edge lines.

[0096] S4-4, using the mine fusion image to perform overall color smoothing and brightness adjustment in sequence to obtain the mine fusion image to be spliced;

[0097] S4-4-1. Color smoothing: Due to different shooting conditions of different cameras, the colors between images may be inconsistent. Therefore, global color adjustment is required during the fusion process. Commonly used algorithms include brightness matching, color histogram equalization, etc., to ensure that the entire stitched image remains consistent in color and brightness.

[0098] S4-4-2. Brightness adjustment: Adjust the local brightness of different images to eliminate the problem of uneven brightness at seams caused by exposure differences.

[0099] S4-5, using the mine fusion images to be stitched to perform stitching processing to obtain a panoramic image of mine safety production as a mine safety production monitoring result;

[0100] Stitching is complete: Through the above steps, the images from multiple cameras are fused into a complete panorama. The generation of a panorama usually requires stitching the processed images to a uniform output resolution to ensure seamless connection of the image edges.

[0101] Output format: Panoramic images can be saved in common image formats (such as JPEG, PNG), or as frame data for video output for subsequent display, storage or transmission.

[0102] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0105] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A mine safety production monitoring method based on a multi-eye optical system, characterized in that: The specific steps include: Use multi-eye optical system to obtain original mine images; Acquire internal parameters and external parameters of the multi-view optical system as parameter information of the multi-view optical system; Acquiring a mine image using the original mine image according to parameter information of the multi-vision optical system; Pipeline splicing processing is performed based on the mine image to obtain mine production safety monitoring results.

2. The mine safety production monitoring method based on a multi-view optical system according to claim 1 is characterized in that: The multi-optical system includes a plurality of optical components, each of which is equipped with infrared and visible light sensors.

3. The mine safety production monitoring method based on a multi-view optical system according to claim 1 is characterized in that: Obtaining the internal and external parameters of the multi-eye optical system includes: Obtaining the focal length, principal point coordinates, pixel size ratio, radial distortion coefficient and tangential distortion coefficient of the multi-vision optical system as internal parameters of the multi-vision optical system; A rotation matrix and a translation vector of the polynomial optical system are obtained as external parameters of the polynomial optical system.

4. The mine safety production monitoring method based on a multi-view optical system according to claim 2 is characterized in that: Acquiring a mine image using the original mine image according to the parameter information of the multi-vision optical system includes: Using the original mine image to obtain pixel coordinates of the initial mine image; Acquiring the world coordinates of the initial image of the mine using the parameter information of the multi-vision optical system according to the pixel coordinates of the initial image of the mine; The mine image is acquired according to the world coordinates of the initial mine image.

5. The mine safety production monitoring method based on a multi-view optical system according to claim 4 is characterized in that: Using the original mine image to obtain pixel coordinates of the initial mine image includes: Get denoising, auto exposure, auto white balance and color correction as ISP parameters; Preprocessing the original mine image according to the ISP parameters to obtain the initial mine image; The pixel coordinates of the initial mine image are acquired according to the initial mine image.

6. The mine safety production monitoring method based on a multi-view optical system according to claim 4 is characterized in that: Acquiring the world coordinates of the initial image of the mine using the parameter information of the multi-vision optical system according to the pixel coordinates of the initial image of the mine includes: Acquiring image coordinates of the initial mine image according to pixel coordinates of the initial mine image; Based on the image coordinates of the initial image of the mine and the internal parameters of the multi-vision optical system, the NPU acceleration processing algorithm is used to perform distortion correction to obtain the target distortion-free image coordinates of the initial image of the mine; Acquiring the camera coordinates of the initial image of the mine using a perspective projection matrix according to the target undistorted image coordinates of the initial image of the mine; The world coordinates of the initial mine image are obtained according to the camera coordinates of the initial mine image by using a rigid body transformation matrix and external parameters of the polyhedral optical system.

7. The mine safety production monitoring method based on a multi-view optical system according to claim 6 is characterized in that: The undistorted image coordinates of the initial mine image are: Among them, (x distorted ,y distorted ) is the undistorted image coordinate of the initial image of the mine, (x, y) is the image coordinate of the initial image of the mine, is the radial distance, k1, k2, k3 are the radial distortion coefficients, and p1, p2 are the tangential distortion coefficients.

8. The mine production safety monitoring method based on a multi-view optical system according to claim 7 is characterized in that: The NPU acceleration processing algorithm is: Among them, (x undistorted ,y undistorted ) is the undistorted image coordinates of the initial image of the mine after NPU acceleration processing, that is, the target undistorted image coordinates of the initial image of the mine, and f(·) represents the correction function processed by NPU acceleration.

9. The mine safety production monitoring method based on a multi-view optical system according to claim 8, characterized in that: The world coordinates of the initial image of the mine are: in, In the formula, (X w ,Y w ,Z w ) is the world coordinate of the initial image of the mine, (X c ,Y c ,Z c ) is the camera coordinate of the initial image of the mine, [R 3x4 ,T 3x1 ] is the rigid body transformation matrix, P 3x4 is the perspective projection matrix.

10. The mine safety production monitoring method based on a multi-view optical system according to claim 1, characterized in that: The mine safety production monitoring results obtained by performing pipeline splicing processing according to the mine image include: Performing feature point detection and matching according to the mine image to obtain feature point pairs of the mine image; Performing image registration according to the feature point pairs of the mine images to obtain an optimal transformation matrix between adjacent mine images; According to the optimal transformation matrix between the adjacent mine images, the mine images are sequentially subjected to multi-band fusion and fade-in and fade-out to obtain a mine fusion image; Using the mine fusion image, overall color smoothing and brightness adjustment are performed in sequence to obtain a mine fusion image to be spliced; The mine fusion images to be spliced ​​are used for splicing processing to obtain a mine safety production panoramic image as the mine safety production monitoring result.

Images