An intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tyred vehicle in a well
By combining infrared thermal imaging with improved image processing algorithms and models, the safety hazards in the transportation of trackless rubber-wheeled vehicles have been solved, achieving efficient safety monitoring and early warning, and reducing the risk of accidents.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUZHOU UNICORN COMM TECH CO LTD
- Filing Date
- 2023-06-20
- Publication Date
- 2026-05-12
AI Technical Summary
Existing intelligent image algorithms have high computational complexity and high resource consumption during trackless rubber-tired vehicle transportation, resulting in an inability to effectively reduce safety hazards and meet the safety monitoring needs in coal production environments.
An intelligent safety early warning method based on infrared thermal imaging is adopted. Target information is acquired through an infrared thermal imaging camera, and combined with an improved GPA-LWIFCM infrared image fuzzy segmentation algorithm and morphological weighted voting method, along with a Kalman filter model and a strong tracking model, personnel location and danger assessment are performed to output early warning information.
It enables efficient and safe monitoring of the trackless rubber-tired vehicle transportation process in complex environments, reduces safety hazards, and achieves a "zero" accident effect.
Smart Images
Figure CN117132482B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image data processing technology, and specifically to an intelligent safety early warning method based on infrared thermal imaging of an underground trackless rubber-wheeled vehicle. Background Technology
[0002] With the continuous development and improvement of the intelligence level of coal production enterprises, the number of accidents caused by violations of regulations by personnel in the transportation support system has increased, resulting in personal injury, production accidents and economic losses. In particular, the use of trackless rubber-tired vehicles for personnel and material transportation has led to blind spots in the driver's field of vision due to the complexity of the underground environment and the structural design of the trackless rubber-tired vehicles. Miners have also violated regulations by appearing in the areas where trackless rubber-tired vehicles are operating, resulting in irreparable safety accidents.
[0003] Traditional intelligent image algorithms employ methods from CNN, SVM, RF, FLD, and R-CNN for image target detection and classification, suitable for fields such as autonomous driving monitoring and medical and industrial manufacturing. However, these algorithms have high computational complexity, require substantial computing resources, necessitate large and diverse datasets for training, and require continuous trial and error adjustments to hyperparameters. Otherwise, they suffer from low computational accuracy, limitations due to data quality, and high resource consumption costs. Furthermore, they pose significant safety hazards during trackless rubber-tired vehicle transportation in complex environments. Therefore, existing intelligent image algorithms cannot be efficiently applied to trackless rubber-tired vehicles used in coal production, thus failing to minimize or eliminate safety hazards.
[0004] In the coal industry, for underground trackless rubber-tired vehicle transportation environments, the thermal imaging cameras on the trackless rubber-tired vehicles are crucial for monitoring, identifying, analyzing, and judging the image information of the objects being identified based on the flow characteristics of people, vehicles, and goods. Therefore, this application adopts an intelligent image algorithm based on infrared thermal imaging. By acquiring the target information, temperature characteristics, and distance of the target object through the thermal imaging camera, a comprehensive judgment is made on the classification of the target object and its distance from the trackless rubber-tired vehicle. Then, the algorithm board inside the trackless rubber-tired vehicle performs image calculation and information processing, and outputs the results to a voice alarm for playback, thereby effectively providing early warning, prediction, and alertness to the driver's driving behavior. Summary of the Invention
[0005] Technical problems to be solved
[0006] To address the shortcomings of existing technologies, this invention provides an intelligent safety early warning method based on infrared thermal imaging of underground trackless rubber-tired vehicles. By acquiring target information, temperature characteristics, and distance to the target using a thermal imaging camera, the method comprehensively judges the classification of the target and its distance to the trackless rubber-tired vehicle, thus solving a series of safety hazards that arise during the transportation of trackless rubber-tired vehicles under complex environmental conditions as mentioned in the background technology.
[0007] Technical solution
[0008] To achieve the above objectives, the present invention provides the following technical solution: an intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines, comprising the following steps:
[0009] S1. An intelligent image recognition algorithm board, microcontroller, infrared thermal imager and voice alarm are installed in the trackless rubber-wheeled vehicle to process image data information. By utilizing the principle of infrared thermal imaging, a GPA-LWIFCM infrared image fuzzy segmentation algorithm based on improved local mean decomposition is designed.
[0010] S2. By fusing the infrared image fuzzy segmentation algorithm of GPA-LWIFCM and the morphological weighted voting method to obtain personnel motion feature data, the position of personnel in the trackless rubber-wheel travel area underground is detected.
[0011] S3. At the same time, the intelligent image recognition algorithm board on the trackless rubber-tired vehicle establishes and calculates the theoretical and practical data of the Kalman filter model, and combines the strong tracking model to estimate and judge the position of people in the trackless rubber-tired vehicle's travel area.
[0012] S4. Based on the estimation method of the location of the person who mistakenly entered the trackless rubber-wheeled vehicle's travel area, the information is substituted into the hazard assessment model to further determine the safety of the person.
[0013] S5. Finally, the infrared thermal imaging camera of the trackless rubber-wheeled vehicle is used to identify the imaging area of people. Based on the imaging ratio of people, the distance information of people is calculated using big data algorithms. The calculated data is then output from the Ethernet interface to the joint debugging and linkage microcontroller via TCP / IP protocol. The microcontroller converts the digital information into voice content and then transmits it to the voice alarm for playback, thereby providing early warning, prediction and alertness to the driver of the trackless rubber-wheeled vehicle.
[0014] Furthermore, the intelligent image recognition algorithm board is a visual recognition algorithm chip that performs target detection, image segmentation, and image classification on the images captured by the infrared thermal imaging camera. It also includes image enhancement, image noise reduction, and preprocessing algorithms for the acquired image data to optimize image quality and improve the accuracy of the algorithm. The infrared thermal imaging camera is installed on a trackless rubber-wheeled vehicle.
[0015] Furthermore, the principle of infrared thermal imaging is to detect the infrared radiation emitted from the surface of the target object, and use an infrared thermal imager to convert the radiation energy into digital signals for acquisition, analysis, and display. Specifically, because the surface temperature of the target object is different, the intensity of the infrared radiation it emits is also different. The infrared radiation energy of the target object is collected by the optical lens of the infrared thermal imager, and the infrared radiation is converted into corresponding electrical signals. These signals are then identified, detected, and transmitted to the signal processor. The signal processor receives the electrical signals, performs digital processing, and converts them into digital images to display thermal distribution images.
[0016] Furthermore, the GPA-LWIFCM infrared image blur segmentation algorithm utilizes image enhancement and segmentation techniques to process infrared thermal imaging images, exhibiting high accuracy and robustness in target detection and segmentation. It also incorporates the LWIFCM algorithm, introducing a local weighting strategy to assign different weights to each pixel, thus better addressing issues such as uneven grayscale and noise in the image. The processing steps are as follows:
[0017] A1. Image preprocessing: Preprocessing methods are used to reduce noise and smooth the original infrared image. An improved Gaussian-Laplace filter is used to smooth the image, remove background and noise from the image, and make the image texture and structure clearer.
[0018] A2. Local Mean Decomposition: The Local Mean Decomposition (LMD) method is used to decompose infrared images. Each sub-band obtained by LMD is processed and enhanced separately, and then fed into the LWIFCM algorithm for local weighted calculation. LMD is an adaptive signal decomposition technique that combines extreme points with smoothing to decompose complex non-stationary signals into a set of simple intrinsic mode functions for signal denoising and decomposition.
[0019] A3. Wavelet decomposition: Wavelet decomposition is performed on each sub-band obtained by LMD decomposition to enhance high-frequency components with high energy, highlight the characteristics of the target edge, and suppress low-frequency components. Wavelet decomposition transforms the signal and decomposes it into frequency components at different scales. It decomposes the signal into a set of basis functions to facilitate the analysis and processing of signal characteristics. Its steps include selecting wavelet basis, decomposition, recursion and reconstruction.
[0020] A4. Endpoint Detection: Endpoint detection algorithms are used to detect and segment edge regions in order to extract target edge information. Endpoint detection algorithms use speech signal processing techniques to extract useful information from speech signals for subsequent speech recognition and audio processing tasks.
[0021] A5. Connected Component Segmentation: The connected component segmentation algorithm divides a set of target pixels into a consistent target region. The connected component segmentation algorithm groups adjacent pixels with common features in an image into a connected region for image segmentation.
[0022] A6. Post-processing: During post-processing, morphological processing and region growing algorithms are used to further extract target information and correct and improve the segmentation results.
[0023] Furthermore, the LMD method determines the boundary of each mode function through a local linear fit and describes each mode function as an orthogonal sequence of functions, with the following steps:
[0024] B1. Initialization: For a given signal, define the initial component h0(x) as the original signal;
[0025] B2. Auxiliary component: Define the auxiliary component g0(x) as the difference between the signal h0(x) and its mean;
[0026] B3. Mode decomposition: For the auxiliary component g0(x), find all its extreme points and perform linear interpolation to form the first component h1(x). Then subtract g0(x) from h1(x) to obtain a new auxiliary component g1(x).
[0027] B4. Repeat steps B2 and B3 above until the residual amount meets the stopping criterion or reaches the preset amount.
[0028] Furthermore, the LWIFCM algorithm is a fuzzy clustering image segmentation method. By introducing a local weighting strategy, it can better handle problems such as uneven gray levels and noise in the image, resulting in better image segmentation performance. Its steps are as follows:
[0029] C1. Initialization: Read in the image to be processed and initialize it. Represent the input image as a grayscale matrix. Set N-iter=1. Generate the center matrix V and key parameters m and a of LWIFCM, and calculate the expected number of clusters c and the fuzziness factor n.
[0030] C2. Weight Calculation: For each pixel in the image, its corresponding weight is obtained through local weighting calculation, where the weight is inversely proportional to the distance from the pixel;
[0031] C3. Cluster center initialization: Randomly initialize the c cluster centers;
[0032] C4. Distance Calculation: For each pixel, calculate its distance to each cluster center;
[0033] C5. Membership Degree Calculation: Based on the distance calculation results above, the membership degree of each pixel to each cluster center is calculated according to the LWIFCM algorithm.
[0034] C6. Cluster Center Update: Based on the membership degree of each pixel to each cluster center calculated in the previous step, update the position of each cluster center, that is, update the cluster center V and the fuzzy membership matrix U.
[0035] C7. Termination condition judgment: Determine whether the update of cluster centers has converged. If it has converged, jump to step 10; otherwise, continue to steps C5-C7.
[0036] C8. Correction of pixel membership by gray level: Correction of pixel membership using an improved Gaussian potential smoothing function;
[0037] C9. Seed point selection: The pixels corresponding to the center of each cluster obtained from image segmentation are used as seed points for the region growing method.
[0038] C10, Region Growing: The segmented image is obtained through the region growing algorithm.
[0039] Furthermore, the morphological weighted voting method is an image processing algorithm that combines morphological operations and weighted voting to handle the segmentation problem of binarized or grayscale images. Its processing steps are as follows:
[0040] D1. Perform infrared thermal imaging processing on the original image to obtain an infrared personnel image sequence;
[0041] D2. Based on the scenario and personnel involved in the operation of the trackless rubber-wheeled vehicle, define channel structural elements for morphological operations. The channels include Channel 1 and Channel 2. Channel 1 determines the position of personnel based on their movement information; Channel 2 interprets the content of the infrared image based on the image segmentation results.
[0042] D3. Perform morphological operations on the original image and morphological structural elements to obtain the results of personnel motion feature extraction and personnel infrared image segmentation.
[0043] D4. Based on actual needs, perform morphological weighted voting on the personnel target information and morphological skeleton in the morphological segmentation results to obtain the final personnel location information detection results.
[0044] Furthermore, the Kalman filter model employs Bayes' theorem, utilizing the relationship between prior and posterior probabilities to estimate the positions of personnel during the operation of the trackless rubber-tired vehicle. Since both its state sequence and measured values follow a Gaussian distribution, it exhibits optimality within a short time segment of the trackless rubber-tired vehicle's real-time operating state, with a small average estimation error. The calculation formula for the Kalman filter model includes the theoretical prediction formula for time t-t+1. In the formula, It is an n-dimensional vector of state components; It is a known n×n state transition matrix; It is a new variable that is subject to external control; B is an n×c matrix; It is the noise that follows a Gaussian distribution during the prediction process, and Corresponding The noise in each component is Gaussian white noise with an expectation of 0 and a covariance of Q, i.e. - , To generate noise; the formula for the actual measured value at time t-t+1 is: In the formula, It is an m×n transition matrix of observed state variables, used to transform m-dimensional measurements to n-dimensional values; It is observation noise, and Follows a Gaussian distribution, corresponding to - , To measure noise, the state at time t+1 is estimated by combining the posterior estimate at time t, thus obtaining the prior estimate at time t+1, which is then substituted into the formula. In the formula, It is the prior estimate of the covariance at time t+1; It is the assigned weight value; It is the state transition matrix; It is the posterior estimate of the covariance at time t; It is the process excitation noise covariance.
[0045] Furthermore, the strong tracking model is a computer vision algorithm for real-time tracking of moving targets. It includes target detection, feature extraction, tracker, occlusion detection, re-identification, and pose estimation. It tracks the position and pose of moving targets in continuous frames within the travel range of trackless rubber-tired vehicles in mines, and features high accuracy and real-time performance.
[0046] Furthermore, the aforementioned hazard assessment model is a mathematical model used to assess the probability of accidents and disasters in order to predict potential hazardous events in advance and take corresponding measures to minimize risk losses. It uses a strong tracking model to predict the location of personnel in the trackless rubber-tired vehicle's operating area underground, making hazard assessments. This model is based on a warning line equation, which predicts the distance to personnel positions to determine their safety. The warning line equation is y=kx+b, where k is the slope of the line and b is the intercept. When the trajectory of a target personnel crosses the warning line, the y-value of the coordinates will exceed the function value of the line, used to monitor the intrusion behavior of moving targets. When the position of the tracked target exceeds the warning line, a warning signal is issued, causing the voice alarm to play and triggering the corresponding control actions of the trackless rubber-tired vehicle.
[0047] This invention provides an intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines, which has the following beneficial effects:
[0048] 1. This invention employs a GPA-LWIFCM infrared image fuzzy segmentation algorithm based on improved local mean decomposition, incorporating the LMD method into the traditional GPA-LWIFCM infrared image fuzzy segmentation algorithm. This enables the layer-by-layer in-depth identification, classification, and segmentation of target features, achieving optimal identification and judgment results.
[0049] 2. This invention, by employing personnel information fusion based on morphological weighted voting, realizes the function of determining the location of target objects and predicting their positions based on current statistical models and strong tracking models. Furthermore, by combining with a hazard determination model, it makes early warning decisions on the highest risk coefficient for personnel who mistakenly enter the operating area of trackless rubber-tired vehicles in underground mines, thereby reducing safety hazards and achieving a "zero" accident effect in underground coal mine transportation operations. Attached Figure Description
[0050] Figure 1 This is a flowchart of the intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mine, according to the present invention.
[0051] Figure 2 This is a schematic diagram of the principle of the present invention based on infrared thermal imaging.
[0052] Figure 3 This is a flowchart of the infrared image fuzzy segmentation method based on GPA-LWIFCM of the present invention.
[0053] Figure 4 This is a flowchart of the personnel information fusion method based on morphological weighted voting method of the present invention.
[0054] Figure 5 This is a flowchart illustrating the budget method for determining the location of a person who has mistakenly entered the travel area of a trackless rubber-tired vehicle based on a strong tracking model, as described in this invention.
[0055] Figure 6 This is a flowchart of the hazard determination model of the present invention. Detailed Implementation
[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0057] Example 1
[0058] Please see Figure 1 This invention provides an intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in an underground mine, comprising the following steps:
[0059] S1. An intelligent image recognition algorithm board, microcontroller, infrared thermal imaging camera and voice alarm are installed in the trackless rubber-wheeled vehicle to process image data information. By utilizing the principle of infrared thermal imaging, a GPA-LWIFCM infrared image fuzzy segmentation algorithm based on improved local mean decomposition is designed.
[0060] S2. By fusing the infrared image fuzzy segmentation algorithm of GPA-LWIFCM and the morphological weighted voting method to obtain personnel motion feature data, the position of personnel in the travel area of the trackless rubber-tired vehicle in the mine is detected.
[0061] S3. At the same time, the intelligent image recognition algorithm board on the trackless rubber-tired vehicle establishes and calculates the theoretical and practical data of the Kalman filter model, and combines the strong tracking model to estimate and judge the position of people in the trackless rubber-tired vehicle's travel area.
[0062] S4. Based on the estimation method of the location of the person who mistakenly entered the trackless rubber-wheeled vehicle's travel area, the information is substituted into the hazard assessment model to further determine the safety of the person.
[0063] S5. Finally, the infrared thermal imaging camera of the trackless rubber-wheeled vehicle is used to identify the imaging area of people. Based on the imaging ratio of people, the distance information of people is calculated using big data algorithms. The calculated data is then output from the Ethernet interface to the joint debugging and linkage microcontroller via TCP / IP protocol. The microcontroller converts the digital information into voice content and then transmits it to the voice alarm for playback, thereby providing early warning, prediction and alertness to the driver of the trackless rubber-wheeled vehicle.
[0064] Example 2
[0065] Please see Figure 2-6This invention provides an intelligent safety early warning method based on infrared thermal imaging of an underground trackless rubber-tired vehicle. This embodiment is a further disclosure of Embodiment 1. The intelligent image recognition algorithm board is a visual recognition algorithm chip that performs target detection, image segmentation, and image classification on images acquired by the infrared thermal imaging camera. It also includes image enhancement, image noise reduction, and preprocessing algorithms for the acquired image data to optimize image quality and improve the accuracy of the algorithm. The infrared thermal imaging camera is installed on the trackless rubber-tired vehicle. The high-definition camera utilizes advanced sensor technology and has a resolution of ≥640*512, used to capture high-resolution images and videos in underground coal mines. The infrared thermal imager is an instrument that uses the principle of thermal radiation to capture the infrared radiation energy of the target surface using a thermal imaging sensor, converting the surface temperature distribution of the object into a digital image for quickly and accurately determining the thermal characteristics of the target.
[0066] In a preferred embodiment, the principle of infrared thermal imaging is to detect the infrared radiation emitted from the surface of a target object, and then use an infrared thermal imaging camera to convert the radiated energy into digital signals for acquisition, analysis, and display. Specifically, because the surface temperature of a target object varies, the intensity of the infrared radiation it emits also varies. The infrared radiation energy of the target object is acquired through the optical lens of the infrared thermal imager, and the infrared radiation is converted into corresponding electrical signals. These signals are then identified, detected, and transmitted to a signal processor. The signal processor receives the electrical signals, performs digital processing, and converts them into digital images, displaying a thermal distribution image. The infrared light used is of wavelength 3-5 nm. Or 8-14 It is used to quickly and non-contactly detect the thermal state of a target and determine its temperature based on the intensity of the infrared radiation signal emitted by the surface temperature of the object.
[0067] In a preferred embodiment, the GPA-LWIFCM infrared image blur segmentation algorithm utilizes image enhancement and segmentation techniques to identify infrared thermal images, exhibiting high accuracy and robustness in target detection and segmentation. Furthermore, it integrates the LWIFCM algorithm, introducing a local weighting strategy to assign different weights to each pixel, thereby better addressing issues such as uneven grayscale and noise in the image. The processing steps are as follows:
[0068] A1. Image preprocessing: Preprocessing methods are used to reduce noise and smooth the original infrared image. An improved Gaussian-Laplace filter is used to smooth the image, remove background and noise from the image, and make the image texture and structure clearer.
[0069] A2. Local Mean Decomposition: The Local Mean Decomposition (LMD) method is used to decompose infrared images. Each sub-band obtained by LMD is processed and enhanced separately, and then fed into the LWIFCM algorithm for local weighted calculation. LMD is an adaptive signal decomposition technique that combines extreme points with smoothing to decompose complex non-stationary signals into a set of simple intrinsic mode functions for signal denoising and decomposition.
[0070] A3. Wavelet Decomposition: Wavelet decomposition is performed on each sub-band obtained from LMD decomposition. High-frequency components with high energy are enhanced to highlight the characteristics of the target edge, while low-frequency components are suppressed. Wavelet decomposition transforms the signal, decomposing it into frequency components at different scales. The signal is decomposed into a set of basis functions for convenient analysis and processing of signal features. The steps include selecting a wavelet basis, decomposition, recursion, and reconstruction. Selecting the wavelet basis involves choosing wavelet basis functions of different shapes based on the signal characteristics and the needs of the processing task. Decomposition involves filtering and downsampling the original signal through the wavelet basis functions to obtain low-frequency and high-frequency sub-bands at different scales. Recursion involves repeating the above steps for the low-frequency sub-bands, gradually refining them to obtain detailed information at different scales. Reconstruction involves reconstructing the original signal from all the decomposed sub-bands through the inverse transform of the wavelet basis functions.
[0071] A4. Endpoint Detection: Endpoint detection algorithms are used to detect and segment edge regions in order to extract target edge information. Endpoint detection algorithms use speech signal processing techniques to extract useful information from speech signals for subsequent speech recognition and audio processing tasks.
[0072] A5. Connected Component Segmentation: The connected component segmentation algorithm divides a set of target pixels into a consistent target region. The connected component segmentation algorithm groups adjacent pixels with common features in an image into a connected region for image segmentation.
[0073] A6. Post-processing: During post-processing, morphological processing and region growing algorithms are used to further extract target information and correct and improve the segmentation results.
[0074] In a preferred embodiment, the LMD method determines the boundary of each mode function through a local linear fit and describes each mode function as an orthogonal sequence of functions, with the following steps:
[0075] B1. Initialization: For a given signal, define the initial component h0(x) as the original signal;
[0076] B2. Auxiliary component: Define the auxiliary component g0(x) as the difference between the signal h0(x) and its mean;
[0077] B3. Mode decomposition: For the auxiliary component g0(x), find all its extreme points and perform linear interpolation to form the first component h1(x). Then subtract g0(x) from h1(x) to obtain a new auxiliary component g1(x).
[0078] B4. Repeat steps B2 and B3 above until the residual amount meets the stopping criterion or reaches the preset amount.
[0079] In a preferred embodiment, the LWIFCM algorithm is a fuzzy clustering image segmentation method. By introducing a local weighting strategy, it can better handle problems such as uneven gray levels and noise in the image, resulting in better image segmentation performance. The steps are as follows:
[0080] C1. Initialization: Read in the image to be processed and initialize it. Represent the input image as a grayscale matrix. Set N-iter=1. Generate the center matrix V and key parameters m and a of LWIFCM, and calculate the expected number of clusters c and the fuzziness factor n.
[0081] C2. Weight Calculation: For each pixel in the image, its corresponding weight is obtained through local weighting calculation, where the weight is inversely proportional to the distance from the pixel;
[0082] C3. Cluster center initialization: Randomly initialize the c cluster centers;
[0083] C4. Distance Calculation: For each pixel, calculate its distance to each cluster center;
[0084] C5. Membership Degree Calculation: Based on the distance calculation results above, the membership degree of each pixel to each cluster center is calculated according to the LWIFCM algorithm.
[0085] C6. Cluster Center Update: Based on the membership degree of each pixel to each cluster center calculated in the previous step, update the position of each cluster center, that is, update the cluster center V and the fuzzy membership matrix U.
[0086] C7. Termination condition judgment: Determine whether the update of cluster centers has converged. If it has converged, jump to step 10; otherwise, continue to steps C5-C7.
[0087] C8. Correction of pixel membership by gray level: Correction of pixel membership using an improved Gaussian potential smoothing function;
[0088] C9. Seed point selection: The pixels corresponding to the center of each cluster obtained from image segmentation are used as seed points for the region growing method.
[0089] C10, Region Growing: The segmented image is obtained through the region growing algorithm.
[0090] In a preferred embodiment, the morphological weighted voting method is an image processing algorithm that combines morphological operations and weighted voting to handle the segmentation problem of binarized or grayscale images. Its processing steps are as follows:
[0091] D1. Perform infrared thermal imaging processing on the original image to obtain an infrared personnel image sequence;
[0092] D2. Based on the scenario and personnel involved in the operation of the trackless rubber-wheeled vehicle, define channel structural elements for morphological operations. The channels include Channel 1 and Channel 2. Channel 1 determines the position of personnel based on their movement information; Channel 2 interprets the content of the infrared image based on the image segmentation results.
[0093] D3. Perform morphological operations on the original image and morphological structural elements to obtain the results of personnel motion feature extraction and personnel infrared image segmentation.
[0094] D4. Based on actual needs, perform morphological weighted voting on the personnel target information and morphological skeleton in the morphological segmentation results to obtain the final personnel location information detection results.
[0095] In a preferred embodiment, the Kalman filter model employs Bayes' theorem, utilizing the relationship between prior and posterior probabilities to estimate the positions of personnel during the operation of the trackless rubber-tired vehicle. Since both the state sequence and measured values follow a Gaussian distribution, it exhibits optimality within a short time segment of the trackless rubber-tired vehicle's real-time operating state, with a small average estimation error. The calculation formula for the Kalman filter model includes the theoretical prediction formula for time t-t+1. In the formula, It is an n-dimensional vector of state components; It is a known n×n state transition matrix; It is a new variable that is subject to external control; B is an n×c matrix; It is the noise that follows a Gaussian distribution during the prediction process, and Corresponding The noise in each component is Gaussian white noise with an expectation of 0 and a covariance of Q, i.e. - , To generate noise; the formula for the actual measured value at time t-t+1 is: In the formula, It is an m×n transition matrix of observed state variables, used to transform m-dimensional measurements to n-dimensional values; It is observation noise, and Follows a Gaussian distribution, corresponding to - , To measure noise, the state at time t+1 is estimated by combining the posterior estimate at time t, thus obtaining the prior estimate at time t+1, which is then substituted into the formula. In the formula, It is the prior estimate of the covariance at time t+1; It is the assigned weight value; It is the state transition matrix; It is the posterior estimate of the covariance at time t; It is the process excitation noise covariance.
[0096] In a preferred embodiment, the strong tracking model is a computer vision algorithm for real-time tracking of moving targets. It includes target detection, feature extraction, a tracker, occlusion detection, re-identification, and pose estimation. This model tracks the position and pose of moving targets within the travel range of a trackless rubber-tired vehicle in an underground mine across consecutive frames, exhibiting high accuracy and real-time performance. Target detection is the starting point of the strong tracking model, identifying the target from consecutive frames. Feature extraction transforms the detected target region into easily processed vectors. The tracker maintains tracking by detecting the position and pose of the identified target in new consecutive frames. Occlusion detection detects target occlusion and updates the tracker's state accordingly. Re-identification helps the tracker reconfirm the target's identity and continue tracking when the target is occluded or similar regions interfere with tracking. Pose estimation estimates the target's rotation or shape changes.
[0097] In a preferred embodiment, the hazard assessment model is a mathematical model used to assess the probability of accidents and disasters in order to predict potential hazardous events in advance and take corresponding measures to minimize risk losses. It uses a strong tracking model to predict the location of personnel in the trackless rubber-tired vehicle's operating area underground, making hazard assessments. This model is based on a warning line equation, which predicts the distance to personnel positions to determine their safety. The warning line equation is y=kx+b, where k is the slope of the line and b is the intercept. When the trajectory of a target personnel crosses the warning line, the y-value of the coordinates will exceed the function value of the line, used to monitor the intrusion behavior of moving targets. When the position of the tracked target exceeds the warning line, a warning signal is issued, causing a voice alarm to play and triggering corresponding control actions on the trackless rubber-tired vehicle.
[0098] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.
[0099] The technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0100] In conclusion, the above descriptions are merely examples of units and algorithm steps described in the disclosed embodiments of the present invention, which can be implemented using electronic hardware or a combination of computer software and electronic hardware. They are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art within the spirit and principles of the present invention and within the scope of the technology disclosed in this application should be included within the protection scope of the present invention.
Claims
1. An intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines, characterized in that: The steps include the following: S1. An intelligent image recognition algorithm board, microcontroller, infrared thermal imager and voice alarm are installed in the trackless rubber-wheeled vehicle to process image data information. By utilizing the principle of infrared thermal imaging, a GPA-LWIFCM infrared image fuzzy segmentation algorithm based on improved local mean decomposition is designed. S2. By fusing the GPA-LWIFCM infrared image fuzzy segmentation algorithm and the morphological weighted voting method to obtain personnel motion feature data, the location of personnel in the underground trackless rubber-tired vehicle's travel area is detected. The GPA-LWIFCM infrared image fuzzy segmentation algorithm utilizes image enhancement and segmentation techniques to process infrared thermal images, and integrates the LWIFCM algorithm, introducing a local weighting strategy to assign different weights to each pixel. The processing steps are as follows: A1. Image preprocessing: Preprocessing methods are used to reduce noise and smooth the original infrared image; A2. Local Mean Decomposition: The infrared image is decomposed using the Local Mean Decomposition (LMD) method. Each sub-band obtained by LMD is processed and enhanced separately, and then fed into the LWIFCM algorithm for local weighted calculation. The LMD method determines the boundary of each modal function through a local linear fitting and describes each modal function as an orthogonal sequence of functions. The steps are as follows: B1. Initialization: For a given signal, define the initial component h0(x) as the original signal; B2. Auxiliary component: Define the auxiliary component g0(x) as the difference between the signal h0(x) and its mean; B3. Mode decomposition: For the auxiliary component g0(x), find all its extreme points and perform linear interpolation to form the first component h1(x). Then subtract g0(x) from h1(x) to obtain a new auxiliary component g1(x). B4. Repeat steps B2 and B3 above until the residual amount meets the stopping criterion or reaches the preset amount of amount; A3. Wavelet decomposition: Wavelet decomposition is performed on each sub-band obtained by LMD decomposition to enhance the high-frequency components with high energy, highlight the features of the target edge, and suppress the low-frequency components. A4. Endpoint Detection: Endpoint detection algorithms are used to detect and segment edge regions in order to extract target edge information; A5. Connected Component Segmentation: The connected component segmentation algorithm is used to divide the target pixel set into a consistent target region. A6. Post-processing: During post-processing, morphological processing and region growing algorithms are used to further extract target information and correct and improve the segmentation results. S3. At the same time, the intelligent image recognition algorithm board on the trackless rubber-tired vehicle establishes and calculates the theoretical and practical data of the Kalman filter model, and combines the strong tracking model to estimate and judge the position of people in the trackless rubber-tired vehicle's travel area. S4. Based on the estimation method of the location of the person who mistakenly entered the trackless rubber-wheeled vehicle's travel area, the information is substituted into the hazard assessment model to further determine the safety of the person. S5. Finally, the infrared thermal imaging camera of the trackless rubber-wheeled vehicle is used to identify the imaging area of people. Based on the imaging ratio of people, the distance information of people is calculated using big data algorithms. The calculated data is then output from the Ethernet interface to the joint debugging and linkage microcontroller via TCP / IP protocol. The microcontroller converts the digital information into voice content and then transmits it to the voice alarm for playback, thereby providing early warning, prediction and alertness to the driver of the trackless rubber-wheeled vehicle.
2. The intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines according to claim 1, characterized in that: The intelligent image recognition algorithm board is a visual recognition algorithm chip that performs target detection, image segmentation, and image classification on images acquired by an infrared thermal imaging camera. It also includes image enhancement, image noise reduction, and preprocessing algorithms for the acquired image data.
3. The intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines according to claim 2, characterized in that: The principle of infrared thermal imaging is that the infrared radiation intensity emitted by a target object varies depending on its surface temperature. The infrared radiation energy of the target object is collected by the optical lens of the infrared thermal imaging camera, and the infrared radiation is converted into a corresponding electrical signal. This signal is then identified, detected, and transmitted to the signal processor. The signal processor receives the electrical signal, performs digital processing, and converts it into a digital image, which is then displayed as a thermal distribution image.
4. The intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines according to claim 1, characterized in that: The LWIFCM algorithm is a fuzzy clustering image segmentation method, and its steps are as follows: C1. Initialization; C2. Weight calculation; C3. Cluster center initialization; C4. Distance calculation; C5. Membership degree calculation; C6. Cluster center update; C7. Termination condition determination; C8, grayscale level correction of pixel membership; C9. Seed point selection; C10, Regional Growth.
5. The intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines according to claim 1, characterized in that: The morphological weighted voting method is an image processing algorithm that combines morphological operations and weighted voting to handle the segmentation problem of binarized or grayscale images. Its processing steps are as follows: D1. Perform infrared thermal imaging processing on the original image to obtain an infrared personnel image sequence; D2. Based on the operating scenario and personnel of the trackless rubber-wheeled vehicle, define the channel structural elements for morphological operations; D3. Perform morphological operations on the original image and morphological structural elements to obtain the results of personnel motion feature extraction and personnel infrared image segmentation. D4. Based on actual needs, perform morphological weighted voting on the personnel target information and morphological skeleton in the morphological segmentation results to obtain the final personnel location information detection results.
6. The intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines according to claim 1, characterized in that: The Kalman filter model uses Bayes' theorem and the relationship between prior and posterior probabilities to estimate the positions of personnel during the movement of the trackless rubber-tired vehicle. The calculation formula for the Kalman filter model includes the theoretical prediction formula for time t-t+1. In the formula, It is an n-dimensional vector of state components; It is a known n×n state transition matrix; It is a new variable that is subject to external control; B is an n×c matrix; It is the noise that follows a Gaussian distribution during the prediction process, and Corresponding The noise in each component is Gaussian white noise with an expectation of 0 and a covariance of Q, i.e. - , To generate noise; the formula for the actual measured value at time t-t+1 is: In the formula, It is an m×n transition matrix of observed state variables, used to transform m-dimensional measurements to n-dimensional values; It is observation noise, and Follows a Gaussian distribution, corresponding to - , To measure noise, the state at time t+1 is estimated by combining the posterior estimate at time t, thus obtaining the prior estimate at time t+1, which is then substituted into the formula. In the formula, It is the prior estimate of the covariance at time t+1; It is the assigned weight value; It is the state transition matrix; It is the posterior estimate of the covariance at time t; It is the process excitation noise covariance.
7. The intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines according to claim 1, characterized in that: The strong tracking model is a computer vision algorithm for real-time tracking of moving targets. It includes target detection, feature extraction, tracker, occlusion detection, re-identification, and pose estimation. It tracks the position and pose of moving targets in continuous frames within the travel range of trackless rubber-tired vehicles in underground mines.
8. The intelligent safety early warning method based on infrared thermal imaging of a trackless rubber-tired vehicle in underground mines according to claim 1, characterized in that: The aforementioned hazard assessment model is a mathematical model used to assess the probability of accidents and disasters in order to predict potential hazardous events in advance and take corresponding measures to minimize risk and loss. It uses a strong tracking model to predict the location of personnel in the trackless rubber-tired vehicle operating area in the mine, and performs hazard assessment. It belongs to the warning line equation and predicts the distance to the personnel's location to determine whether the personnel are safe.