Visible light and infrared thermal imaging combined material field overtemperature detection filtering method

Abstract

The invention discloses a visible light and infrared thermal imaging combined material field over-temperature detection filtering method, which specifically comprises the following steps: step 1, constructing a visible light image acquisition system and an infrared thermal imager, and setting point locations of the visible light image acquisition system and an infrared binocular image acquisition system in a material field, with the visible light image acquisition system and the infrared binocular image acquisition system being at the same visual angle; 2, collecting material field image data through a visible light image collection system, and carrying out vehicle identification on the collected image; 3, the infrared thermal imager carries out temperature measurement; 4, when the temperature, detected by the infrared thermal imager, of a certain area exceeds a preset threshold value, it is determined that the area is a high-temperature fire point; and 5, judging whether the high-temperature fire point is the fire of the material pile or not, and if yes, giving an alarm. According to the method, the detection precision can be remarkably improved, the environmental adaptability is enhanced, the early warning efficiency is improved, and comprehensive information analysis is realized.

Description

Technical Field

[0001] The invention relates to the technical field of machine vision, and in particular to a material field over-temperature detection and filtering method combining visible light and infrared thermal imaging. Background Art

[0002] During the temperature detection process of the biomass material yard, high-temperature vehicles will affect the normal surface temperature detection of the pile. In the over-temperature warning of infrared detection, the monitoring temperature of the pile is lower than the temperature of the vehicle hood and exhaust pipe, and the interference item cannot be avoided by range setting. Simply using thermal imaging for image recognition, and because the thermal imaging color gamut is small and the features are not obvious, the vehicle detection model is poor and the accuracy is low. In the prior art, only infrared cameras are used for monitoring. In fact, there are many vehicle interferences in factory operations. If the alarm is issued according to the difference between the temperature measurement point and the external temperature, there are many false alarms. In addition, the camera is only facing the pile, and the viewing range is small. Each pile must be equipped with an infrared camera, which is costly. Summary of the invention

[0003] Purpose of the invention: In order to solve the problems existing in the above-mentioned prior art, the present invention provides a material field overtemperature detection and filtering method combining visible light and infrared thermal imaging.

[0004] Technical solution: The present invention discloses a material field over-temperature detection and filtering method combining visible light and infrared thermal imaging, which specifically includes the following steps:

[0005] Step 1: Build a visible light image acquisition system and an infrared thermal imager, set up the visible light image acquisition system and the infrared thermal imager at the material yard, and both have the same viewing angle;

[0006] Step 2: Collect the material field image data through the visible light image acquisition system, and perform vehicle recognition on the collected images;

[0007] Step 3: Use an infrared thermal imager to measure temperature;

[0008] Step 4: When the temperature of a certain area detected by the infrared thermal imager exceeds a preset threshold, the area is identified as a high-temperature fire point, and the image at this time is enlarged; the image collected by the visible light image acquisition system and the enlarged image are calibrated for dual-viewing angles, and the target area is cropped out from the image collected by the visible light image acquisition system;

[0009] Step 5: Determine whether the high-temperature fire point is a pile fire, and sound an alarm if it is a pile fire.

[0010] Furthermore, in step 2, the vehicles in the material yard image data collected by the visible light image acquisition system are annotated and used as a training set, the training set is used to train the YOLO model, and the trained YOLO model is used to perform vehicle recognition.

[0011] Furthermore, the present invention also includes using an image segmentation algorithm to perform fine segmentation on the identified vehicles.

[0012] Furthermore, the image calibration in step 4 is specifically as follows:

[0013] Step 4.1: Obtain the magnification z of the infrared thermal imager; and the length w and width h of the image acquired by the visible light image acquisition system at the corresponding time and viewing angle;

[0014] Step 4.2: Crop the target image from the image acquired by the visible light image acquisition system according to the following formula:

[0015]

[0016] Among them, h1 is the width of the target image, w1 is the length of the target image, (a0, b0) is the upper left corner coordinate of the target image in the image captured by the visible light image acquisition system, and (a1, b1) is the lower right corner coordinate of the target image in the image captured by the visible light image acquisition system.

[0017] Furthermore, the step 5 is specifically as follows: determining whether there is a vehicle in the target image; if not, determining that the pile of materials is on fire; if there is a vehicle in the target image, determining whether the high-temperature fire point is at the vehicle position; if so, determining that the high-temperature fire point is the vehicle, not the pile of materials on fire; otherwise, determining that the pile of materials is on fire.

[0018] An electronic device / system for a material yard overtemperature detection and filtering method comprises a processor and a memory, wherein the memory stores execution instructions of the processor, and the processor is configured to execute the execution instructions to implement the material yard overtemperature detection and filtering method.

[0019] A computer-readable storage medium is used to store a program, and the program is executed to implement the material yard overtemperature detection and filtering method.

[0020] Beneficial effects:

[0021] 1. The present invention uses visible light and infrared thermal imaging to identify high temperature warnings in material yards, eliminating vehicle interference factors. This method can significantly improve detection accuracy, enhance environmental adaptability, improve warning efficiency and realize comprehensive information analysis.

[0022] 2. The present invention uses visible light to detect vehicle targets, with good recognition effect and high accuracy. Visible light imaging can capture rich information such as color and texture on the vehicle surface, which is useful for identifying characteristic information such as the type and color of the vehicle. Infrared thermal imaging mainly displays the temperature distribution of objects, has a small color gamut, and lacks color and texture information. In addition, in an environment with sufficient light, visible light imaging can provide high-definition images with sharp edges and rich details, which helps to more accurately identify vehicle targets.

[0023] 3. The present invention uses infrared to detect high temperature and fire with high accuracy. Infrared thermal imaging technology can detect the temperature distribution on the surface of the obstructed object by receiving infrared radiation emitted by the surface of the object. It can penetrate smoke, haze and other obstructions and detect the temperature distribution on the surface of the obstructed object. This non-contact temperature measurement method can obtain its temperature information in real time and accurately without interfering with the object being measured. At the same time, combined with vehicle identification under visible light, the high temperature interference of the vehicle is filtered to improve the accuracy of the high temperature ignition point. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of the present invention.

[0025] Figure 2 This is a diagram of image calibration. DETAILED DESCRIPTION

[0026] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] like Figure 1 As shown, the present invention provides a method for detecting and filtering over-temperature in a material field by combining visible light and infrared thermal imaging. Specifically:

[0028] S0 builds a visible light image acquisition system and an infrared thermal imager. Set up a visible light image acquisition system and an infrared thermal imager in the material yard. The binocular camera that combines visible light and infrared has the same viewing angle. Collect image data of crushers, forklifts, and trucks under visible light as a training set.

[0029] The visible light image acquisition system and the infrared thermal imager are identified as one machine with the same point, and the center point of the detection angle of view is consistent. The magnification of the infrared thermal imager is consistent with the default magnification of the visible light image acquisition system. The shooting screen of the visible light image acquisition system and the infrared thermal imager is consistent, which is a different imaging method of the same screen. When the infrared thermal imager adjusts the magnification, the screen is enlarged, and the center of the angle of view remains consistent with the visible light image acquisition system.

[0030] S1 classifies and annotates the image dataset, annotates it as vehicles, and frames the vehicles in the visible light perspective of the discharge yard, providing a training set for subsequent target detection.

[0031] Collect target vehicle images and video datasets including crushers, forklifts, trucks and other mobile high-temperature vehicles, covering different angles, lighting conditions, occlusions and background environments to improve the generalization ability of the model. Furthermore, you can also filter from public image datasets as a supplement to the training set. Annotate the collected datasets, identify the bounding boxes of high-temperature vehicles such as crushers, forklifts, trucks, and assign corresponding category labels. Use the deep learning framework to build the YOLO model for training.

[0032] S2 uses the open source YOLO algorithm to detect vehicles. Use the training set in step S1 to train the model and obtain a vehicle detection model. The vehicle detection model is used in target detection under the perspective of the visible light image acquisition system. The effect achieved by this model is to identify vehicles in video surveillance images under the perspective of the visible light image acquisition system and to frame vehicle targets in the image.

[0033] S3 uses an open-source image segmentation algorithm to segment the target recognition results of the YOLO model, segment the framed vehicle target, and segment the vehicle area. The effect achieved by this model is to segment the outline, position, and area of ​​the vehicle. The vehicle can be identified in a refined manner.

[0034] In this embodiment, a trained image segmentation model can be selected, such as a Mask R-CNN model, which should include vehicle categories. The target detection result of the YOLO model, that is, the vehicle bounding box, is used as the processing area of ​​the Mask R-CNN model. That is, the result bounding box of the YOLO model is first cropped out, and then input into the Mask R-CNN model for instance segmentation, to obtain the contour, position, and area of ​​the vehicle that are accurately segmented, and displayed in the image in the form of a mask. The display result will also be integrated back into the original image. That is, the vehicle photographed under visible light is accurately masked to outline the contour and area of ​​the vehicle.

[0035] This embodiment can also use other high temperature abnormal target filtering. High temperature moving vehicles are common abnormal values ​​in the over-temperature alarm of the material yard, so the present invention focuses on identifying and eliminating high temperature moving vehicles. If other related high temperature interference items appear in the material yard, the above target detection method can also be combined with the contour recognition method to determine the exact location of the interference object.

[0036] S4 infrared thermal imager over-temperature alarm filtering. The logic of infrared over-temperature alarm is to set the temperature threshold temp within the viewing angle range. If the temperature threshold is exceeded, the over-temperature alarm area will be displayed in the infrared thermal image.

[0037] S5 dual-viewing angle image scale calibration. The infrared thermal imager measures temperature through infrared radiation. If high temperature is detected under wide viewing angle, the image will be magnified, and the focal length and magnification of the image will be flexibly adjusted for detection, increasing accuracy, confirming the accurate temperature value and providing feedback. The focal length and magnification of visible light are not synchronously zoomed with infrared imaging, and visible light has a fixed magnification. Infrared has a magnification range that can magnify the viewing angle. Therefore, the viewing angle range of visible light is greater than or equal to that of the infrared thermal imager. The image captured by the visible light image acquisition system (visible light image) needs to be scaled with the image of the infrared thermal imager, that is, the thermal imaging window in the visible light image needs to be cropped out. For example Figure 2 As shown, first obtain the magnification z of the infrared thermal imager, the length and width of the visible light image are w and h, and calculate the length and width of the target image: w1 and h1, where: w1 = w / z; h1 = h / z. Find the upper left and lower right coordinates (a0, b0) and (a1, b1) for cropping the target image in the visible light image, where a0 = (w-w1) / 2, b0 = (h-h1) / 2, a1 = (w+w1) / 2, b1 = (h+h1) / 2. According to the coordinates, the target is intercepted to obtain a picture with the same viewing angle as the infrared thermal imager, that is, the visible light image and infrared image under the same viewing angle range are different imaging methods of the same picture.

[0038] S6 Vehicle interference filtering. The visible light image after target detection, vehicle segmentation, and scale change is compared with the corresponding infrared image. If the infrared thermal imager detects a high-temperature fire point, it returns to the visible light image for comparison. If there is a vehicle in the results of target detection and image segmentation in the visible light, it is determined whether the fire point is within the area of ​​the vehicle. If the fire point is within the target area of ​​the vehicle, the high temperature alarm is for the vehicle, not the fire point, that is, no fire alarm is issued. If the fire point is not within the target area of ​​the vehicle, but outside the target area of ​​the vehicle, it is not the high temperature interference of the vehicle, that is, a high temperature alarm is issued.

[0039] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.

Claims

1. A material field overtemperature detection and filtering method combining visible light and infrared thermal imaging, characterized in that: The specific steps include: Step 1: Build a visible light image acquisition system and an infrared thermal imager, set up the visible light image acquisition system and the infrared thermal imager at the material yard, and both have the same viewing angle; Step 2: Collect the material field image data through the visible light image acquisition system, and perform vehicle recognition on the collected images; Step 3: Use an infrared thermal imager to measure temperature; Step 4: When the temperature of a certain area detected by the infrared thermal imager exceeds the preset threshold, the area is identified as a high-temperature fire point, and the image at this time is enlarged; Performing dual-view image scale calibration on the image collected by the visible light image collection system and the enlarged graphic, and cropping the target area from the image collected by the visible light image collection system; Step 5: Determine whether the high-temperature fire point is a pile fire, and sound an alarm if it is a pile fire.

2. The method for detecting and filtering over-temperature in a material field by combining visible light and infrared thermal imaging according to claim 1 is characterized in that: In the step 2, the vehicles in the material yard image data collected by the visible light image acquisition system are marked and used as a training set. The training set is used to train the YOLO model, and the trained YOLO model is used to perform vehicle recognition.

3. The method for detecting and filtering over-temperature in a material field by combining visible light and infrared thermal imaging according to claim 2 is characterized in that: The present invention also includes using an image segmentation algorithm to perform fine segmentation on the identified vehicle.

4. The method for detecting and filtering over-temperature in a material field by combining visible light and infrared thermal imaging according to claim 1 is characterized in that: The image calibration in step 4 is specifically as follows: Step 4.1: Obtain the magnification z of the infrared thermal imager; and the length w and width h of the image acquired by the visible light image acquisition system at the corresponding time and viewing angle; Step 4.2: Crop the target image from the image acquired by the visible light image acquisition system according to the following formula: Among them, h1 is the width of the target image, w1 is the length of the target image, (a0, b0) is the upper left corner coordinate of the target image in the image captured by the visible light image acquisition system, and (a1, b1) is the lower right corner coordinate of the target image in the image captured by the visible light image acquisition system.

5. The method for detecting and filtering over-temperature in a material field by combining visible light and infrared thermal imaging according to claim 1 is characterized in that: The step 5 is specifically as follows: determining whether there is a vehicle in the target image; if not, determining that the pile of materials is on fire; if there is a vehicle in the target image, determining whether the high-temperature fire point is at the position of the vehicle; if so, determining that the high-temperature fire point is the vehicle, not the pile of materials on fire; otherwise, determining that the pile of materials is on fire.

6. An electronic device / system for a material field overtemperature detection and filtering method, characterized in that: It comprises a processor and a memory, wherein the memory stores execution instructions of the processor, and the processor is configured to execute the execution instructions to implement the material yard overtemperature detection and filtering method as described in any one of claims 1-5.

7. A computer-readable storage medium for storing a program, characterized in that: Execute the program to implement the material yard overtemperature detection and filtering method described in any one of claims 1-5.

Images