Fire source recognition and positioning method and system based on infrared image and visible light image

Abstract

The application provides a fire source recognition and positioning method and system based on infrared images and visible light images, and relates to the technical field of fire fighting.The application monitors fire occurrence in a charging place through a fire alarm system, continuously acquires infrared images and visible light images of the charging place and performs pretreatment; analyzes the light flow characteristics of the infrared images and the contrast of the visible light images to generate a hot air flow recognition coefficient; divides a high-temperature fire area into a hot air flow area and a fire source area through the hot air flow recognition coefficient; studies the flickering condition of the fire source and divides the flickering grades; divides the fire source into a lithium battery fire source, an electrical equipment fire source and other fire sources according to the flickering grades and color characteristics; acquires camera parameters, and positions the fire source according to the camera parameters and the position coordinates of the fire source.The application can accurately distinguish the fire source from the hot air flow, improves the positioning precision of the fire source, accurately identifies the type of the fire source, and helps to develop more professional and targeted fire extinguishing strategies.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of fire fighting technology, in particular to a fire source recognition and positioning method and system based on infrared images and visible light images. BACKGROUND

[0002] With the widespread use of electric vehicles and energy storage devices, the fire risk of charging sites is gradually increasing. Especially during the charging process, the fire risk is affected by factors such as battery charging status, equipment failure, etc. Traditional fire monitoring systems often rely on smoke, temperature sensing, and flame alarms, etc. However, these systems have certain limitations in complex environments, especially in terms of accurate positioning and determination of fire sources.

[0003] Although the prior art can mark the position of the fire source by temperature, this method has certain limitations. The hot air flow generated by the fire source also has high temperature characteristics, which makes it difficult for temperature marking to accurately distinguish between the fire source itself and the surrounding hot air flow area. In practical applications, hot air flow may cover a large area, and its temperature value is similar to that of the fire source area, causing important interference to the identification of the fire source. Due to the diffusion characteristics of hot air flow, the traditional temperature marking method often cannot effectively isolate the boundary between the fire source and the hot air flow, resulting in inaccurate positioning of the fire source, and even possible misjudgment or omission of the position of the fire source. This problem is particularly prominent in complex environments, especially in high-temperature and high-risk environments such as charging sites, where it is particularly important to accurately identify the boundary between the fire source and the hot air flow.

[0004] Although the prior art can identify the position of the fire source, it still has obvious deficiencies in the accurate identification of the type of fire source. Different types of fire sources directly affect the development and implementation of fire extinguishing strategies. For example, lithium battery fires and electrical equipment fires have significant differences in heat release, smoke characteristics, and flame performance during combustion, but existing technologies usually only use temperature or flame detection to identify the position of the fire source, and cannot effectively distinguish between different types of fire sources. This makes it difficult to quickly determine the nature of the fire source when a fire occurs, thereby affecting the timeliness and targeting of the fire extinguishing strategy.

[0005] The above information disclosed in the background section is only used to enhance the understanding of the background of the present disclosure, and therefore it can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The purpose of the present application is to provide a fire source recognition and positioning method and system based on infrared images and visible light images to solve the problems raised in the background technology.

[0007] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0008] A fire source recognition and positioning method based on infrared images and visible light images, the specific steps comprising:

[0009] Step 1: Monitor the fire occurrence in the charging place through the fire alarm system, if the fire alarm system sends an alarm signal, continuously acquire the infrared image and the visible light image of the charging place, and one-to-one map each pixel point of the infrared image and the visible light image of the charging place;

[0010] Step 2: Mark the high-temperature pixel points in the thermal imaging image, and mark the high-temperature area; use Lucas-Kanade method to calculate the horizontal light flow and vertical light flow of each pixel point in the high-temperature area; map the high-temperature area to the corresponding current visible light image, calculate the local contrast of each pixel point in the high-temperature area, calculate the thermal airflow recognition coefficient according to the local contrast, horizontal light flow and vertical light flow of each pixel point in the high-temperature area, and divide the high-temperature fire area of the infrared image into a thermal airflow area and a fire source area by the thermal airflow recognition coefficient; and map the division result to the visible light image at the same time;

[0011] Step 3: Acquire the previous visible light image, analyze the brightness value of the fire source area of the current visible light image and the corresponding position of the previous visible light image to obtain the flicker coefficient of the fire source area; according to the size of the flicker coefficient of the fire source area, the fire source flicker level is divided, and the fire source flicker level includes flicker 1 level, flicker 2 level and flicker 3 level;

[0012] Step 4: According to the flicker level, color feature of the fire source area and color feature of the thermal airflow area of the current visible light image, the fire source is divided into lithium battery fire source, electrical equipment fire source and other fire source;

[0013] Step 5: Acquire the camera parameters, and position the fire source according to the camera parameters and the position coordinates of the fire source.

[0014] Further, the fire alarm system comprises a smoke alarm, a temperature sensing alarm and a flame alarm; if the smoke alarm, the temperature sensing alarm and the flame alarm send alarm signals at the same time, the fire alarm system sends a fire alarm signal;

[0015] The pixel points of the infrared image and the visible light image are coordinate labeled, the lowest pixel is the first row, and the leftmost pixel column is the first column. The pixel points of the infrared image and the visible light image are one-to-one mapped through coordinates, so that each pixel point has a unique coordinate value in the same image, and each pixel point in the infrared image has a pixel point corresponding to it in the visible light image.

[0016] Further, a preset high-temperature pixel identification threshold is set, and a pixel point in the current infrared image whose pixel value is higher than the high-temperature pixel identification threshold is marked as a high-temperature pixel point, and a region formed by the high-temperature pixel points is referred to as a high-temperature region;

[0017] The horizontal light flow and the vertical light flow are calculated; the specific logic is that the horizontal gradient and the vertical gradient of each pixel point in the high-temperature region are obtained, a spatial gradient matrix is formed by the horizontal gradient and the vertical gradient, a time gradient matrix is calculated according to the pixel value of the high-temperature region in the current infrared image and the pixel value of the corresponding position of the high-temperature region in the previous infrared image, and the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region are calculated according to the spatial gradient matrix and the time gradient matrix;

[0018] The specific logic for calculating the horizontal gradient and the vertical gradient is that:

[0019]

[0020]

[0021] wherein, is the horizontal gradient of the i-th pixel point in the high-temperature region of the current infrared image, is the vertical gradient of the i-th pixel point in the high-temperature region of the current infrared image; is the pixel value of the i-th pixel point in the high-temperature region of the current infrared image, , is a coordinate parameter, is an index of the pixel point in the high-temperature region; The spatial gradient matrix is represented as:

[0022]

[0023]

[0024] wherein, is the spatial gradient matrix of the i-th pixel point in the high-temperature region of the current infrared image; The time gradient matrix is represented as:

[0025]

[0026]

[0027]

[0028] wherein, is the time gradient matrix of the i-th pixel point in the high-temperature region of the current infrared image, is the time gradient matrix of the i-th pixel point in the high-temperature region of the current infrared image, is the time gradient matrix of the i-th pixel point in the high-temperature region of the current infrared image, ​​​​a time gradient of the pixel point, a pixel value of the pixel point corresponding to the high-temperature region of the current infrared image,

[0029] Since the analysis object is a single pixel point, the time gradient matrix is one-dimensional, and since the time interval of the compared infrared images is only one frame, the difference between the pixel points is directly expressed as the time gradient.

[0030] The specific formula for calculating the horizontal light flow and the vertical light flow is:

[0031]

[0032] wherein, a horizontal light flow of the pixel point corresponding to the high-temperature region of the current infrared image, a vertical light flow of the pixel point corresponding to the high-temperature region of the current infrared image,

[0033] The specific formula for calculating the local contrast is:

[0034]

[0035] wherein, a local contrast of the pixel point corresponding to the high-temperature region of the current visible light image, a brightness of the pixel point corresponding to the high-temperature region of the current visible light image, an average brightness of all the pixel points corresponding to the high-temperature region of the current visible light image.

[0036] The specific formula for calculating the hot airflow recognition coefficient is:

[0037]

[0038] wherein, a hot airflow recognition coefficient of the pixel point corresponding to the high-temperature region of the current infrared image, a local contrast of the pixel point corresponding to the high-temperature region of the current visible light image, a horizontal light flow of the pixel point corresponding to the high-temperature region of the current infrared image, a vertical light flow of the pixel point corresponding to the high-temperature region of the current infrared image.

[0039] ​​​​​​​​​Further, the specific logic for dividing the high-temperature fire area into the hot air flow area and the fire source area according to the hot air flow identification coefficient is as follows: a hot air flow identification threshold is preset, the hot air flow identification coefficient is compared with the hot air flow identification threshold; the area with the hot air flow identification coefficient greater than the hot air flow identification threshold is divided into the hot air flow area; and the area with the hot air flow identification coefficient less than or equal to the hot air flow identification threshold is divided into the fire source area.

[0040] Further, the specific formula for calculating the flicker coefficient is as follows:

[0041] The specific formula for calculating the flicker coefficient is as follows:

[0042]

[0043] wherein, is the flicker coefficient of the fire source area; is the brightness of the i th pixel point of the fire source area in the current visible light image;

[0044] is the brightness of the i th pixel point of the fire source area in the previous visible light image corresponding to the current visible light image; is the total number of pixel points of the fire source area; is the index of the pixel point of the fire source area in the current visible light image; the specific logic for dividing the flicker level is as follows: a flicker threshold and are preset; if , the fire source flicker level is flicker level 1; if , the fire source flicker level is flicker level 2; and if , the fire source flicker level is flicker level 3.

[0045] Further, the color feature of each pixel point is the R, G and B value;

[0046] If the fire source flicker level of the fire source area is flicker level 3, the corresponding fire source is judged to be a lithium battery fire source; if the fire source level of the fire source area is flicker level 2 and the following determination condition is met, the corresponding fire source is a lithium battery fire source.

[0047] The determination condition is that the pixel points of the fire source area all satisfy:

[0048]

[0049] The color feature of the pixel point is the color of blue-violet flame in this interval;

[0050] and more than half of the pixel points of the hot air flow area satisfy:

[0051] ​​

[0052] The pixel color feature in this interval is the color of black smoke;

[0053] If the flicker level of the fire source is flicker level 2, and the above determination condition is not met, the fire source is an electrical equipment fire source;

[0054] If the flicker level of the fire source is flicker level 1, it is determined that the fire source is other fire sources.

[0055] Further, the camera parameters include camera intrinsic parameters and camera extrinsic parameters; the camera intrinsic parameters include horizontal focal length, vertical focal length and image center position; the camera extrinsic parameters include a rotation matrix and a translation vector of the camera;

[0056] A camera matrix is formed according to the intrinsic parameters of the camera, and is represented as:

[0057]

[0058] Wherein, is the camera matrix, is the horizontal focal length of the camera, is the vertical focal length of the camera, is the coordinate of the image center;

[0059] The specific formula for positioning the fire source is:

[0060]

[0061] Wherein, , is the fire source coordinate in the visible light image, , , is the fire source coordinate in space; is the rotation matrix, which is a 3*3 matrix, is the translation vector, which is a 3*1 vector.

[0062] The application further provides a fire source identification and positioning system based on infrared images and visible light images, which is used for realizing the fire source identification and positioning method based on infrared images and visible light images, and specifically comprises:

[0063] An image acquisition module is used for monitoring fire occurrence in the charging place through a fire alarm system, and if an alarm signal is sent by the fire alarm system, infrared images and visible light images of the charging place are continuously acquired, and each pixel point of the infrared images and the visible light images of the charging place is one-to-one mapped;

[0064] A region identification module is configured to mark high-temperature pixel points in the thermal imaging image and mark a high-temperature region; the Lucas-Kanade method is used to calculate the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region; the high-temperature region is mapped to a corresponding current visible light image, the local contrast of each pixel point in the high-temperature region is calculated, the thermal airflow identification coefficient is calculated according to the local contrast, the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region, the high-temperature fire region of the infrared image is divided into a thermal airflow region and a fire source region by the thermal airflow identification coefficient, and the division result is mapped to the visible light image at the same time;

[0065] A flicker analysis module is configured to obtain a previous frame of visible light image, analyze the brightness values of the fire source region of the current visible light image and the corresponding positions of the previous frame of visible light image to obtain the flicker coefficient of the fire source region; the fire source flicker level is divided according to the size of the flicker coefficient of the fire source region, and the fire source flicker level includes flicker level 1, flicker level 2 and flicker level 3;

[0066] A fire source identification module is configured to divide the fire source into a lithium battery fire source, an electrical equipment fire source and other fire sources according to the flicker level, the color feature of the fire source region of the current visible light image and the color feature of the thermal airflow region;

[0067] A fire source positioning module is configured to obtain camera parameters and position the fire source according to the camera parameters and the position coordinates of the fire source.

[0068] Compared with the prior art, the present application has the following advantages:

[0069] The present application accurately calibrates the high-temperature region and performs thermal airflow analysis by combining the multi-source information of the infrared image and the visible light image, effectively avoiding the recognition interference caused by the similar temperature of the thermal airflow and the fire source. This innovation not only ensures the accurate identification of the fire source position, but also accurately distinguishes the fire source from the thermal airflow, improves the accuracy of fire source positioning, and avoids the misjudgment and omission problems in the traditional temperature marking method.

[0070] The present application also introduces a fire source type recognition technology based on the fire source flicker coefficient and the color feature. By analyzing the flicker level and the color feature of the fire source region, different types of fire sources such as lithium battery fire sources and electrical equipment fire sources can be clearly distinguished. The present application can accurately identify the type of fire source, help to develop more professional and targeted fire extinguishing strategies, and greatly improve the efficiency and safety of fire extinguishing work. BRIEF DESCRIPTION OF DRAWINGS

[0071] Figure 1 The figure is a schematic diagram of the overall method of the present application;

[0072] Figure 2 The figure is a data fitting diagram of the thermal airflow identification coefficient and the local contrast;

[0073] Figure 3 Data fitting graph of heat flow identification coefficient and horizontal light flux;

[0074] Figure 4 Data fitting graph of heat flow identification coefficient and vertical light flux;

[0075] Figure 5 Graph of heat flow identification coefficient changing with local contrast;

[0076] Figure 6 Graph of heat flow identification coefficient changing with horizontal light flux;

[0077] Figure 7 Graph of heat flow identification coefficient changing with vertical light flux;

[0078] Figure 8 The overall system structure diagram of the present application. DETAILED DESCRIPTION

[0079] In order to make the objects, technical solutions and advantages of the present application clearer and more comprehensible, the present application will be further described in detail below in combination with specific embodiments.

[0080] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present application should be understood as the general meaning understood by those skilled in the art to which the present application belongs. The terms "first", "second" and similar terms used in the present application do not represent any order, quantity or importance, but are only used to distinguish different components. The terms "include" or "contain" and similar terms mean that the elements or objects before the terms cover the elements or objects listed after the terms and their equivalents, without excluding other elements or objects. The terms "connect" or "connected" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "up", "down", "left", "right" and the like are only used to represent relative positional relationships, and when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0081] EMBODIMENT:

[0082] Please refer to Figures 1-7 The present application provides a technical solution:

[0083] Step 1: Monitor the fire occurrence of the charging place through the fire alarm system, if the fire alarm system sends an alarm signal, continuously acquire the infrared image and the visible light image of the charging place, and one-to-one map each pixel point of the infrared image and the visible light image of the charging place;

[0084] The fire alarm system comprises a smoke alarm, a temperature sensing alarm and a flame alarm; if the smoke alarm, the temperature sensing alarm and the flame alarm simultaneously send alarm signals, the fire alarm system sends a fire alarm signal;

[0085] The pixel points of the infrared image and the visible light image are marked with coordinates, the pixel in the lowermost side is the first row, the pixel in the leftmost side is the first column, the pixel points of the infrared image and the visible light image are one-to-one mapped through coordinates, so that each pixel point has a unique coordinate value in the same image, and each pixel point in the infrared image has a corresponding pixel point in the visible light image. Due to the one-to-one mapping relationship between the two images, the pixel points of the two images adopt the same index

[0086] Step 2: Mark the high-temperature pixel points in the thermal imaging image, and mark the high-temperature region; calculate the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region using the Lucas-Kanade method; map the high-temperature region to the corresponding current visible light image, calculate the local contrast of each pixel point in the high-temperature region, calculate the thermal airflow recognition coefficient according to the local contrast, the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region, divide the high-temperature fire region of the infrared image into a thermal airflow region and a fire source region through the thermal airflow recognition coefficient; and map the division result to the visible light image at the same time;

[0087] A preset high-temperature pixel recognition threshold is used to mark the pixel points in the current infrared image as high-temperature pixel points if the pixel value of the pixel points is higher than the high-temperature pixel recognition threshold, and the region formed by the high-temperature pixel points is called a high-temperature region;

[0088] The horizontal light flow and the vertical light flow are calculated; the specific logic is as follows: the horizontal gradient and the vertical gradient of each pixel point in the high-temperature region are obtained, the horizontal gradient and the vertical gradient are used to form a spatial gradient matrix, the time gradient matrix is calculated through the pixel value of the high-temperature region of the current infrared image and the pixel value of the corresponding position in the previous infrared image, and the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region are calculated according to the spatial gradient matrix and the time gradient matrix;

[0089] The specific logic for calculating the horizontal gradient and the vertical gradient is as follows:

[0090]

[0091]

[0092] wherein, is the horizontal gradient of the i-th pixel point in the high-temperature region of the current infrared image, is the vertical gradient of the i-th pixel point in the high-temperature region of the current infrared image, is the horizontal gradient of the i-th pixel point in the high-temperature region of the current infrared image, The vertical gradient of each pixel; The first high temperature area in the current infrared image The pixel value of each pixel, 、 is the coordinate parameter, is the index of the pixel in the high temperature area;

[0093] The spatial gradient matrix is ​​expressed as:

[0094]

[0095] in, The first high temperature area in the current infrared image The spatial gradient matrix of pixels;

[0096] The time gradient matrix is ​​expressed as:

[0097]

[0098]

[0099] in, The first high temperature area in the current infrared image The temporal gradient matrix of pixels, The first high temperature area in the current infrared image The temporal gradient of pixels, The previous infrared image corresponds to the high temperature area of ​​the current infrared image. The pixel value of each pixel;

[0100] Since the analysis object is a single pixel, the time gradient matrix is ​​one-dimensional. Since the time interval of the compared infrared images is only one frame, the time gradient is directly expressed by the difference between the pixels.

[0101] The specific formulas for calculating horizontal and vertical light flux are:

[0102]

[0103] in, The first high temperature area in the current infrared image The horizontal optical flow of pixels, The first high temperature area in the current infrared image The vertical optical flow of each pixel;

[0104] The specific formula for calculating local contrast is:

[0105]

[0106] in, the local contrast of the pixel point corresponding to the high-temperature region of the current visible light image; the local contrast of the pixel point corresponding to the high-temperature region of the current visible light image; the brightness of the pixel point corresponding to the high-temperature region of the current visible light image, the brightness of the pixel point corresponding to the high-temperature region of the current visible light image, the average brightness of all pixel points corresponding to the high-temperature region of the current visible light image;

[0107] The specific formula for calculating the hot air flow recognition coefficient is as follows:

[0108]

[0109] wherein, the hot air flow recognition coefficient of the pixel point corresponding to the high-temperature region of the current infrared image; the local contrast of the pixel point corresponding to the high-temperature region of the current visible light image; the local contrast of the pixel point corresponding to the high-temperature region of the current visible light image; the horizontal light flow of the pixel point corresponding to the high-temperature region of the current infrared image, the horizontal light flow of the pixel point corresponding to the high-temperature region of the current infrared image; the vertical light flow of the pixel point corresponding to the high-temperature region of the current infrared image; Generally, a fire source will cause a hot air flow around the environment; the generated hot air flow has a very high temperature; on an infrared image, it is difficult to identify the hot air flow and the fire source because the temperatures of the hot air flow and the fire source are both very high; the high temperature generated by the fire source will cause refraction of the surrounding air, thereby changing the propagation path of the light. Specifically, the hot air flow on the image will show unclear and appear a wave similar to "water surface reflection";

[0110] The hot air flow recognition coefficient is obtained by coupling the horizontal light flow, the vertical light flow and the local contrast; the horizontal light flow and the vertical light flow reflect the light and shadow characteristics of the image; the local contrast reflects the blur degree of the local image; the hot air flow recognition coefficient reflects the degree of blur and light and shadow effect of the pixel point in the image caused by the hot air flow, the greater the value, the more significant the image blur caused by the hot air flow, the more significant the light and shadow effect caused by the hot air flow; then the probability of the pixel point being the hot air flow generated by the fire source is greater.

[0111] Table 1 shows the change of the hot air flow recognition coefficient with the local contrast, the horizontal light flow and the vertical light flow.

[0112] Table 1 shows the change of the hot air flow recognition coefficient with the local contrast, the horizontal light flow and the vertical light flow.

[0113] Table 1 shows the change of the hot air flow recognition coefficient with the local contrast, the horizontal light flow and the vertical light flow.

[0114]

[0115] For example, Figures 2-7 ​From the 1st to the 29th data, as the local contrast in the high temperature area decreases, the thermal identification coefficient increases significantly. The thermal identification coefficient and the local contrast show an obvious inverse correlation, as shown in Figure 2. Figure 5 As shown in the figure, this anti-correlation is closer to an inverse proportional relationship in form. The local contrast reflects the blur of the image. The larger its value, the clearer the image, and the smaller the probability that the pixel is the thermal airflow generated by the fire source. As the horizontal light flow and vertical light flow increase, the thermal airflow identification coefficient also increases. The horizontal light flow and the thermal airflow identification coefficient are obviously correlated, and the vertical light flow and the thermal airflow identification coefficient are also obviously correlated, as shown in Figure 2. Figure 6 、 Figure 7 This positive correlation is closer to a proportional relationship. The horizontal light flow and the vertical light flow jointly reflect the light and shadow effect of the pixel point. The larger the value, the more significant the light and shadow effect of the pixel point, and the greater the probability that the pixel point is the hot air flow generated by the fire source.

[0116] is the integrated optical flow, The comprehensive contribution of horizontal light flow and vertical light flow to the light and shadow effect is comprehensively expressed in the form of ; the larger the value, the greater the comprehensive contribution; and the size of the local contrast is inversely correlated with the image blur caused by the thermal airflow; therefore, through to calculate the thermal identification coefficient, where This is to prevent local contrast from being negative.

[0117] The specific logic for dividing the high-temperature fire area into the thermal airflow area and the fire source area by the thermal airflow identification coefficient is as follows: presetting the thermal airflow identification threshold, comparing the thermal airflow identification coefficient with the thermal airflow identification threshold; dividing the area where the thermal airflow identification coefficient is greater than the thermal airflow identification threshold into the thermal airflow area; and dividing the area where the thermal airflow identification coefficient is less than or equal to the thermal airflow identification threshold into the fire source area.

[0118] Step 3: Obtain the previous visible light image, analyze the brightness values ​​of the fire source area in the current visible light image and the corresponding position in the previous visible light image to obtain the flicker coefficient of the fire source area; and classify the fire source flicker level according to the flicker coefficient of the fire source area, wherein the fire source flicker level includes flicker level 1, flicker level 2, and flicker level 3.

[0119] The specific formula used to calculate the flicker coefficient is:

[0120]

[0121] in, is the flicker coefficient of the fire source area; The first The brightness of each pixel;

[0122] The first visible light image of the previous frame corresponds to the fire source area of ​​the current visible light image. The brightness of each pixel; is the total number of pixels in the fire source area; The index of the pixel point in the fire source area of ​​the current visible light image. Fire sources in charging locations may be affected by the power system of the battery chemical reaction. This can cause the fire source in the charging location to produce flashes and arcs due to reasons such as lithium battery explosion or short circuit. Flashes and arcs can cause the brightness of the fire source to change significantly in an instant, resulting in the fire source appearing to flicker momentarily.

[0123] The flicker coefficient reflects the flicker characteristics of the fire source and the surrounding thermal area by comparing the brightness of the two frames before and after. The larger the value, the more obvious the flicker.

[0124] The specific logic for dividing the flicker levels is as follows: preset flicker threshold and ;like , then the fire source flicker level is flicker level 1; flicker level 1 means that the brightness change between the previous and next two frames is small, usually no flicker is involved; if , then the fire source flicker level is flicker 2; flicker 2 means that the brightness change between the two frames is large; it means that the fire source has obvious flickering; this flicker is usually caused by power system faults such as flicker; if , the fire source flicker level is flicker 3. Flicker 3 indicates a huge change in brightness between the previous and next frames. This huge brightness change cannot be caused by flicker caused by electrical faults. It is usually related to the flash caused by lithium battery explosion.

[0125] Step 4: Based on the flicker level and color characteristics of the fire source area in the current visible light image and the color characteristics of the thermal area, the fire sources are classified into lithium battery fire, electrical equipment fire, and other fire sources.

[0126] The color feature takes the R, G, and B values ​​of each pixel;

[0127] If the fire source flash level in the fire source area is flashing level 3, the corresponding fire source is judged to be a lithium battery fire source; if the fire source level in the fire source area is flashing level 2 and meets the following judgment conditions, the corresponding fire source is a lithium battery fire source;

[0128] The judgment condition is: all pixels in the fire source area meet the following conditions:

[0129]

[0130] The pixel color feature in this range appears as a blue-purple flame color;

[0131] And more than half of the pixels in the thermal area meet the following requirements:

[0132]

[0133] The pixel color feature in this range appears as black smoke;

[0134] Since lithium battery fires are always accompanied by electrical faults, the flash level of lithium battery fires must be Flashing Level 3 or Flashing Level 2. Lithium battery fires are accompanied by chemical reactions, and the flames are blue-purple and accompanied by a large amount of black smoke. If the lithium battery fire reaction is complete, the flame will turn normal orange-red and no black smoke will be produced. Although the location is the same, the fire source is not considered a lithium battery fire at this time.

[0135] If the fire source flash level is flash level 2 and does not meet the above judgment conditions, the fire source is an electrical equipment fire;

[0136] If the fire source flash level is flashing level 1, it is judged that the fire source is other fire sources.

[0137] Step 5: Obtain camera parameters and locate the fire source based on the camera parameters and the location coordinates of the fire source.

[0138] The camera parameters include camera intrinsic parameters and camera extrinsic parameters; the camera intrinsic parameters include horizontal focal length, vertical focal length and image center position; the camera extrinsic parameters include camera rotation matrix and translation vector;

[0139] The horizontal focal length and vertical focal length are obtained by consulting the camera parameters; the image center position is the coordinate of the center of the visible light image;

[0140] The rotation matrix and translation vector can be calibrated using a checkerboard calibration plate and Zhang Zhengyou camera calibration method; specifically:

[0141] The checkerboard calibration plate is composed of a series of square grids of known sizes. The side length of each grid and the geometric shape of the calibration plate are determined in advance; and the coordinates of the corner points of each positive direction grid are calibrated in the spatial coordinate system. The corner points are the four intersection points of each square in the checkerboard. Use a camera to take multiple checkerboard images at different angles, process the taken images, and extract the checkerboard corner points in each image. The corner point detection algorithms in libraries such as OpenCV can be used to automatically identify the corner points of the checkerboard. Using the known three-dimensional information of the calibration plate and the position of the corner points in the image, Zhang Zhengyou's camera calibration method is used to solve the external parameters of the camera; the rotation matrix and translation vector can be calibrated using the checkerboard calibration plate and Zhang Zhengyou's camera calibration method. This is a common technical means used by those skilled in the art and will not be elaborated here.

[0142] The camera matrix is ​​constructed according to the intrinsic parameters of the camera, which is expressed as:

[0143]

[0144] wherein, is the camera matrix, is the horizontal focal length of the camera, is the vertical focal length of the camera, is the coordinate of the image center;

[0145] The specific formula for locating the fire source is:

[0146]

[0147] wherein, , is the fire source coordinate in the visible light image, , , is the fire source coordinate in the space; is a rotation matrix, which is a 3*3 matrix, is a translation vector, which is a 3*1 vector;

[0148] Since step 1 has one-to-one mapping of the coordinates of the visible light image and the infrared image, , is both the coordinate parameter of the visible light image and the coordinate parameter of the infrared image.

[0149] Referring to Figure 8 , the present application further provides a fire source identification and positioning system based on infrared images and visible light images, which is used to realize the fire source identification and positioning method based on infrared images and visible light images, and specifically comprises:

[0150] An image acquisition module is used to monitor the fire occurrence in the charging place through the fire alarm system, and if the fire alarm system sends an alarm signal, the infrared image and the visible light image of the charging place are continuously acquired, and each pixel point of the infrared image and the visible light image of the charging place is one-to-one mapped.

[0151] A region identification module is used to mark the high-temperature pixel points in the thermal imaging image and mark the high-temperature region, calculate the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region using the Lucas-Kanade method, map the high-temperature region to the corresponding current visible light image, calculate the local contrast of each pixel point in the high-temperature region, calculate the thermal airflow identification coefficient according to the local contrast, the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region, divide the high-temperature fire region of the infrared image into a thermal airflow region and a fire source region through the thermal airflow identification coefficient, and map the division result to the visible light image synchronously.

[0152] The flicker analysis module is configured to obtain a previous visible light image, analyze a luminance value of a fire source region in the current visible light image and a corresponding position of the previous visible light image to obtain a flicker coefficient of the fire source region, and divide the fire source flicker level according to the size of the flicker coefficient of the fire source region, wherein the fire source flicker level includes flicker level 1, flicker level 2 and flicker level 3.

[0153] The fire source recognition module is configured to divide the fire source into a lithium battery fire source, an electrical equipment fire source and other fire sources according to the flicker level, the color feature of the fire source region and the color feature of the hot air flow region in the current visible light image.

[0154] The fire source positioning module is configured to obtain camera parameters, and position the fire source according to the camera parameters and the position coordinates of the fire source.

[0155] The above formulas are all dimensionless values, and the formulas are obtained by collecting a large amount of data to simulate a formula of the nearest real situation, and the preset parameters in the formula are set by a person skilled in the art according to the actual situation.

[0156] The above embodiments can be realized wholly or partially by software, hardware, firmware or any combination thereof. When realized by software, the above embodiments can be realized in the form of a computer program product wholly or partially. Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized by hardware or software methods depends on the specific application and design constraints of the technical solutions.

[0157] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, which can be located in one place or distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0158] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can not easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A fire source recognition and positioning method based on infrared images and visible light images, characterized in that, The specific steps include: Step 1: Monitor the occurrence of a fire in the charging place through a fire alarm system, and if the fire alarm system issues an alarm signal, real-time infrared images and visible light images of the charging place are acquired, and each pixel point of the infrared images and the visible light images of the charging place is one-to-one mapped; Step 2: Mark the high-temperature pixel points in the current thermal imaging image and mark the high-temperature region; use the Lucas-Kanade method to calculate the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region; map the high-temperature region to the corresponding current visible light image, calculate the local contrast of each pixel point in the high-temperature region, calculate the thermal airflow recognition coefficient according to the local contrast, the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region, and divide the high-temperature fire region of the infrared image into a thermal airflow region and a fire source region by the thermal airflow recognition coefficient; and map the division result to the visible light image at the same time; Step 3: Acquire the previous visible light image, analyze the brightness values of the corresponding positions of the previous visible light image and the fire source region of the current visible light image to obtain the flicker coefficient of the fire source region; divide the fire source flicker level according to the size of the flicker coefficient of the fire source region, and the fire source flicker level includes flicker level 1, flicker level 2 and flicker level 3; Step 4: According to the flicker level, color feature of the fire source region and color feature of the thermal airflow region of the current visible light image, the fire source is divided into a lithium battery fire source, an electrical equipment fire source and other fire sources; Step 5: Acquire the camera parameters, and position the fire source according to the camera parameters and the position coordinates of the fire source region; The specific formula for calculating the thermal airflow recognition coefficient is: in, The first high temperature area in the current infrared image Thermal identification coefficient of each pixel; The first high temperature area in the current visible light image The local contrast of each pixel; The first high temperature area in the current infrared image The horizontal optical flow of pixels, The first high temperature area in the current infrared image The vertical optical flow of each pixel.

2. The fire source recognition and positioning method based on infrared image and visible light image according to claim 1, characterized in that: The fire alarm system includes a smoke alarm, a temperature sensing alarm and a flame alarm; if the smoke alarm, the temperature sensing alarm and the flame alarm issue alarm signals at the same time, the fire alarm system issues a fire alarm signal; The pixel points of the infrared image and the visible light image are coordinate labeled, the lowest pixel is the first row, the leftmost pixel column is the first column, and the pixel points of the infrared image and the visible light image are one-to-one mapped through coordinates, so that each pixel point has a unique coordinate value in the same image, and each pixel point in the infrared image has a corresponding pixel point in the visible light image.

3. The fire source recognition and positioning method based on infrared image and visible light image according to claim 1, characterized in that: A preset high-temperature pixel recognition threshold is set, and the pixel points in the current infrared image whose pixel values are higher than the high-temperature pixel recognition threshold are marked as high-temperature pixel points, and the region formed by the high-temperature pixel points is called a high-temperature region; The horizontal light flow and the vertical light flow are calculated; the specific logic is as follows: the horizontal gradient and the vertical gradient of each pixel point in the high-temperature region are acquired, the horizontal gradient and the vertical gradient form a spatial gradient matrix, the pixel value of the current infrared image high-temperature region and the pixel value of the corresponding position in the previous infrared image are used to calculate a time gradient matrix, and the horizontal light flow and the vertical light flow of each pixel point in the high-temperature region are calculated according to the spatial gradient matrix and the time gradient matrix; The specific logic for calculating the horizontal gradient and the vertical gradient is as follows: wherein, is a horizontal gradient of a pixel point in a high temperature region of a current infrared image, is a vertical gradient of a pixel point in a high temperature region of a current infrared image, is a pixel value of a pixel point in a high temperature region of a current infrared image, , is a coordinate parameter, is an index of a pixel point in a high temperature region.​​​ The spatial gradient matrix is represented as: wherein, is a spatial gradient matrix of the current infrared image high temperature region pixel point; The time gradient matrix is represented as: wherein, is a time gradient matrix of the current infrared image high temperature region first pixel point, is a time gradient of the current infrared image high temperature region first pixel point, is a pixel value of the previous infrared image corresponding to the current infrared image high temperature region first pixel point. Since the analysis object is a single pixel point, the time gradient matrix is one-dimensional, and since the time interval of the compared infrared images is only one frame, the difference between the pixel points is directly expressed as the time gradient; The specific formula for calculating the horizontal light flow and the vertical light flow is: wherein, is the horizontal light flux of the i-th pixel point in the high temperature region of the current infrared image, is the vertical light flux of the i-th pixel point in the high temperature region of the current infrared image, is the horizontal light flux of the i-th pixel point in the high temperature region of the current infrared image, is the vertical light flux of the i-th pixel point in the high temperature region of the current infrared image, The specific formula for calculating the local contrast is: wherein, is the local contrast of the current visible light image high temperature region i-th pixel point; is the brightness of the current visible light image high temperature region i-th pixel point, is the average brightness of all pixel points of the current visible light image high temperature region.​​ 4. The fire source recognition and positioning method based on infrared image and visible light image according to claim 1, characterized in that: The specific logic for dividing the high-temperature fire area into a hot air flow area and a fire source area through the hot air flow identification coefficient is: preset a hot air flow identification threshold, compare the hot air flow identification coefficient with the hot air flow identification threshold; divide the area with the hot air flow identification coefficient greater than the hot air flow identification threshold into the hot air flow area; divide the area with the hot air flow identification coefficient less than or equal to the hot air flow identification threshold into the fire source area.

5. The fire source recognition and positioning method based on infrared image and visible light image according to claim 1, characterized in that: The specific formula for calculating the flicker coefficient is: wherein, is a flicker coefficient for the fire source region; is a luminance of the i-th pixel point of the fire source region of the current visible light image; is a luminance of the i-th pixel point of the fire source region of the previous visible light image corresponding to the current visible light image; is a luminance of the i-th pixel point of the fire source region of the previous visible light image corresponding to the current visible light image; is a luminance of the i-th pixel point of the fire source region of the previous visible light image corresponding to the current visible light image; is a total number of pixel points of the fire source region; is an index of the i-th pixel point of the fire source region of the current visible light image; The specific logic for dividing the flicker level is: preset flicker threshold and ; if , the fire source flicker level is flicker level 1; if , the fire source flicker level is flicker level 2; if , the fire source flicker level is flicker level 3.

6. The fire source recognition and positioning method based on infrared image and visible light image according to claim 1, characterized in that: The color feature is the R, G, and B values of each pixel point; If the fire source flicker level of the fire source area is flicker level 3, the corresponding fire source is judged to be a lithium battery fire source; if the fire source level of the fire source area is flicker level 2, and the following determination condition is met, the corresponding fire source is a lithium battery fire source; The determination condition is that the pixel points of the fire source area all satisfy: The color feature of the pixel point in this interval is blue-violet color; More than half of the pixel points in the hot air flow area satisfy: The color feature of the pixel point in this interval is black smoke color; If the fire source flicker level is flicker level 2, and the above determination condition is not met, the fire source is an electrical equipment fire source; If the fire source flicker level is flicker level 1, the fire source is judged to be another fire source.

7. The fire source recognition and positioning method based on infrared image and visible light image according to claim 1, characterized in that: The camera parameters include camera intrinsic parameters and camera extrinsic parameters; the camera intrinsic parameters include horizontal focal length, vertical focal length, and image center position; the camera extrinsic parameters include the rotation matrix and translation vector of the camera; The camera matrix is formed according to the intrinsic parameters of the camera, and is represented as: wherein, is the camera matrix, is the horizontal focal length of the camera, is the vertical focal length of the camera, is the coordinate of the image center; The specific formula for positioning the fire source is: wherein , is the fire source coordinate in the visible light image, , , is the fire source coordinate in space; is a rotation matrix, which is a 3*3 matrix, is a translation vector, which is a 3*1 vector.

8. A fire source identification and location system based on infrared and visible light images, characterized by: The system is used to implement the fire source identification and positioning method based on infrared images and visible light images according to any one of claims 1-7, and specifically includes: An image acquisition module is configured to monitor the occurrence of a fire in a charging place through a fire alarm system, and if an alarm signal is sent by the fire alarm system, continuously acquire infrared images and visible light images of the charging place, and one-to-one map each pixel point of the infrared images and the visible light images of the charging place; A region identification module is configured to mark high-temperature pixel points in a thermal image and mark a high-temperature area, calculate the horizontal light flow and the vertical light flow of each pixel point in the high-temperature area using the Lucas-Kanade method, map the high-temperature area to the corresponding current visible light image, calculate the local contrast of each pixel point in the high-temperature area, calculate a hot air flow identification coefficient according to the local contrast, the horizontal light flow, and the vertical light flow of each pixel point in the high-temperature area, divide the high-temperature fire area of the infrared image into a hot air flow area and a fire source area through the hot air flow identification coefficient, and map the division result to the visible light image synchronously; The flicker analysis module is configured to acquire a previous visible light image, analyze a luminance value of a fire source region in the current visible light image and a corresponding position of the previous visible light image to obtain a flicker coefficient of the fire source region, and divide a fire source flicker level according to the flicker coefficient of the fire source region, wherein the fire source flicker level includes a flicker level 1, a flicker level 2 and a flicker level 3. The fire source recognition module is configured to divide the fire source into a lithium battery fire source, an electrical equipment fire source and other fire sources according to the flicker level, color features of the fire source region in the current visible light image and color features of the hot air flow region. The fire source positioning module is configured to acquire camera parameters and position the fire source according to the camera parameters and position coordinates of the fire source.

Images