A method and system for identifying mountain fires at night

Abstract

The present invention discloses a method and system for identifying wildfires at night, and relates to the technical field of wildfire monitoring, comprising the steps of: obtaining a training set of camera nighttime flame interception pictures and a training set of camera nighttime light interception pictures corresponding to each camera; obtaining a flame candidate area in an image to be identified and recording a target shooting camera corresponding to the flame candidate area; determining a distance matching score, a similarity matching score, and a light frequency score; obtaining a flame confidence of the flame candidate area; and determining whether it is a real wildfire area according to the flame confidence. In the scheme of the present invention, the flame confidence is obtained based on full consideration of image color characteristics, appearance time node characteristics, and appearance position characteristics, and the flame candidate area is identified based on the flame confidence, so that wildfire scenes at night can be effectively identified, and the method has a wide range of application scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of mountain fire monitoring, and in particular to a method and system for identifying mountain fires at night. Background Art

[0002] Wildfire is a serious natural disaster with the characteristics of fast speed, suddenness and great destructiveness, which brings great challenges to disaster relief work. Once a wildfire occurs, the fire will spread rapidly and become the biggest threat to forest safety in mountainous areas. In addition, many transmission line corridors are rich in vegetation, and wildfires can also cause serious damage to the power grid system. For example, the fire spreads to the poles and transmission lines, causing the lines to break or burn, causing damage to the power lines; the heat and fire of the wildfire may have a direct impact on the substation and substation equipment, and equipment such as transformers, switchgear and protective devices may be burned, damaged or overheated, causing equipment failure and shutdown; the high temperature and flames caused by the wildfire may cause damage to underground cables. The cable sheath burns or melts, causing the cable to short-circuit, break or poor contact, affecting power transmission and power supply quality. In this context, accurate detection and early warning of mountain fires have become the key to ensuring forest safety and stable operation of the power grid.

[0003] At present, relevant scholars and enterprises have proposed a variety of wildfire detection technologies based on mountainous area images, such as using brightness flicker based on the YUV color model to identify suspected fire areas, or using image recognition algorithms based on convolutional neural networks (CNNs). Some studies have also proposed using deep learning image recognition algorithms based on Transformers. However, these existing methods are mainly aimed at daytime wildfire identification. It is difficult to effectively identify wildfires at night when the pictures are not clear. There are problems such as limited versatility and inaccurate identification of wildfires at night, which makes it difficult to meet the requirements of nighttime wildfire identification.

[0004] In view of this, it is necessary to propose a method and system for identifying wildfires at night to solve or at least partially solve the above technical problems. Summary of the invention

[0005] The main purpose of the present invention is to provide a method and system for identifying wildfires at night, aiming to solve the technical problem that the existing wildfire identification methods are mainly used for wildfire identification during the day and are difficult to accurately identify wildfires at night.

[0006] To achieve the above object, the present invention provides a method for identifying wildfires at night, comprising the steps of:

[0007] S100, by selecting and marking the night label pictures and processing them, obtain the camera night flame interception picture training set Da and the camera night light interception picture training set Hb corresponding to each camera, and the camera night light interception picture training set Hb is the image label interception area with light annotation The camera night flame interception picture training set Da is the image label interception area with flame annotation A collection of images captured by cameras at night, recording the label interception area of ​​each image in the training set Hb The shooting timestamp;

[0008] S200, determining a color space range representing color features of nighttime wildfire pixel values ​​based on at least one camera's nighttime fire interception picture training set Da; obtaining a flame candidate region in the image to be identified by using a color feature matching method based on the color space range , and record the flame candidate area The corresponding target shooting camera;

[0009] S300, based on flame candidate regions Traverse the camera night flame interception picture training set Da corresponding to the target shooting camera to perform flame area matching, and obtain the camera night flame interception picture training set Da that matches the flame candidate area The first K image labels with the largest distance matching score are cut out. To match the flame area set Q;

[0010] Based on flame candidate region The distance between each matching flame area in the matching flame area set Q determines the distance matching score S1;

[0011] S400, the flame candidate area Input into the flame recognition deep convolutional neural network model to obtain the image region representation vector ;

[0012] The image label interception area in the camera night flame interception picture training set Da corresponding to the target shooting camera Input them into the flame recognition deep convolutional neural network model respectively to obtain each cropped area The corresponding image region representation vector ;

[0013] The cosine similarity is used for calculation to obtain the twin similarity matching score, and the largest twin similarity matching score is determined as the similarity matching score S2;

[0014] S500, based on flame candidate regions Traverse the camera night light interception picture training set Hb corresponding to the target shooting camera to match the light area, and obtain the camera night light interception picture training set Hb that matches the flame candidate area The image label clipping area with annotated information that meets the preset distance range threshold as a set of matching light areas V, each matching light area in the matching light area set is marked with a corresponding shooting timestamp;

[0015] Get the flame candidate area based on the shooting timestamp The light frequency score S3 of each matching light area;

[0016] S600, obtaining a flame confidence S of the flame candidate area based on the distance matching score S1, the similarity matching score S2 and the light frequency score S3;

[0017] S700, if the flame confidence S is not lower than the flame recognition threshold, the flame candidate area is determined to be a real wildfire area; if the flame confidence S is lower than the flame recognition threshold, the flame candidate area is determined to be a real light area.

[0018] Furthermore, step S100 specifically includes:

[0019] S101, select and mark the night label image and process it to obtain the image label clipping area with the label information, wherein the label information includes the light label or the flame label, and the image label clipping area with the flame label Combining the flame image dataset Ra, the image label cutout area with light annotation Combined to form the light image dataset Rb;

[0020] S102, determining the image label interception area corresponding to the same camera in the flame image dataset Ra The image collection is the training collection Da of nighttime flame images captured by the camera;

[0021] Determine the image label interception area corresponding to the same camera in the light image dataset Rb The image set is the camera night light interception picture training set Hb, Hb∈Rb, and the label interception area of ​​each image in the camera night light interception picture training set Hb is recorded The shooting timestamp.

[0022] Furthermore, when performing box selection annotation, the annotation area is (x, y, w, h), where x and y represent the horizontal and vertical coordinates of the upper left corner of the wildfire area or the light area, respectively, and w and h represent the width and height of the annotation area, respectively.

[0023] Further, in step S300,

[0024] Using formula

[0025] Calculate and obtain the flame candidate area The i-th image label interception area in the training set Da of night flame interception pictures captured by the camera Distance ,

[0026] in, Represents the flame candidate area The i-th image label interception area in the training set Da of night flame interception pictures captured by the camera The overlap of Represents the flame candidate area The area of Represents the i-th image label interception area in the camera night flame interception picture training set Da The area of represents the overlapping area, and S() represents the area calculation operation;

[0027] Take the top k image label screenshot areas with the largest calculated distance values Constructed to match the flame area set Q;

[0028] Using formula Calculate the distance matching score S1, where k represents the first K matching image label clipping areas with annotated information. .

[0029] Further, in step S400,

[0030] Using formula ,

[0031] Calculate and obtain the similarity matching score S2, where: Represents the flame candidate area The image region representation vector obtained by inputting into the flame recognition deep convolutional neural network model, It represents the screenshot area of ​​the i-th image label with annotation information in the training set Da of the camera night flame interception pictures corresponding to the target shooting camera The image region representation vector obtained by inputting into the flame recognition deep convolutional neural network model, cos() represents the cosine similarity calculation function.

[0032] Further, in step S500,

[0033] Using formula

[0034] Calculate and obtain the flame candidate area The jth image label cropping area in the training set Hb of the camera night light interception picture Light matching distance ,

[0035] in, To meet the jth image label screenshot area with annotation information within the preset distance range, Represents the flame candidate area The jth image label cropping area in the training set Hb of the camera night light interception picture The overlap degree of S(Xrec) represents the flame candidate region The area, Represents the area of ​​the jth image label screenshot region with annotation information within the preset distance range. represents the overlapping area, and S() represents the area calculation operation;

[0036] Using formula Determine the set of matching light areas V with annotation information within the preset distance range, and the image label screenshot area They are marked with the corresponding shooting timestamps, and the total number of matching light areas is |V|. Represents the jth image label screenshot area in the matching light area set V , Screenshot area for the jth image label The corresponding shooting timestamp, L is the preset distance range threshold.

[0037] Furthermore, using the formula Calculate the obtained light frequency score S3, where T is the shooting time node of the flame candidate area, The light image label screenshot area within the j-th preset distance range The corresponding shooting timestamp, is the time normalization coefficient.

[0038] Furthermore, using the formula Calculate and obtain the flame confidence S,

[0039] in , , are the weights of the distance matching score S1, similarity matching score S2, and light frequency score S3, respectively. .

[0040] Furthermore, the method further comprises step S700:

[0041] If the flame candidate area is a real wildfire area, the flame candidate area is updated to the camera night flame interception picture training set Da,

[0042] If the flame candidate area is a real light area, the flame candidate area is updated to the camera night light interception picture training set Hb.

[0043] The present invention also provides a nighttime mountain fire identification system, which includes a processor and a memory connected to each other, and the processor is used to execute the nighttime mountain fire identification method.

[0044] Compared with the prior art, the method for identifying wildfires at night provided by the present invention has the following beneficial effects:

[0045] The present invention provides a method for identifying wildfires at night, which obtains a camera night flame interception picture training set Da and a camera night light interception picture training set Hb corresponding to each camera by selecting and marking the night label picture and processing it, and records each image label interception area in the camera night light interception picture training set Hb. The shooting timestamp; determine the color space range that characterizes the color features of the nighttime wildfire pixel values ​​according to the flame image dataset Ra, and after obtaining the current nighttime shot picture, use the color feature matching method based on the color space range to obtain the flame candidate area in the image to be identified , and record the target shooting camera corresponding to the flame candidate area Xrec; then based on the flame candidate area Traverse the nighttime flame interception picture training set Da corresponding to the target shooting camera to perform flame area matching to obtain multiple matching flame areas, and obtain the distance matching score S1 based on the distance between the flame candidate area and each matching flame area; based on the flame recognition deep convolutional neural network model, the flame candidate area Processing is performed to obtain the image region representation vector Based on the flame recognition deep convolutional neural network model, the image label interception area with annotated information in the camera night flame interception picture training set Da Get the image region representation vector , use cosine similarity to calculate, obtain twin similarity matching score, determine the maximum twin similarity matching score as similarity matching score S2; based on the flame candidate area, traverse the camera night light interception picture training set Hb corresponding to the target shooting camera to match the light area, and obtain the camera night light interception picture set Hb that matches the flame candidate area The image label clipping area with annotated information that meets the preset distance range threshold As the matching light area set V, each matching light area in the matching light area set V is marked with a corresponding shooting timestamp; based on the shooting timestamp, the light frequency score S3 of the flame candidate area and each matching light area V is obtained; finally, the flame confidence S of the flame candidate area is obtained based on the distance matching score S1, the similarity matching score S2 and the light frequency score S3; finally, if the flame confidence S is not lower than the flame recognition threshold, the flame candidate area is determined to be a real wildfire area, and if the flame confidence S is lower than the flame recognition threshold, the flame candidate area is determined to be a real light area. By adopting the method of the present invention, when performing nighttime wildfire identification, the characteristic points of nighttime lights and nighttime wildfires are fully considered. After obtaining the camera nighttime flame interception picture training set Da and the camera nighttime light interception picture training set Hb, the flame candidate area is determined for the current nighttime picture by using a color feature matching method based on the color space range. Based on the image color characteristics, the occurrence time node characteristics and the occurrence position characteristics, the similarity matching score S2, the distance matching score S1 and the light frequency score S3 are fully considered. Finally, the flame confidence S is obtained according to the similarity matching score S2, the distance matching score S1 and the light frequency score S3. The flame candidate area is judged based on the flame confidence S, which can effectively identify the nighttime wildfire scene and significantly reduce the misidentification caused by car lights, street lights, etc., and has a wide range of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0047] Figure 1 The figure is a flow chart of the method for identifying wildfires at night of the present invention.

[0048] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0052] Please refer to Figure 1 The present invention provides a method for identifying mountain fires at night, comprising the steps of:

[0053] S100, by selecting and marking the night label pictures and processing them, obtain the camera night flame interception picture training set Da and the camera night light interception picture training set Hb corresponding to each camera, the camera night light interception picture training set Hb is a set of image label interception areas Ycropb with light annotations, the camera night flame interception picture training set Da is a set of image label interception areas Ycropa with flame annotations, and record the shooting timestamp of each image label interception area Ycropb in the camera night light interception picture training set Hb;

[0054] S200, determining a color space range (R1-R2, g1-g2, b1-b2) representing color features of nighttime wildfire pixel values ​​based on at least one camera's nighttime fire interception picture training set Da; obtaining a flame candidate region in the image to be identified by using a color feature matching method based on the color space range , and record the flame candidate area The corresponding target shooting camera;

[0055] S300, based on flame candidate regions Traverse the camera night flame interception picture training set Da corresponding to the target shooting camera to perform flame area matching, and obtain the camera night flame interception picture training set Da that matches the flame candidate area The first K image labels with the largest distance matching score are cut out. is a matching flame region set Q; wherein the matching flame region set Q includes a plurality of matching flame regions, Q={Q1, Q2, ..., Qk};

[0056] Based on flame candidate region The distance between each matching flame area in the matching flame area set Q determines the distance matching score S1;

[0057] S400, the flame candidate area Input into the flame recognition deep convolutional neural network model to obtain the image region representation vector ;

[0058] The image label interception area in the camera night flame interception picture training set Da corresponding to the target shooting camera Input them into the flame recognition deep convolutional neural network model respectively to obtain each cropped area The corresponding image region representation vector ;

[0059] The cosine similarity is used for calculation to obtain the twin similarity matching score, and the largest twin similarity matching score is determined as the similarity matching score S2;

[0060] S500, based on flame candidate regions Traverse the camera night light interception picture training set Hb corresponding to the target shooting camera to match the light area, and obtain the camera night light interception picture training set Hb that matches the flame candidate area The image label clipping area with annotated information that meets the preset distance range threshold as a set of matching light areas V, each matching light area in the matching light area set is marked with a corresponding shooting timestamp;

[0061] Get the flame candidate area based on the shooting timestamp The light frequency score S3 of each matching light area;

[0062] S600, obtaining a flame confidence S of the flame candidate area based on the distance matching score S1, the similarity matching score S2 and the light frequency score S3;

[0063] S700, if the flame confidence S is not lower than the flame recognition threshold, the flame candidate area is determined to be a real wildfire area; if the flame confidence S is lower than the flame recognition threshold, the flame candidate area is determined to be a real light area.

[0064] The present invention provides a method for identifying wildfires at night, which obtains a camera night flame interception picture training set Da and a camera night light interception picture training set Hb corresponding to each camera by selecting and marking the night label picture and processing it, and records each image label interception area in the camera night light interception picture training set Hb. The shooting timestamp; determine the color space range that characterizes the color features of the nighttime wildfire pixel values ​​according to the flame image dataset Ra, and after obtaining the current nighttime shot picture, use the color feature matching method based on the color space range to obtain the flame candidate area in the image to be identified , and record the flame candidate area The corresponding target shooting camera; then based on the flame candidate area Traverse the nighttime flame interception picture training set Da corresponding to the target shooting camera to perform flame area matching to obtain multiple matching flame areas, and obtain the distance matching score S1 based on the distance between the flame candidate area and each matching flame area; based on the flame recognition deep convolutional neural network model, the flame candidate area Processing is performed to obtain the image region representation vector Based on the flame recognition deep convolutional neural network model, the image label interception area with annotated information in the camera night flame interception picture training set Da Get the image region representation vector , use cosine similarity to calculate, obtain twin similarity matching score, determine the maximum twin similarity matching score as similarity matching score S2; based on the flame candidate area, traverse the camera night light interception picture training set Hb corresponding to the target shooting camera to match the light area, and obtain the camera night light interception picture set Hb that matches the flame candidate area The image label clipping area with annotated information that meets the preset distance range threshold As the matching light area set V, each matching light area in the matching light area set V is marked with a corresponding shooting timestamp; based on the shooting timestamp, the light frequency score S3 of the flame candidate area and each matching light area V is obtained; finally, the flame confidence S of the flame candidate area is obtained based on the distance matching score S1, the similarity matching score S2 and the light frequency score S3; finally, if the flame confidence S is not lower than the flame recognition threshold, the flame candidate area is determined to be a real wildfire area, and if the flame confidence S is lower than the flame recognition threshold, the flame candidate area is determined to be a real light area. By adopting the method of the present invention, when performing nighttime wildfire identification, the characteristic points of nighttime lights and nighttime wildfires are fully considered. After obtaining the camera nighttime flame interception picture training set Da and the camera nighttime light interception picture training set Hb, the flame candidate area is determined for the current nighttime picture by using a color feature matching method based on the color space range. Based on the image color characteristics, the occurrence time node characteristics and the occurrence position characteristics, the similarity matching score S2, the distance matching score S1 and the light frequency score S3 are fully considered. Finally, the flame confidence S is obtained according to the similarity matching score S2, the distance matching score S1 and the light frequency score S3. The flame candidate area is judged based on the flame confidence S, which can effectively identify the nighttime wildfire scene and significantly reduce the misidentification caused by car lights, street lights, etc., and has a wide range of application scenarios.

[0065] It is understandable that steps S300, S400 and S500 may be performed simultaneously or not simultaneously, and the present invention does not specifically limit this. In the present invention, a wildfire log may be used to obtain nighttime label images. Represents the i-th image label cropping area.

[0066] It is understandable that when performing nighttime wildfire identification, the main consideration is to identify each bright spot area of ​​the current nighttime image to determine whether the bright spot area is a wildfire area or a light area. It is understandable that in this solution, the image label interception area with flame annotation is recorded as Ycropa, and the image label interception area with light annotation is recorded as Ycropb. Optionally, the flame identification threshold can be set according to actual conditions. The larger the value of the flame identification threshold, the more accurate the nighttime flame identification. Optionally, in the present invention, the flame identification threshold is set to 0.8. Optionally, in the present invention, a rectangular frame is used for frame selection when performing frame selection annotation.

[0067] Optionally, a training set Da of nighttime flame intercepted images from one camera may be used to determine the color space range that characterizes the color features of nighttime mountain fire pixel values; a training set Da of nighttime flame intercepted images from two or more cameras may be used to determine the color space range that characterizes the color features of nighttime mountain fire pixel values. In a preferred embodiment of the present invention, a training set Da of nighttime flame intercepted images from all cameras is used to determine the color space range that characterizes the color features of nighttime mountain fire pixel values.

[0068] Furthermore, step S100 specifically includes:

[0069] S101, select and mark the night label image and process it to obtain the image label clipping area with label information, wherein the label information includes light label or flame label, the image label clipping area with flame label Ycropa is combined to form the flame image dataset Ra, and the image label clipping area with light label Ycropb is combined to form the light image dataset Rb;

[0070] S102, determining that the image set of the image label interception area Ycropa corresponding to the same camera in the flame image data set Ra is the camera night flame interception picture training set Da, Da∈Ra;

[0071] Determine the image set of the image label cropped area Ycropb corresponding to the same camera in the light image dataset Rb as the camera night light cropped picture training set Hb, Hb∈Rb, and record the shooting timestamp of each image label cropped area Ycropb in the camera night light cropped picture training set Hb.

[0072] Furthermore, when performing box selection annotation, the annotation area is (x, y, w, h), where x and y represent the horizontal and vertical coordinates of the upper left corner of the wildfire area or the light area, respectively, and w and h represent the width and height of the annotation area, respectively.

[0073] Understandably, in a specific embodiment of the present invention, step S100 may include: wildfire information annotation, annotating the night tag pictures, using rectangular frames to distinguish and annotate the flame areas and the light areas respectively, with the rectangular frames surrounding the bright point areas. For the case where there are multiple flame areas and light areas in a single night tag picture, all of them are annotated in sequence; wildfire annotation information collection, for the night wildfire pictures with existing annotation information, intercepting the image areas annotated as flames to obtain the image tag intercepted area Ycropa (night flame intercepted picture), and intercepting the image areas annotated as lights to obtain the image tag intercepted area Ycropb (night light intercepted picture). Here, let the annotation area be (x, y, w, h), where x and y respectively represent the horizontal and vertical coordinates of the upper left corner of the wildfire area or the light area, and w and h respectively represent the width and height of the annotation area; flame database construction, for all the image tag intercepted areas Ycropa, integrating the image tag intercepted areas Ycropa and distinguishing the camera storage to construct the camera night flame intercepted picture training set Da; light database construction, for all the image tag intercepted areas Ycropb, obtaining the picture shooting time, integrating the image tag intercepted areas Ycropb and their corresponding shooting times, and distinguishing the camera storage to construct the camera night light intercepted picture training set Hb. Understandably, in the present invention, for each image tag intercepted area Ycropb, there is a shooting time stamp and the box selection annotation information.

[0074] Understandably, in a specific embodiment of the present invention, step S200 may include: color space selection, according to the image tag intercepted area Ycropa in the camera night flame intercepted picture training set Da, selecting the color range containing the wildfire pixel values. The color range can adopt various color spaces such as the RGB color space and the HSV color space. The color value range should include all kinds of night flame colors, such as including all the color areas from dark red to bright yellow. Let the color space be RGB, and the selected color space range is (R1 - R2, g1 - g2, b1 - b2); color feature matching, for the night wildfire pictures, selecting the rectangular areas that meet the wildfire color range to obtain each flame candidate area in the current night shot picture. Preferably, for the input current night shot picture, using the similarity color filtering method, selecting the partial intercepted area X that meets the flame color area. For each pixel Xpix in the partial intercepted area X, there is (Ra, g1, b1) < Xpix < (Rb, g2, b2). For the obtained partial intercepted area X, constructing the minimum covering rectangular area, that is, obtaining the flame candidate area .

[0075] Further, in step S300,

[0076] In step S300,

[0077] Using formula Calculate and obtain the flame candidate area The i-th image label interception area in the training set Da of night flame interception pictures captured by the camera Distance ,

[0078] in, Represents the flame candidate area The i-th image label interception area in the training set Da of night flame interception pictures captured by the camera The overlap of Represents the flame candidate area The area of Represents the area of ​​the i-th image label interception region Ycropa_i in the camera night flame interception picture training set Da, represents the overlapping area, and S() represents the area calculation operation;

[0079] Take the first k image label screenshot areas Ycropa with the largest calculated distance values ​​to construct the matching flame area set Q;

[0080] Using formula Calculate the distance matching score S1, where k represents the first K matching image label clipping areas with annotated information. , where k∈V. Optionally, the closer the distance, The larger the value, the more k represents the The largest first K image label screenshot areas Ycropa with annotated information; if there are no satisfying k areas, all image label screenshot areas Ycropa are selected to construct the matching flame area set Q.

[0081] In the scheme of the present invention, the distance matching score S1 is set to fully consider the forest fire situation caused by fixed working habits of personnel (for example, residents stack firewood at fixed locations, burn rice straw at fixed locations, etc.). The larger the distance matching score S1, the more likely the current fire candidate area is. The closer to the historical real flame area, the smaller the distance matching score S1 is, indicating the current flame candidate area The farther away from the historical real flame area, the historical position of the image label interception area Ycropa in each frame in the camera night flame interception picture training set Da is used as an influencing factor for discrimination. Specifically, the first K image label interception areas with annotated information that match the flame candidate area in the camera night flame interception picture training set Da are obtained as matching flame areas {Q1, Q2, ..., Qk}, where K is a positive integer, and K can be 5, or other values ​​such as 6 or 8.

[0082] Further, in step S400,

[0083] Using formula ,

[0084] Calculate and obtain the similarity matching score S2, where: Represents the flame candidate area The image region representation vector obtained by inputting into the flame recognition deep convolutional neural network model, It represents the screenshot area of ​​the i-th image label with annotation information in the training set Da of the camera night flame interception pictures corresponding to the target shooting camera The image region representation vector obtained by inputting into the flame recognition deep convolutional neural network model, cos() represents the cosine similarity calculation function.

[0085] In the solution of the present invention, the similarity matching score S2 is set by taking into full consideration the characteristic points of the current matching light area and nighttime lights and nighttime wildfires to avoid misjudgment caused by the influence of red tubes or fireflies. In a specific implementation, the steps include: twin network construction, constructing a VGG model (flame recognition deep convolutional neural network model) based on a full convolutional network, and initializing model parameters using pre-trained weights. The VGG network is a classic convolutional neural network that uses small filters, so it can better capture local features in the image; image feature calculation, converting the flame candidate area One-to-one matching is performed with the image label interception area Ycropa in the camera night flame interception picture training set Da, and input into the VGG model to obtain the image area representation vector and ; , , where i represents the label cutout area of ​​the i-th frame image ; Twin similarity matching, representing the image and The cosine similarity is used for calculation to obtain the twin similarity matching score, and the maximum twin similarity matching score is retained.

[0086] Further, in step S500,

[0087] Using formula Calculate and obtain the flame candidate area The jth image label cropping area in the training set Hb of the camera night light interception picture Light matching distance ,

[0088] in, To meet the jth image label screenshot area with annotation information within the preset distance range, Represents the flame candidate area The jth image label cropping area in the training set Hb of the camera night light interception picture The overlap degree of S(Xrec) represents the flame candidate region The area of Represents the area of ​​the jth image label screenshot region with annotation information within the preset distance range. represents the overlapping area, and S() represents the area calculation operation;

[0089] Using formula Determine the set V of matching light areas with annotated information within the preset distance range. The image label screenshot areas Ycropb_j are marked with corresponding shooting timestamps. The total number of matching light areas is |V|. Represents the jth image label screenshot area in the matching light area set V , Screenshot area for the jth image label The corresponding shooting timestamp, L is the preset distance range threshold.

[0090] Alternatively, use the formula

[0091] Calculate and obtain the light matching distance, where: To meet the j-th image label interception area with annotated information within the preset distance range, the total number of matching light areas is V frames. (It can be understood that the number of elements in the set V of light areas whose shooting timestamps match each other is , i.e. the V frame described in the present invention), the value of L is set according to actual conditions.

[0092] Furthermore, using the formula Calculate the obtained light frequency score S3, where T is the shooting time node of the flame candidate area, It is understandable that if there is no picture within the preset distance threshold, S3 is a value much greater than zero. The closer S3 is to zero, the more likely it is that the current flame candidate area is The more likely it is to be a real light.

[0093] In the solution of the present invention, the light frequency score S3 is set by taking into full consideration the influence of the time interval. If the light frequency score S3 is larger, it indicates that the current flame candidate area is The smaller the probability of being a real light, the smaller the light frequency score S3 is, indicating that the current flame candidate area The greater the possibility of being a real light. In a specific scenario, if the interval between the flame candidate area and the matching light area V is 1 day, it means that this place was a light area at this time yesterday, so it can be understood that it is very likely to be a light area at this time today, and the possibility of being a flame area is very small. If the interval between the flame candidate area and the matching light area V is 30 days, since the light area that appeared 30 days ago did not appear later, it can be appropriately judged that the light area that appeared 30 days ago was a car light area or other accidental light area, so it can be understood that it is less likely to be a light area at this time today, and the possibility of being a flame area is very high. In specific implementation, the preset distance range is set according to actual conditions. In the present invention, K areas that meet the preset distance range are described as matching light areas V.

[0094] It can be understood that in the solution of the present invention, the timestamp can be recorded in days, hours, minutes or months, and is not specifically limited here.

[0095] Furthermore, using the formula Calculate the flame confidence S, where , , are the weights of the distance matching score S1, the wildfire area twin similarity matching score Score_s, and the light frequency score Score_t, Optionally, in the present invention, The range of value is [0.2, 0.5], The range of value is [0.2, 0.5], The range of value is [0.2, 0.5].

[0096] Furthermore, the method further comprises step S700:

[0097] If the flame candidate area is a real wildfire area, the flame candidate area is updated to the camera night flame interception picture training set Da; if the flame candidate area is a real light area, the flame candidate area is updated to the camera night light interception picture training set Hb.

[0098] The present invention also provides a nighttime mountain fire identification system, comprising a processor and a memory connected to each other, wherein the processor is used to execute the nighttime mountain fire identification method.

[0099] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0100] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for identifying wildfires at night, characterized in that: Includes steps: S100, by selecting and marking the night label pictures and processing them, obtain the camera night flame interception picture training set Da and the camera night light interception picture training set Hb corresponding to each camera, and the camera night light interception picture training set Hb is the image label interception area with light annotation The camera night flame interception picture training set Da is the image label interception area with flame annotation A collection of images captured by cameras at night, recording the label interception area of ​​each image in the training set Hb The shooting timestamp; S200, determining a color space range representing color features of nighttime wildfire pixel values ​​based on at least one camera's nighttime fire interception picture training set Da; obtaining a flame candidate region in the image to be identified by using a color feature matching method based on the color space range , and record the flame candidate area The corresponding target shooting camera; S300, based on flame candidate regions Traverse the camera night flame interception picture training set Da corresponding to the target shooting camera to perform flame area matching, and obtain the camera night flame interception picture training set Da that matches the flame candidate area The first K image labels with the largest distance matching score are cut out. To match the flame area set Q; Based on flame candidate region The distance between each matching flame area in the matching flame area set Q determines the distance matching score S1; S400, the flame candidate area Input into the flame recognition deep convolutional neural network model to obtain the image region representation vector ; The image label interception area in the camera night flame interception picture training set Da corresponding to the target shooting camera Input them into the flame recognition deep convolutional neural network model respectively to obtain each cropped area The corresponding image region representation vector ; The cosine similarity is used for calculation to obtain the twin similarity matching score, and the largest twin similarity matching score is determined as the similarity matching score S2; Using formula , Calculate and obtain the similarity matching score S2, where: Represents the flame candidate area The image region representation vector obtained by inputting into the flame recognition deep convolutional neural network model, It represents the screenshot area of ​​the i-th image label with annotation information in the training set Da of the camera night flame interception pictures corresponding to the target shooting camera The image region representation vector obtained by inputting into the flame recognition deep convolutional neural network model, cos() represents the cosine similarity calculation function; S500, based on flame candidate regions Traverse the camera night light interception picture training set Hb corresponding to the target shooting camera to match the light area, and obtain the camera night light interception picture training set Hb that matches the flame candidate area The image label clipping area with annotated information that meets the preset distance range threshold as a set of matching light areas V, each matching light area in the matching light area set is marked with a corresponding shooting timestamp; Using formula Calculate and obtain the flame candidate area The jth image label cropping area in the training set Hb of the camera night light interception picture Light matching distance , in, To meet the jth image label screenshot area with annotation information within the preset distance range, Represents the flame candidate area The jth image label cropping area in the training set Hb of the camera night light interception picture The overlap degree of S(Xrec) represents the flame candidate region The area, Represents the area of ​​the jth image label screenshot region with annotation information within the preset distance range. represents the overlapping area, and S() represents the area calculation operation; Using formula Determine the set of matching light areas V with annotation information within the preset distance range, and the image label screenshot area They are marked with the corresponding shooting timestamps, and the total number of matching light areas is |V|. Represents the jth image label screenshot area in the matching light area set V , Screenshot area for the jth image label The corresponding shooting timestamp, L is the preset distance range threshold; Using formula Calculate and obtain the light frequency score S3, where T is the shooting time node of the flame candidate area, is the time normalization coefficient; Get the flame candidate area based on the shooting timestamp The light frequency score S3 of each matching light area; S600, obtaining a flame confidence S of the flame candidate area based on the distance matching score S1, the similarity matching score S2 and the light frequency score S3; S700, if the flame confidence S is not lower than the flame recognition threshold, the flame candidate area is determined to be a real wildfire area; if the flame confidence S is lower than the flame recognition threshold, the flame candidate area is determined to be a real light area.

2. The method for identifying wildfires at night according to claim 1, characterized in that: Step S100 specifically includes: S101, select and mark the night label image and process it to obtain the image label clipping area with the label information, wherein the label information includes the light label or the flame label, and the image label clipping area with the flame label Combining the flame image dataset Ra, the image label cutout area with light annotation Combined to form the light image dataset Rb; S102, determining an image set of an image label interception area Ycropa corresponding to a same camera in the flame image data set Ra as a camera night flame interception picture training set Da; Determine the image label interception area corresponding to the same camera in the light image dataset Rb The image set is the camera night light interception picture training set Hb, Hb∈Rb, and the label interception area of ​​each image in the camera night light interception picture training set Hb is recorded The shooting timestamp.

3. The method for identifying wildfires at night according to any one of claims 1 or 2, characterized in that: When performing box selection annotation, the annotation area is (x, y, w, h), where x and y represent the horizontal and vertical coordinates of the upper left corner of the wildfire area or the light area, respectively, and w and h represent the width and height of the annotation area, respectively.

4. The method for identifying wildfires at night according to claim 1, characterized in that: In step S300, Using formula Calculate and obtain the flame candidate area The i-th image label interception area in the training set Da of night flame interception pictures captured by the camera Distance , in, Represents the flame candidate area The i-th image label interception area in the training set Da of night flame interception pictures captured by the camera The overlap of Represents the flame candidate area The area of Represents the i-th image label interception area in the camera night flame interception picture training set Da The area of represents the overlapping area, and S() represents the area calculation operation; Take the first k image label screenshot areas Ycropa with the largest calculated distance values ​​to construct the matching flame area set Q; Using formula Calculate the distance matching score S1, where k represents the first K matching image label clipping areas with annotated information. .

5. The method for identifying wildfires at night according to claim 1, characterized in that: Using formula Calculate the flame confidence S, where , , are the weights of the distance matching score S1, similarity matching score S2, and light frequency score S3, respectively. .

6. The method for identifying wildfires at night according to claim 1, characterized in that: The step S700 is also included: If the flame candidate area is a real wildfire area, then the flame candidate area is updated to the camera night flame interception picture training set Da, If the flame candidate area is a real light area, the flame candidate area is updated to the camera night light interception picture training set Hb.

7. A nighttime mountain fire identification system, characterized in that: The nighttime wildfire identification system comprises a processor and a memory connected to each other, and the processor is used to execute the nighttime wildfire identification method according to any one of claims 1-6.

Images