Forest fire smoke recognition and spreading trend judgment method and device, medium and equipment

Abstract

The invention discloses a forest fire smoke recognition and spreading trend judgment method and device, a medium and equipment, and the method comprises the steps: firstly obtaining a plurality of frames of images in real time, analyzing the intensity change of pixels, and judging whether moving pixels exist in the images or not, thereby constructing a background image of a previous frame and a target foreground image of a current frame; and further determining a suspected smoke region according to the foreground value of the target foreground image and calculating an optical flow vector of the suspected smoke region so as to obtain two smoke characteristic values, namely an area ratio and a fuzzy degree. And when the two characteristic values both exceed a preset threshold value, determining that the forest fire smoke exists in the image. According to the comprehensive monitoring mode, the accuracy and the real-time response capability of forest fire detection can be remarkably improved, and then the overall fire early warning effect is improved.

Description

Technical Field

[0001] This invention relates to the field of monitoring and identification technology for wildfires in and near power transmission line corridors, and in particular to a method, device, medium, and equipment for identifying wildfire smoke and judging its spread trend. Background Technology

[0002] Smoke significantly impacts the insulation performance of power transmission lines. The particles within the smoke can trigger corona discharge in the conductors, potentially leading to line-to-ground discharge and power outages. Currently, most wildfire monitoring systems rely on a combination of video surveillance and manual patrols. While video monitoring systems can capture images and videos of the fire line in real time and upload them to a backend system, the development trend of smoke still requires manual observation and is heavily influenced by environmental factors such as wind direction. Personnel must remain on-site for extended periods until the fire is completely extinguished. This method is inefficient, time-consuming, and labor-intensive, severely hindering the overall effectiveness of wildfire monitoring. Summary of the Invention

[0003] Therefore, it is necessary to provide methods, devices, media, and equipment for identifying wildfire smoke and judging its spread trend in order to solve the problem of poor overall effectiveness of wildfire monitoring.

[0004] A method for identifying wildfire smoke and determining its spread trend, the method comprising:

[0005] The system acquires a first image, a second image, and a third image in real time; wherein the first image is the current frame image, the second image is the previous frame image of the first image, and the third image is the previous frame image of the second image.

[0006] Based on the intensity value change of each pixel in the second image, it is determined whether there is a moving pixel in the second image. If there is a moving pixel in the second image, the background value of the moving pixel is calculated based on the background value and intensity value of each corresponding pixel in the third image, and the background value of the non-moving pixel in the third image is inherited to obtain the background image of the second image; wherein, the moving pixel and the corresponding pixel are in the same position.

[0007] The foreground value of each pixel in the first image is estimated based on the intensity value of the first image and the background value in the background image of the second image to obtain the target foreground image of the first image;

[0008] Based on the correlation degree of foreground values ​​in the target foreground image, a suspected smoke region in the target foreground image is determined, and an optical flow vector of the suspected smoke region is calculated; wherein, the optical flow vector is used to indicate the spread direction of the suspected smoke region;

[0009] A first smoke feature value is calculated based on the number of suspected smoke regions along the direction of the optical flow vector, and a second smoke feature value is calculated based on the Weber contrast method; wherein, the first smoke feature value represents the area ratio of the suspected smoke regions in the first image, and the second feature value represents the blur degree of the first image;

[0010] If the first smoke feature value is greater than a preset area ratio threshold and the second smoke feature value is greater than a preset blur level threshold, then it is determined that there is wildfire smoke in the first image.

[0011] In one embodiment, determining whether there are moving pixels in the second image based on the intensity value change of each pixel in the second image includes:

[0012] When a target pixel in the second image meets a preset intensity change condition, the target pixel is determined to be a moving pixel; wherein, the target pixel is any pixel in the second image, and the intensity change condition is expressed as follows:

[0013] |I t-1 (x,y)-I t-2 (x,y)|>T t-1 (x,y)&|I t-1 (x,y)-I t-3 (x,y)|>T t-1 (x,y)

[0014] Where (x,y) represents the coordinates of the target pixel, I t-1 (x,y) represents the intensity value of the target pixel in the second image, I t-2 (x,y) represents the intensity value of the target pixel in the third image, I t-3 (x, y) represents the intensity value of the target pixel in the fourth image, which is the previous frame of the third image, T t-1 (x,y) represents the intensity value change threshold of the second image, T t-1 (x,y) satisfies:

[0015]

[0016] Where α is a preset coefficient, T t-2 (x,y) represents the intensity value change threshold of the third image, B t-2 (x, y) represents the background value of the target pixel in the third image, and the static condition is expressed as follows:

[0017] I t-1 (x,y)=I t-2 (x,y)=I t-3(x,y).

[0018] In one embodiment, the step of calculating the background value of the moving pixel based on the background value and intensity value of each first corresponding pixel in the third image, and inheriting the background values ​​of non-moving pixels in the third image, includes:

[0019] If the target pixel is a moving pixel, then calculate:

[0020] B t-1 (x,y)=αB t-2 (x,y)+(1-α)I t-1 (x,y)

[0021] If the target pixel is a non-moving pixel, then calculate:

[0022] B t-1 (x,y)=B t-2 (x,y).

[0023] In one embodiment, estimating the foreground value of each pixel in the first image based on the intensity value of the first image and the background value in the background image of the second image is expressed as:

[0024]

[0025] Where β is a preset coefficient, F t (x,y) represents the foreground value of the target pixel.

[0026] In one embodiment, the method further includes:

[0027] Obtain a training image set; wherein the training image set includes multiple first training images and multiple second training images, the first training images are training images carrying smoke labels, and the second training images are training images without smoke labels;

[0028] Multiple sets of coefficients are set; each set of coefficients includes preset α and β, and the coefficients set for different sets of coefficients are different.

[0029] Under the premise of setting different coefficient groups, the true positive rate and false positive rate of the training image set in judging whether there is wildfire smoke are obtained respectively, so as to obtain the true positive rate and false positive rate of each coefficient group;

[0030] The coefficient group that has a true positive rate falling within a preset first interval and a false positive rate falling within a preset second interval is selected as the target coefficient group.

[0031] In one embodiment, determining the suspected smoke region in the target foreground image based on the correlation degree of foreground values ​​in the target foreground image includes:

[0032] Obtain the foreground image of the second image, and divide the target foreground image and the foreground image of the second image into multiple pixel blocks of the first image and multiple pixel blocks of the second image with the same size and position;

[0033] Calculate the correlation between the first pixel block and each corresponding second pixel block; wherein the first pixel block is any one of the multiple pixel blocks in the second image, and the second pixel block is a pixel block in the first image that is adjacent to the first pixel block. The formula for calculating the correlation is:

[0034]

[0035] Where, q t-1 (x,y) represents the foreground value of the first pixel block, q t (x+k1d,y+k2d) represents the foreground value of the second pixel block, k1 represents the search coefficient in the x direction, k2 represents the search coefficient in the y direction, d represents the unit length of each pixel block, min(,) represents taking the minimum value, and min(,) represents taking the maximum value.

[0036] The maximum value among the multiple correlations of the first pixel block is taken as the target correlation of the first pixel block. If the target correlation is greater than a preset correlation threshold, the second pixel block corresponding to the target correlation is taken as a suspected smoke area.

[0037] In one embodiment, the calculation of the optical flow vector of the suspected smoke region is expressed as:

[0038]

[0039] in, Represents pixel block q t The p-th pixel in (x+k1d,y+k2d) is represented by [CsxCsy], which represents the optical flow vector.

[0040] In one embodiment, the formula for calculating the first smoke feature value is expressed as follows:

[0041]

[0042] Where c represents the total number of the second pixel blocks, and cs represents the number of suspected smoke regions in the first image along the direction of the optical flow vector;

[0043] The formula for calculating the second smoke characteristic value is as follows:

[0044]

[0045] Where n represents the number of pixels in the first image.

[0046] A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the above-described method for identifying wildfire smoke and determining its spread trend.

[0047] A terminal device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the above-described method for identifying wildfire smoke and determining its spread trend.

[0048] This invention provides a method, apparatus, medium, and device for identifying wildfire smoke and determining its spread trend. First, multiple frames of images are acquired in real time, and pixel intensity changes are analyzed to determine if moving pixels exist in the images, thereby constructing a background image from the previous frame and a target foreground image for the current frame. Further, suspected smoke areas are determined based on the foreground values ​​of the target foreground image, and the optical flow vector of these areas is calculated, resulting in two smoke feature values: area proportion and blur degree. If both feature values ​​exceed preset thresholds, wildfire smoke is determined to be present in the image. This comprehensive monitoring method can significantly improve the accuracy and real-time response capability of wildfire detection, thereby enhancing the overall fire early warning effect. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] in:

[0051] Figure 1 A flowchart illustrating the method for identifying wildfire smoke and determining its spread trend;

[0052] Figure 2 This is a structural block diagram of the terminal device. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0055] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0056] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a method for identifying and determining the spread trend of wildfire smoke in one embodiment. The steps provided by the method for identifying and determining the spread trend of wildfire smoke in this embodiment include:

[0057] S101, acquire the first image, the second image and the third image in real time.

[0058] The first image is the current frame image, that is, the latest image acquired from the video stream or image sensor at this moment. The second image is the previous frame image of the first image, and the third image is the previous frame image of the second image.

[0059] Each acquired image frame undergoes preprocessing, which can be broadly categorized into two cases:

[0060] For images with backgrounds primarily consisting of buildings, forests, or mountains, frame preprocessing includes three steps: grayscale conversion, histogram equalization, and discrete wavelet transform. This preprocessing module, comprised of these steps, improves the performance of smoke detection algorithms and reduces false alarms. This module is not only suitable for handling image contrast in different scenes but also addresses variations in image sensor lighting conditions. Specifically, discrete wavelet transform is applied along the vertical and diagonal directions of the image to perform multi-scale processing, resizing the image and removing high-frequency noise in the horizontal direction.

[0061] For images captured from towers on mountaintops or high-altitude towers in flat areas, the preprocessing process often requires Gaussian filtering due to the presence of complex background elements such as distant clouds. A two-dimensional Gaussian kernel (usually odd in number to ensure a center point) is generated by setting a standard deviation σ and applied to each pixel of the original image. Neighborhood convolution is then performed on each pixel to generate a new pixel value. Convolution is not performed on pixels at image boundaries. Next, a Laplacian filter is used to process the image. Convolving each pixel with the Laplacian operator results in a Laplacian image that highlights edges or areas of change. Thresholding or enhancement processing can be applied to the Laplacian image to further emphasize edge information. For example, a weighted sum of the Laplacian image and the original image can enhance edge details. These transformations improve the contrast quality of the input frame while reducing the computational burden of subsequent image processing, resulting in an output image frame that meets the requirements.

[0062] S102, determine whether there are moving pixels in the second image based on the intensity value change of each pixel in the second image. If there are moving pixels in the second image, calculate the background value of the moving pixels based on the background value and intensity value of each corresponding pixel in the third image, and inherit the background value of the non-moving pixels in the third image to obtain the background image of the second image.

[0063] The intensity value refers to the grayscale or brightness value of each pixel in the image. A moving pixel is a pixel whose intensity value changes. When comparing adjacent frames (the second image and the third image), if the intensity value of a pixel changes significantly, that pixel is considered a moving pixel, possibly indicating object movement or other dynamic changes (such as the movement of smoke). Moving pixels are located in the same position as their corresponding pixels. The background value refers to the background information of each pixel in the image. By calculating the background value of moving pixels and inheriting the background values ​​of non-moving pixels, a background image is generated, primarily used to eliminate dynamic objects or changing parts while preserving the static scene background.

[0064] In one specific embodiment, determining whether there are moving pixels in the second image based on the intensity value change of each pixel in the second image includes:

[0065] When a target pixel in the second image meets a preset intensity change condition, the target pixel is determined to be a moving pixel; whereby the target pixel is any pixel in the second image, and the intensity change condition is expressed as follows:

[0066] |I t-1 (x,y)-I t-2 (x,y)|>T t-1 (x,y)&|I t-1 (x,y)-I t-3(x,y)|>T t-1 (x,y)

[0067] Where (x,y) represents the coordinates of the target pixel, I t-1 (x,y) represents the intensity value of the target pixel in the second image, I t-2 (x,y) represents the intensity value of the target pixel in the third image, I t-3 (x, y) represents the intensity value of the target pixel in the fourth image, which is the frame preceding the third image. t-1 (x, y) represents the intensity value change threshold of the second image. This means that if the intensity change of a pixel exceeds the threshold T between consecutive image frames, the image is considered indicative of an indicative change. t-1 If (x,y), then the pixel is determined to be a moving pixel, possibly due to the movement of smoke or dynamic changes of other objects.

[0068] T t-1 (x,y) is a dynamically changing value, while the first intensity value change threshold T0(x,y) is a pre-set small value, generally close to 0.01 after normalization. t-1 (x,y) satisfies:

[0069]

[0070] Here, α is a preset coefficient, between 0 and 1. As the smoke diffuses, α gradually approaches 1, indicating that the threshold will gradually stabilize. t-2 (x,y) represents the intensity value change threshold of the third image, B t-2 (x,y) represents the background value of the target pixel in the third image.

[0071] This means that when a pixel does not meet the static condition (i.e., the pixel intensity value changes significantly between multiple frames), the threshold T of the second image... t-1 (x,y) will be dynamically updated based on the threshold of the previous frame and the difference between the pixel intensity and the background value. If the pixel meets the static condition (the pixel has not changed in multiple frames), the threshold of the previous frame will remain unchanged.

[0072] Static conditions are expressed as:

[0073] I t-1 (x,y)=I t-2 (x,y)=I t-3 (x,y).

[0074] In other words, if a pixel has the same intensity value in three consecutive frames, it is considered a static pixel. This means that the pixel is in the background and has not changed, such as a fixed background of buildings or trees.

[0075] In the above specific embodiments, by introducing a weighted dynamic threshold based on time and background information, the system can effectively reduce false alarms caused by non-target events such as changes in lighting and camera shake, thereby improving the accuracy of smoke detection.

[0076] In one specific embodiment, the background value of the moving pixel is calculated based on the background value and intensity value of each first corresponding pixel in the third image, and the background value of the non-moving pixel in the third image is inherited, including:

[0077] If the target pixel is a moving pixel, then calculate:

[0078] B t-1 (x,y)=αB t-2 (x,y)+(1-α)I t-1 (x,y)

[0079] Understandably, when smoke appears, the intensity value of the smoke pixels will gradually approach the intensity value of the background. To adapt to this change, the background value B... t-1 (x,y) is based on the pixel intensity value I of the current frame. t-1 The (x, y) values ​​are updated so that the background values ​​more accurately reflect the current scene as the smoke moves. An adaptive coefficient α is used to balance the influence of historical background values ​​and current pixel values. As α gradually increases, the background values ​​are updated more slowly, maintaining high stability.

[0080] If the target pixel is a non-moving pixel, then calculate:

[0081] B t-1 (x,y)=B t-2 (x,y).

[0082] Non-moving pixels represent stationary parts of the image, such as fixed objects or buildings. For these pixels, the background value does not change, so there is no need to recalculate; the background value from the previous frame can be used directly. This approach reduces unnecessary computation, improves the efficiency of the algorithm, and maintains the stability of the background.

[0083] S103, estimate the foreground value of each pixel in the first image based on the intensity value of the first image and the background value in the background image of the second image, so as to obtain the target foreground image of the first image.

[0084] In one specific embodiment, the foreground value of each pixel in the first image is estimated based on the intensity value of the first image and the background value in the background image of the second image, expressed as:

[0085]

[0086] Among them, F t(x,y) represents the foreground value of the target pixel. β is a preset coefficient that determines the weight of the background value and the current frame pixel intensity value in the foreground estimation. Through experiments, we determined that the optimal value of β is 0.38.

[0087] In one specific embodiment, the coefficients α and β are set by the following steps:

[0088] A1. Obtain the training image set.

[0089] The training image set includes multiple first training images and multiple second training images. The first training images are those labeled with smoke, and the second training images are those without smoke labels. These two types of images provide the model with positive and negative samples needed for smoke detection, helping the model learn how to distinguish between images with smoke and those without.

[0090] A2. Set multiple sets of coefficient groups.

[0091] Each coefficient group includes preset α and β values, with different coefficients set for different groups. This is used to test the impact of different combinations on the detection results.

[0092] A3. Under the premise of setting different coefficient groups, obtain the true positive rate and false positive rate of the training image set when judging whether there is wildfire smoke, so as to obtain the true positive rate and false positive rate of each coefficient group.

[0093] The true positive rate is the proportion of detected smoke images that actually contain smoke. It reflects the detection accuracy of the algorithm. The false positive rate is the proportion of smoke-free images that are mistakenly identified as smoke. It reflects the false positive rate of the algorithm. Through this process, the actual effect of different coefficient sets on smoke detection can be quantified.

[0094] A4. The coefficient group in which the true positive rate falls within the preset first interval and the false positive rate falls within the preset second interval is taken as the target coefficient group.

[0095] The first interval is used to screen for intervals that meet the required true positive rate, i.e., those requiring high detection accuracy. The second interval is used to screen for intervals that meet the required false positive rate, i.e., those requiring low false positive rate. In this step, the coefficient set that meets the requirements of these two intervals is taken as the target coefficient set, i.e., the optimal parameter combination for smoke detection. These target coefficient sets can achieve good smoke detection performance while balancing detection accuracy and false positive rate.

[0096] Optionally, thresholding can also be applied to generate the binary foreground Fbi n, denoted as:

[0097]

[0098] Optionally, after binary foreground generation, some noise may still remain in the image, requiring morphological operations to remove the noise and process irregular foreground patches. Morphological operations are set-theoretic image processing operations commonly used for noise removal, hole filling, or object separation, including opening operations.

[0099]

[0100] Where S represents the image; M represents the structural region, such as a 3x3 square pixel region. The opening operation involves first performing an erosion operation (-), followed by a dilation operation. In this way, small noise points can be eliminated while preserving larger connected regions.

[0101] And closing operation:

[0102]

[0103] The closing operation is the opposite of the opening operation; it first performs a dilation operation, followed by an erosion operation. The closing operation is primarily used to eliminate small holes or gaps in the foreground and to fill in discontinuities.

[0104] By combining binary processing with morphological operations, invalid contours can be eliminated, resulting in clear foreground targets and significantly reducing the consumption of subsequent computing resources.

[0105] S104, Determine the suspected smoke region in the target foreground image based on the correlation degree of the foreground values ​​in the target foreground image, and calculate the optical flow vector of the suspected smoke region.

[0106] By analyzing the intensity variations and positional relationships between pixels, it can be determined whether these pixels constitute the same target region, such as a smoke area. Relatively smooth and consistent changes in adjacent pixel values ​​usually indicate that they belong to the same suspected smoke area. The optical flow vector is used to indicate the direction of spread of the suspected smoke area.

[0107] In one specific embodiment, determining the suspected smoke region in the target foreground image based on the correlation degree of foreground values ​​in the target foreground image specifically includes the following steps:

[0108] B1. Obtain the foreground image of the second image, and divide the target foreground image and the foreground image of the second image into multiple pixel blocks of the first image and multiple pixel blocks of the second image with the same size and position.

[0109] For example, if the image size is 320×240 pixels, it can be divided into multiple 2×2 pixel blocks. Each block contains 4 pixels, which facilitates subsequent correlation calculations. Understandably, the size of the pixel blocks can be adjusted according to actual needs. Smaller blocks help to precisely locate the edges of the smoke, while larger blocks can quickly filter out large areas of smoke.

[0110] B2. Calculate the correlation between the first pixel block and each corresponding second pixel block.

[0111] Wherein, the first pixel block is any one of the multiple pixel blocks in the second image, and the second pixel block is the pixel block in the first image that is adjacent to the first pixel block. The formula for calculating the correlation is:

[0112]

[0113] Where, q t-1 (x,y) represents the foreground value of the first pixel block, q t (x+k1d,y+k2d) represents the foreground value of the second pixel block, k1 represents the search coefficient in the x direction, k2 represents the search coefficient in the y direction, d represents the unit length of each pixel block, and if the image is divided into multiple 2×2 pixel blocks, then d=2, min(,) represents taking the minimum value among them, and min(,) represents taking the maximum value among them.

[0114] In other words, the correlation calculation uses the ratio of the minimum to the maximum value to measure the similarity of pixel blocks. The closer the ratio is to 1, the more similar the foreground values ​​of the two pixel blocks are, indicating that they are likely to belong to the same suspected smoke region.

[0115] B3. Take the maximum value among the multiple correlations of the first pixel block as the target correlation of the first pixel block. If the target correlation is greater than the preset correlation threshold, then take the second pixel block corresponding to the target correlation as the suspected smoke area.

[0116] The maximum value among multiple correlations reflects the strongest similarity between the pixel block and its surrounding pixel blocks. If the target correlation exceeds a preset correlation threshold, the pixel block and its corresponding second pixel block are considered potentially smoke regions. This method effectively filters out areas containing smoke in the image. Furthermore, by calculating the correlation between pixel blocks, potentially smoke regions can be effectively identified in the image, thus avoiding global analysis of the entire image and improving computational efficiency.

[0117] In one specific embodiment, the optical flow vector of the suspected smoke region is calculated as follows:

[0118]

[0119] in, Represents pixel block q t The p-th pixel in (x+k1d,y+k2d) is represented by [CsxCsy], which is the optical flow vector.

[0120] Here we take a 2x2 pixel block as an example:

[0121]

[0122] Solving this matrix allows us to calculate Csx and Csy, which represent the movement of the smoke in the horizontal and vertical directions.

[0123] S105, calculate the first smoke feature value based on the number of suspected smoke regions along the direction of the optical flow vector, and calculate the second smoke feature value based on the Weber comparison method.

[0124] The first smoke feature value represents the area ratio of the suspected smoke region in the first image, and the second feature value represents the degree of blurring in the first image.

[0125] In one specific embodiment, the formula for calculating the first smoke characteristic value is expressed as follows:

[0126]

[0127] Where c represents the total number of second pixel blocks, and cs represents the number of suspected smoke regions along the direction of the optical flow vector in the first image. This value reflects the proportion of smoke in the image; the larger the value, the wider the area covered by smoke in the image.

[0128] The formula for calculating the second smoke characteristic value is as follows:

[0129]

[0130] Where n represents the number of pixels in the first image. C w It can effectively reflect the degree of blur in an image, thus helping to determine the concentration of smoke. In a dense smoke environment, the blurriness of the image will increase significantly.

[0131] S106, if the first smoke feature value is greater than the preset area ratio threshold and the second smoke feature value is greater than the preset blur degree threshold, then it is determined that there is wildfire smoke in the first image.

[0132] That is, if ρ>ρ threshold And C w >C w threshold If ρ is the smoke in the first image, then it is determined that smoke is present. threshold C represents the preset area percentage threshold. wthreshold This represents the preset blur threshold. It's understandable that the presence of smoke in an image is usually accompanied by blurring. This step, combining the first and second smoke feature values, can effectively determine whether wildfire smoke exists in the image, thus supporting fire monitoring and early warning. This mechanism lays the foundation for the intelligence and efficiency of smoke detection systems.

[0133] Figure 2 An internal structural diagram of a terminal device in one embodiment is shown. Figure 2 As shown, the terminal device includes a processor, memory, and network interface connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program. When executed by the processor, this computer program enables the processor to implement a method for identifying wildfire smoke and determining its spread trend. The internal memory may also store a computer program, which, when executed by the processor, enables the processor to implement the method for identifying wildfire smoke and determining its spread trend. Those skilled in the art will understand that… Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application and does not constitute a limitation on the terminal device to which the solution of this application is applied. A specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0134] A computer-readable storage medium stores a computer program that, when executed by a processor, performs the following steps: acquiring a first image, a second image, and a third image in real time; wherein the first image is the current frame image, the second image is the previous frame image of the first image, and the third image is the previous frame image of the second image; determining whether there are moving pixels in the second image based on the intensity value changes of each pixel in the second image; if moving pixels exist in the second image, calculating the background value of the moving pixels based on the background value and intensity value of each corresponding pixel in the third image, and inheriting the background values ​​of non-moving pixels in the third image to obtain the background image of the second image; wherein the moving pixels are located at the same position as the corresponding pixels; and based on the intensity value of the first image and the intensity value of the second image... The background value in the background image is used to estimate the foreground value of each pixel in the first image to obtain the target foreground image of the first image; the suspected smoke region in the target foreground image is determined according to the correlation of the foreground values ​​in the target foreground image, and the optical flow vector of the suspected smoke region is calculated; wherein, the optical flow vector is used to indicate the spread direction of the suspected smoke region; the first smoke feature value is calculated according to the number of suspected smoke regions along the direction of the optical flow vector, and the second smoke feature value is calculated according to the Weber contrast method; wherein, the first smoke feature value represents the area ratio of the suspected smoke region in the first image, and the second feature value represents the blur degree of the first image; if the first smoke feature value is greater than a preset area ratio threshold, and the second smoke feature value is greater than a preset blur degree threshold, then it is determined that there is wildfire smoke in the first image.

[0135] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: acquiring a first image, a second image, and a third image in real time; wherein the first image is the current frame image, the second image is the previous frame image of the first image, and the third image is the previous frame image of the second image; determining whether there are moving pixels in the second image based on the intensity value changes of each pixel in the second image; if moving pixels exist in the second image, calculating the background value of the moving pixels based on the background value and intensity value of each corresponding pixel in the third image, and inheriting the background values ​​of non-moving pixels in the third image to obtain the background image of the second image; wherein the moving pixels are located at the same position as the corresponding pixels; based on the first image... The intensity value and the background value in the background image of the second image are used to estimate the foreground value of each pixel in the first image to obtain the target foreground image of the first image; the suspected smoke region in the target foreground image is determined according to the correlation of the foreground values ​​in the target foreground image, and the optical flow vector of the suspected smoke region is calculated; wherein, the optical flow vector is used to indicate the spread direction of the suspected smoke region; the first smoke feature value is calculated according to the number of suspected smoke regions along the direction of the optical flow vector, and the second smoke feature value is calculated according to the Weber contrast method; wherein, the first smoke feature value represents the area ratio of the suspected smoke region in the first image, and the second feature value represents the blur degree of the first image; if the first smoke feature value is greater than a preset area ratio threshold, and the second smoke feature value is greater than a preset blur degree threshold, then it is determined that there is wildfire smoke in the first image.

[0136] It should be noted that the above-mentioned method, device, equipment and computer-readable storage medium for identifying wildfire smoke and judging its spread trend belong to the same general inventive concept, and the contents of the embodiments of the method, device, equipment and computer-readable storage medium for identifying wildfire smoke and judging its spread trend are applicable to each other.

[0137] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for identifying mountain fire smoke and determining its spreading trend, characterized in that: The method comprises: Acquire a first image, a second image, and a third image in real time; wherein the first image is a current frame image, the second image is a previous frame image of the first image, and the third image is a previous frame image of the second image; Determine whether there is a moving pixel in the second image according to the change of the intensity value of each pixel point in the second image, and if there is a moving pixel in the second image, calculate the background value of the moving pixel according to the background value and intensity value of each corresponding pixel in the third image, and inherit the background value of the non-moving pixel in the third image to obtain the background image of the second image; wherein the moving pixel and the corresponding pixel have the same position; estimating a foreground value of each pixel in the first image according to the intensity value of the first image and the background value in the background image of the second image to obtain a target foreground image of the first image; Determine a suspected smoke region in the target foreground image according to the correlation degree of the foreground value in the target foreground image, and calculate an optical flow vector of the suspected smoke region; wherein the optical flow vector is used to indicate a spreading direction of the suspected smoke region; Calculating a first smoke characteristic value according to the number of suspected smoke regions along the direction of the optical flow vector, and calculating a second smoke characteristic value according to the Weber contrast method; wherein the first smoke characteristic value represents the area ratio of the suspected smoke region in the first image, and the second characteristic value represents the blur degree of the first image; If the first smoke characteristic value is greater than a preset area ratio threshold, and the second smoke characteristic value is greater than a preset blur level threshold, it is determined that there is wildfire smoke in the first image.

2. The method according to claim 1, characterized in that The determining whether there is a moving pixel in the second image according to a change in the intensity value of each pixel point in the second image includes: When a target pixel in the second image satisfies a preset intensity value change condition, the target pixel is determined to be a moving pixel; wherein the target pixel is any pixel in the second image, and the intensity change condition is expressed as: |I t-1 (x,y)-I t-2 (x,y)|>T t-1 (x,y)&|I t-1 (x,y)-I t-3 (x,y)|>T t-1 (x,y) Among them, (x, y) represents the coordinates of the target pixel, I t-1 (x, y) represents the intensity value of the target pixel in the second image, I t-2 (x, y) represents the intensity value of the target pixel in the third image, I t-3 (x, y) represents the intensity value of the target pixel in the fourth image, and the fourth image is the previous frame of the third image. t-1 (x, y) represents the intensity value change threshold of the second image, T t-1 (x,y) satisfies: If (x, y) does not meet the static condition, α is the preset coefficient, T t-2 (x, y) represents the intensity value change threshold of the third image, B t-2 (x, y) represents the background value of the target pixel in the third image, and the static condition is expressed as: I t-1 (x,y)=I t-2 (x,y)=I t-3 (x,y)。 3. The method according to claim 2, characterized in that The step of calculating the background value of the moving pixel according to the background value and the intensity value of each first corresponding pixel in the third image and inheriting the background value of the non-moving pixel in the third image includes: If the target pixel is a moving pixel, calculate: B t-1 (x,y)=αB t-2 (x,y)+(1-α)I t-1 (x,y) If the target pixel is a non-moving pixel, calculate: B t-1 (x,y)=B t-2 (x,y)。 4. The method according to claim 3, characterized in that The foreground value of each pixel in the first image is estimated according to the intensity value of the first image and the background value in the background image of the second image, which is expressed as: Among them, β is the preset coefficient, F t (x, y) represents the foreground value of the target pixel.

5. The method according to claim 4, characterized in that The method further comprises: Acquire a training image set; wherein the training image set includes a plurality of first training images and a plurality of second training images, the first training images are training images carrying smoke labels, and the second training images are training images not carrying smoke labels; Setting multiple coefficient groups; wherein each coefficient group includes a preset α and β, and different coefficient groups have different coefficients; Under the premise of setting different coefficient groups, respectively obtaining the true positive rate and false positive rate of the training image set when judging whether there is wildfire smoke, so as to obtain the true positive rate and false positive rate of each coefficient group; The coefficient group whose true positive rate falls within a preset first interval and whose false positive rate falls within a preset second interval is taken as the target coefficient group.

6. The method according to claim 1, characterized in that The determining of the suspected smoke region in the target foreground image according to the correlation degree of the foreground values ​​in the target foreground image comprises: Acquire a foreground image of the second image, and divide the target foreground image and the foreground image of the second image into the same size and position to obtain a plurality of pixel blocks of the first image and a plurality of pixel blocks of the second image; Calculate the correlation between the first pixel block and each corresponding second pixel block; wherein the first pixel block is any one of the plurality of pixel blocks of the second image, and the second pixel block is a pixel block in the first image that is adjacent to the first pixel block, and the calculation formula of the correlation is: Among them, q t-1 (x, y) represents the foreground value of the first pixel block, q t (x+k1d, y+k2d) represents the foreground value of the second pixel block, k1 represents the search coefficient in the x direction, k2 represents the search coefficient in the y direction, d represents the unit length of each pixel block, min(,) represents taking the minimum value, and min(,) represents taking the maximum value; The maximum value among the multiple correlations of the first pixel block is used as the target correlation of the first pixel block. If the target correlation is greater than a preset correlation threshold, the second pixel block corresponding to the target correlation is used as a suspected smoke area.

7. The method according to claim 6, characterized in that The optical flow vector of the suspected smoke area is calculated as follows: in, Represents pixel block q t The p-th pixel in (x+k1d,y+k2d), [Csx Csy] represents the optical flow vector.

8. The method according to claim 6, characterized in that The calculation formula of the first smoke characteristic value is expressed as: Wherein, c represents the total number of the second pixel blocks, and cs represents the number of suspected smoke regions in the first image along the direction of the optical flow vector; The calculation formula of the second smoke characteristic value is expressed as: Here, n represents the number of pixels in the first image.

9. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

10. A terminal device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 8.

Images