Forest Smoke Detection System and Method Based on Moving Target Detection Mechanism

Abstract

The present invention discloses a forest smoke detection system and method based on a moving target detection mechanism, comprising a video acquisition device, a data processing device and a display device; the video acquisition device is used for acquiring real-time videos of forests in a monitoring area; the data processing device is used for calculating the brightness information of each pixel point and obtaining smoke motion response information in the case of increased brightness, so as to obtain enhanced smoke motion response information and motion direction information; after the data processing device obtains the final response coefficient and integrates it with the enhanced smoke motion response information, it outputs the position information of smoke targets in the video; the display device is used for identifying and displaying the output position information of the smoke targets. The present invention can more accurately and reliably detect and give early warnings of smoke in a natural image background and a real natural video.

Description

Technical Field

[0001] The present invention relates to the field of forest fire smoke detection, and particularly to a forest smoke detection system and method based on a moving target detection mechanism. Background Art

[0002] Forest fires are extremely destructive natural disasters that get out of human control and spread freely in mountainous areas, forests, grasslands, etc. Since the fire occurrence locations are often in mountainous areas with sparse population and rugged terrain, where transportation is inconvenient and fire fighting is difficult, it poses a great threat to human life, property and the ecological environment. At present, the forest fire monitoring system in China is not yet perfect, the coverage rate of fire fighting equipment is not wide, there are blind spots, the fire field communication and information sharing are not strong enough, far from meeting the actual work needs. Existing aircraft (such as drones, etc.) can achieve real-time image transmission and have a low usage cost, and have unique advantages in forest fire smoke detection. However, the current automatic smoke detection based on images collected by aircraft is easily affected by complex backgrounds and light, and the effects are often not satisfactory. Moving target detection is an innate ability for birds to survive, which helps them avoid predators and forage. Neuroanatomical studies have shown that the moving target detection of birds in complex backgrounds is mainly completed by the optic tectum visual pathway. And the sensitivity to target movement and the robustness independent of the background benefit from the processing of the optic tectum nuclei. The forest smoke detection method based on the moving target detection mechanism of the avian optic tectum is expected to solve the problem of real-time smoke detection and early warning when a fire occurs in remote areas that cannot be covered by ground patrols. Summary of the Invention

[0003] The purpose of the present invention is to provide a forest smoke detection system and method based on a moving target detection mechanism, which can more accurately and reliably detect and give early warning of smoke in natural image backgrounds and real natural videos.

[0004] The present invention adopts the following technical solutions:

[0005] A forest smoke detection system based on a moving target detection mechanism, comprising a video acquisition device, a data processing device and a display device;

[0006] The video acquisition device is used to acquire real-time videos of the forest in the monitoring area and convert the acquired videos into digital signals and send them to the data processing device;

[0007] A data processing device is used to calculate the brightness information of each pixel in each frame of video image through a brightness perception module, then obtain the smoke movement response information of each pixel in each frame of video image under the condition of increasing brightness through a motion detection module, and then use a spatio-temporal accumulation module to obtain enhanced smoke movement response information and motion direction information through the smoke movement response information of pixels with spatio-temporal correlation in different frames of video images; the data processing device also obtains a final response coefficient for adjusting the enhanced smoke movement response information through a peripheral modulation module, and the final response coefficient includes a motion direction contrast coefficient and a color contrast coefficient. After integrating the enhanced smoke movement response information with the final response coefficient through an information fusion module, the position information of the smoke target in the video is output;

[0008] A display device is used to identify and display the position information of the smoke target output by the data processing device.

[0009] The data processing device described above includes a brightness perception module, a motion detection module, a spatio-temporal accumulation module, a peripheral modulation module, and an information fusion module;

[0010] The brightness perception module is used to calculate the brightness information of each pixel in each frame of the input video and perform smoothing processing on the brightness information;

[0011] The motion detection module is used to calculate the ON-OFF signal of the brightness change of each pixel in each frame of video image and perform delay processing on the separated ON signal; then convolve the delay of the obtained ON signal with the OFF signal of the same pixel in the same frame of video image to obtain the smoke movement response information at the corresponding pixel position;

[0012] The spatio-temporal accumulation module is used to perform spatio-temporal information accumulation operations in eight motion directions, namely up, down, left, right, upper left, lower left, upper right, and lower right, through the motion information of pixels with spatio-temporal correlation in different frames of video images to obtain the motion information values in each motion direction. The motion information value is the enhanced smoke movement response information; finally, select the largest motion information value as the motion direction information at the pixel position, and the corresponding direction is the motion direction of the motion target at the pixel position; if the motion information value in each direction is lower than the set motion information threshold E e , it is considered that there is no motion target at the pixel position;

[0013] The peripheral modulation module is used to obtain the final response coefficient for adjusting the enhanced smoke movement response information through division normalization operation. The final response coefficient includes a motion direction contrast coefficient and a color contrast coefficient;

[0014] An information fusion module is used to integrate the enhanced smoke motion response information output by the spatio-temporal accumulation module and the final response coefficient output by the peripheral modulation module, that is, to convolve the enhanced smoke motion response information and the final response coefficient from the same pixel position to obtain a fusion value, take the maximum value in the fusion value as the fusion comparison value, and then compare the fusion comparison value with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a moving target in the frame of the video image.

[0015] When there is a smoke target in the frame of the video image, the information fusion module also uses the pixel position corresponding to the corresponding fusion comparison value as the reference pixel point, calculates the fusion comparison values corresponding to each pixel point around the reference pixel point, and determines whether it is greater than the set comparison threshold of the fusion comparison value of the reference pixel point, and then takes the positions of the pixel points corresponding to all the fusion comparison values greater than the set comparison threshold and the reference pixel as the appearance position of the smoke target.

[0016] When the spatio-temporal accumulation module obtains the motion information values in each motion direction, the motion information values of the pixel position (x m , y q , t k ) in each motion direction are as follows:

[0017] The motion information value in the rightward motion direction is:

[0018]

[0019] The motion information value in the leftward motion direction is:

[0020]

[0021] The motion information value in the downward motion direction is:

[0022]

[0023] The motion information value in the upward motion direction is:

[0024]

[0025] The motion information value in the left-upward motion direction is:

[0026]

[0027] The motion information value in the left-downward motion direction is:

[0028]

[0029] The motion information value in the right-upward motion direction is:

[0030]

[0031] The motion information value in the downward right motion direction is:

[0032]

[0033] where x m , y q , t k , x m±i , y q±i , t k-i The subscript values represent the position coordinates of the pixel points, M and Q represent the sizes of each video image frame; min(M - m, Q - q) represents the minimum value of M - m and Q - q; min(m - 1, Q - q) represents the minimum value of m - 1 and Q - q; min(M - m, q - 1) represents the minimum value of M - m and q - 1; min(m - 1, q - 1) represents the minimum value of m - 1 and q - 1; W i represents the cumulative weight value of the current video image frame; D(x, y, t) = S OFF (x, y, t) · S D-ON (x, y, t); S OFF (x, y, t) represents the negative of L(x, y, t), where L(x, y, t) is the response output of the brightness change in the motion detection module, and S D-ON (x, y, t) is the delay of the ON signal.

[0034] In the spatio-temporal accumulation module, the maximum motion information value is selected as the motion direction information at the position of the pixel point, and at the same time, the motion direction corresponding to the maximum motion information value is recorded as the motion direction of the moving target at the position of the pixel point; if the motion information values in all eight directions are lower than the set motion information threshold E e , it is considered that there is no moving target at the position of the pixel point;

[0035] The enhanced smoke motion response information at the position of the pixel point (x m , y q , t k ) is:

[0036]

[0037] When dir = 0, it means that there is no moving target at the position of the pixel point; dir = 1, 2, 3, 4, 5, 6, 7, and 8 respectively represent the motion directions of right, left, down, up, upper left, lower left, upper right, and lower right; d represents the motion direction at the position of the pixel point (x m , y q , t k ).

[0038] The peripheral modulation module described above includes a motion direction modulation module;

[0039] The motion direction modulation module is used to obtain the motion direction contrast coefficient R m , y q , t k ) at the pixel position (x dir (x m , y q , t k );

[0040]

[0041] Among them, N dir (x m , y q , t k ) represents the frequency of occurrence of the motion direction at the pixel position (x m , y q , t k ) in the current video image, represents the sum of the frequencies of occurrence of all motion directions. There are 9 groups of directions in total. d = 0 indicates no motion, and d = 1, 2, 3, 4, 5, 6, 7, 8 respectively indicate that the motion directions are right, left, down, up, upper left, lower left, upper right, and lower right.

[0042] The peripheral modulation module described above also includes a color modulation module;

[0043] The color modulation module is used to obtain the color contrast coefficient R m , y q , t k ) at the pixel position (x YIQ (x m , y q , t k ):

[0044]

[0045] Among them, I(x m , y q , t k ), Q(x m , y q , t k ) and Y(x m , y q , t k ) represent the pixel position (x m , y q , t kValues of the I, Q, and Y channels in the YIQ color space at the [[ID=]], and ∑Y represents the sum of the Y-channel values at all pixel positions in the current image frame.

[0046] The final output of the peripheral modulation module includes the motion direction contrast coefficient R dir (x m ,y q ,t k ) and the color contrast coefficient R YIQ (x m ,y q ,t k )'s final response coefficient is:

[0047] R(x m ,y q ,t k ) = w dir *R dir (x m ,y q ,t k )+(1 - w dir )*R YIQ (x m ,y q ,t k );

[0048] Among them, R(x m ,y q ,t k ) represents the final response coefficient at the pixel position (x m ,y q ,t k ) in the peripheral modulation module, and w dir represents the weight coefficient of the motion direction contrast. 1 - w dir represents the weight coefficient of the color contrast.

[0049] The fusion value output by the information fusion module is expressed as:

[0050] S(x m ,y q ,t k ) = F(x m ,y q ,t k )·R(x m ,y q ,t k );

[0051] Among them, F(x m ,y q ,t k ) represents the enhanced smoke motion response information output by the spatio-temporal accumulation module, and R(x m ,yq , t k ) represents the final response coefficient output by the peripheral modulation module.

[0052] The detection method of a forest smoke detection system based on a moving target detection mechanism includes the following steps in sequence:

[0053] A: Use a video acquisition device to collect real-time videos of the forest in the monitoring area;

[0054] B: Use a brightness perception module to calculate the brightness information of each pixel in each frame of the video image and perform smoothing processing on the brightness information;

[0055] C: Use a motion detection module to calculate the ON - OFF signal of the brightness change of each pixel in each frame of the video image, and perform a delay process on the separated ON signal; then convolve the delay of the obtained ON signal with the OFF signal of the same pixel in the same frame of the video image to obtain the smoke motion response information D(x m , y q , t k ) at the position; m , y q , t k );

[0056] D: Use a spatio-temporal accumulation module to perform spatio-temporal information accumulation operations in eight motion directions, namely up, down, left, right, upper left, lower left, upper right, and lower right, through the motion information of pixels with spatio-temporal correlation in different frames of the video image, to obtain the motion information values in each motion direction; finally, select the maximum motion information value as the motion direction information at the position of the pixel, and the corresponding direction is the motion direction of the smoke target at the position of the pixel; if the motion information value in each direction is lower than the set motion information threshold E e , it is considered that there is no smoke target at the position of the pixel; finally, obtain the enhanced smoke motion response information F(x m , y q , t k ) at the position; m , y q , t k , d);

[0057] E: Use the motion direction modulation module and color modulation module in the peripheral modulation module to respectively obtain the motion direction contrast coefficient R m , y q , t k ) at the position; dir (x m , y q , t k ) and color contrast coefficient RYIQ (x m , y q , t k ), the final response coefficient is R(x m , y q , t k );

[0058] F: Using the information fusion module, the enhanced smoke motion response information R m , y q , t k ) at the position of the pixel point (x dir (x m , y q , t k ) and the final response coefficient R(x m , y q , t k ) are convolved to obtain the fusion value S(x m , y q , t k ). The maximum value in the fusion value is used as the fusion comparison value, and then the fusion comparison value is compared with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a smoke target in this frame of video image;

[0059] G: Using the information fusion module, based on the pixel point position corresponding to the corresponding fusion comparison value as the reference pixel point, calculate the fusion comparison values corresponding to each pixel point around the reference pixel point, and determine whether it is greater than 80% of the fusion comparison value of the reference pixel point. Then, the positions of the pixel points corresponding to all fusion comparison values greater than 80% and the position of the reference pixel are used as the appearance positions of the smoke target, and the appearance positions of the smoke target are sent to the display device;

[0060] H: Using the display device to identify and display the appearance position information of the smoke target, and give a reminder through the alarm module.

[0061] The present invention calculates the brightness information of each pixel in each frame of video image through a brightness perception module, then obtains the smoke movement response information of each pixel in each frame of video image under the condition of increased brightness through a motion detection module, and then uses a spatio-temporal accumulation module to obtain the enhanced smoke movement response information and motion direction information of the moving target through the smoke movement response information of pixels with spatio-temporal correlation in different frames of video images; at the same time, the present invention also obtains the final response coefficient for adjusting the enhanced smoke movement response information through a peripheral modulation module, and after integrating the enhanced smoke movement response information with the final response coefficient through an information fusion module, outputs the position information of smoke occurrence in the video. Based on the design idea of spatio-temporal accumulation-dynamic modulation, the present invention can detect smoke targets more accurately and reliably in natural image backgrounds and real natural videos. The present invention does not require training samples and the implementation process is simple, and can accurately and effectively detect smoke targets in a forest fire monitoring system, achieving the effect of early warning. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic flow chart of a smoke detection method based on a moving target detection mechanism in the present invention;

[0063] Figure 2 It is a schematic diagram of the principle of the signal processing process of the motion detection module in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0064] The present invention will be described in detail below with reference to the drawings and embodiments:

[0065] As Figure 1 and Figure 2 shown, the forest smoke detection system based on a moving target detection mechanism described in the present invention includes a video acquisition device, a data processing device, and a display device;

[0066] The video acquisition device is used to collect real-time videos of the forest in the monitoring area and convert the collected videos into digital signals and send them to the data processing device;

[0067] A data processing device is used to calculate the brightness information of each pixel in each frame of video image through a brightness perception module, then obtain the smoke motion response information of each pixel in each frame of video image under the condition of increasing brightness through a motion detection module, and then use a spatio-temporal accumulation module to obtain enhanced smoke motion response information and motion direction information through the smoke motion response information of pixels with spatio-temporal correlation in different frames of video images; the data processing device also obtains a final response coefficient for adjusting the enhanced smoke motion response information through a peripheral modulation module, and the final response coefficient includes a motion direction contrast coefficient and a color contrast coefficient. After integrating the enhanced smoke motion response information with the final response coefficient through an information fusion module, the position information of the smoke target in the video is output;

[0068] A display device is used to identify and display the position information of the smoke target output by the data processing device; the display device can use an existing display screen.

[0069] In the present invention, an alarm module can also be added in cooperation with the display device to warn of the appearance of a smoke target. The alarm module can use an existing audible and visual alarm device.

[0070] In the present invention, the video acquisition device can be carried on an aircraft, such as a drone. The video acquisition device includes a video perception module and a data transmission module; the video perception module can convert the optical signal of the acquired video image into a digital signal, and the data transmission module is used to transmit the digital signal output by the video perception module to the data processing device. In this embodiment, the video perception module can use a camera device carried on a drone or other aircraft; the data transmission module can use a wireless transmission device, such as a wireless signal transmitter.

[0071] In the present invention, the data processing device includes a brightness perception module, a motion detection module, a spatio-temporal accumulation module, a peripheral modulation module and an information fusion module;

[0072] The brightness perception module is used to calculate the brightness information of each pixel in each frame of the input video and perform smoothing processing on the brightness information;

[0073] The brightness perception module is used to simulate the neurons in the retinal layer of birds and can be represented by M×Q photoreceptors in a matrix form, where M and Q represent the size of each frame of video image. Each photoreceptor receives the brightness information of the pixel at the corresponding position in the video image and performs Gaussian blur processing (GB) to simulate the brightness perception and Gaussian processing mechanism of the retina.

[0074] In this embodiment, the brightness perception module can use a spatial domain Gaussian filter to smooth the brightness signal of each pixel in each frame of video image. The video image sequence that makes up the video is represented as The output P(x, y, t) of the brightness perception module is the convolution of the input I(x, y, t) and the Gaussian function ;

[0075]

[0076]

[0077] where x and y represent the abscissa and ordinate of the pixel points in the video image, t represents time, represents the set of real numbers, (u, v) represents the variable (x, y) in the integral with variable limits, and σ1 is the standard deviation of the Gaussian function;

[0078] The motion detection module is used to calculate the ON - OFF signal of the brightness change of each pixel point in each frame of the video image, and perform a delay process on the separated ON signal; then convolve the delay of the obtained ON signal with the OFF signal of the same pixel point in the same frame of the video image to obtain the smoke motion response information at the corresponding pixel point position;

[0079] Since in the forest background, for the smoke target with no fixed form, the motion response information (gray or light - gray moving smoke) in the case of increasing brightness is a relatively reliable smoke detection feature, so in this invention, only the motion response information in the case of increasing brightness is considered.

[0080] The motion detection module is used to simulate the superficial neurons of the optic tectum of birds, directly receive the signals output by the bird's retina layer, and can show strong responses to the decrease (OFF) and increase (ON) of brightness. In the motion detection module, the response generated by the neurons with the change of brightness can be calculated through a time - band - pass filter (HPF).

[0081] In this embodiment, considering the advantages of the gamma kernel in time processing, such as trivial stability, easy adaptability, and decoupling of the impulse response support region and order, the difference between two gamma kernels is used to define the impulse response of the time - band - pass filter H(t), that is

[0082]

[0083]

[0084] where Γ n,τ (t) represents an n - order gamma kernel function with a time constant of τ, and the subscripts in n1, τ1, n2, and τ2 respectively represent the first gamma kernel and the second gamma kernel; then the response output L(x, y, t) of the brightness change in the motion detection module is:

[0085] L(x, y, t) = ∫P(x, y, s)H(t - s)ds;

[0086] Among them, s represents the variable t in the integral with variable upper limit;

[0087] Subsequently, the ON signal and the OFF signal are separated:

[0088] S ON (x, y, t) = [L(x, y, t)] + ;

[0089] S OFF (x, y, t) = [-L(x, y, t)] + ;

[0090] Among them, S ON (x, y, t) and S OFF (x, y, t) respectively represent the positive and negative numbers of L(x, y, t), and are called the ON signal and the OFF signal, respectively reflecting the increase and decrease of the brightness change of each pixel point in each frame of video image; [L(x, y, t)] + means taking its positive value when L(x, y, t) is positive, and taking 0 when L(x, y, t) is non-positive; [-L(x, y, t)] + means taking 0 when L(x, y, t) is positive, and taking the absolute value of its value when L(x, y, t) is non-positive;

[0091] Subsequently, the motion detection module performs a delay process on the separated ON signal, that is, performs gamma convolution to obtain the delay of the ON signal which is:

[0092]

[0093] Among them, the subscripts in n3 and τ3 respectively represent the third gamma kernel, and s represents the variable t in the integral with variable upper limit;

[0094] Generally, the motion response information of the target includes two cases of brightness increase and brightness decrease. However, in the field of smoke detection, since smoke is mostly light gray and white, and the smoke in the forest background is biased towards white gray value, and the motion in the case of brightness decrease is mostly caused by black targets. Therefore, in the present invention, only the smoke motion response information in the case of brightness increase is considered. The smoke motion response information D(x, y, t) at the pixel point (x, y, t) is:

[0095] D(x, y, t) = S OFF (x, y, t) · S D-ON (x, y, t);

[0096] To understand the calculation strategy of the motion detection module simply and intuitively, such asFigure 2 As shown, taking a white moving target as an example, the signal processing process in the motion detection module is demonstrated. When the white target passes through the position (x0, y0), the luminance signal at the pixel position (x0, y0) in the luminance perception module changes with time as shown in Figure 2 step (a) in the figure; the luminance change signal generated in the motion detection module with the change of luminance is as shown in Figure 2 step (b) in the figure. This luminance change is caused by the arrival and departure of the white target. Therefore, the luminance change signal shows a trend of first increasing and then decreasing. Subsequently, the luminance change signal is separated into an ON signal and an OFF signal as shown in Figure 2 step (c) in the figure. The arrival and departure of the white target at the pixel position (x0, y0) occur in different video image frames, that is, the peaks of the OFF signal and the ON signal appear in different video image frames. In order to be able to detect the passage of the smoke target in the same video image frame, we perform delay processing on the ON signal and the OFF signal respectively. We show the ON signal, the OFF signal, and the delayed signal of the ON signal in Figure 2 step (c) in the figure. The first row of sub - figures is the OFF signal, the long dash line in the first row of sub - figures is the ON signal, and the short dash line is the delayed signal of the ON signal. As can be seen from Figure 2 the figure, the peak of the OFF signal and the peak of the delayed signal of the ON signal appear in the same video image frame. Finally, the OFF signal and the delayed signal of the ON signal are convolved to obtain the smoke motion response information at the pixel position (x0, y0) as shown in Figure 2 step (d) in the figure.

[0097] The spatio - temporal accumulation module accumulates spatio - temporal information through the motion information of pixels with spatio - temporal correlation in different frames of video images to obtain the motion information values in each motion direction. The motion information value is the enhanced smoke motion response information. Considering that the shape of the smoke target is not fixed and the motion direction is uncertain, in the present invention, accumulation operations are performed in eight motion directions: up, down, left, right, upper left, lower left, upper right, and lower right; finally, the maximum motion information value is selected as the motion direction information at this pixel position, and the corresponding direction is the motion direction of the smoke target at this pixel position; if the motion information value in each direction is lower than the set motion information threshold E e , it is considered that there is no smoke target at this pixel position. Among them, pixels with spatio - temporal correlation refer to the pixels at the positions where the smoke target appears in a continuous number of video image frames.

[0098] In the present invention, the spatio - temporal accumulation module is used to simulate the deep layer of the optic tectum of birds.

[0099] In this embodiment, the pixel position (x m , yq ,t k ) The motion information values in each motion direction are as follows:

[0100] The motion information value in the rightward motion direction is:

[0101]

[0102] The motion information value in the leftward motion direction is:

[0103]

[0104] The motion information value in the downward motion direction is:

[0105]

[0106] The motion information value in the upward motion direction is:

[0107]

[0108] The motion information value in the upper leftward motion direction is:

[0109]

[0110] The motion information value in the lower leftward motion direction is:

[0111]

[0112] The motion information value in the upper rightward motion direction is:

[0113]

[0114] The motion information value in the lower rightward motion direction is:

[0115]

[0116] Among them, x m , y q , t k , x m±i , y q±i , t k-i The subscript values of represent the position coordinates of the pixel points, M and Q represent the sizes of each video image frame. min(M - m, Q - q) represents the minimum value of M - m and Q - q; min(m - 1, Q - q) represents the minimum value of m - 1 and Q - q; min(M - m, q - 1) represents the minimum value of M - m and q - 1; min(m - 1, q - 1) represents the minimum value of m - 1 and q - 1; W i represents the cumulative weight value of the current video image frame.

[0117] Finally, the motion information value E(x, y, t, d) at the position of a single pixel in each frame of the video image is obtained, where d represents the direction. During the accumulation operation, the accumulation weight W i is represented by an exponential function with the mathematical constant e as the base and a value range of (0, 1).

[0118] For a certain pixel in each frame of the video image, its motion information value includes eight values (representing the responses in eight motion directions respectively). The maximum motion information value is selected as the motion direction information at the position of this pixel, and at the same time, the motion direction corresponding to the maximum motion information value is recorded as the motion direction of the smoke target at the position of this pixel. If the motion information values in all eight directions are lower than the set motion information threshold E e , it is considered that there is no smoke target at the position of this pixel.

[0119] The motion information value (enhanced smoke motion response information) at the position of the pixel (x m , y q , t k ) is as follows:

[0120]

[0121] When dir = 0, it indicates that there is no smoke target at the position of this pixel; dir = 1, 2, 3, 4, 5, 6, 7, and 8 respectively represent the motion directions of right, left, down, up, upper left, lower left, upper right, and lower right; d represents the motion direction at the position of the pixel (x m , y q , t k ).

[0122] The peripheral modulation module is used to obtain the final response coefficient for adjusting the enhanced smoke motion response information through division normalization operation. The final response coefficient is composed of motion direction contrast coefficients and color contrast coefficients with different weights;

[0123] In the present invention, the peripheral modulation module simulates the peripheral modulation mechanism of the optic tectum neurons, that is, the response of neurons to the stimuli within the receptive field will be modulated by the stimuli outside the receptive field. This modulation effect is positively correlated with the center - surround contrast intensity and is also related to the state attributes (static or moving) of the target.

[0124] We model the peripheral modulation effect of the visual cortex as a process of mutual inhibition of neurons to the same detection features. That is, a detection feature that can induce neuron excitation will inhibit the neuron excitation induced by the same detection features around it. The response coefficient is achieved through a biologically reasonable divisive normalization method. In the divisive normalization model, the neuron response is described as the ratio between the excitatory input of the neuron and the normalization signal. That is to say, the response of a neuron depends on the stimuli within the receptive field and the overall response induced by all visual stimuli within its modulation field.

[0125] Therefore, the normalized response R of neuron j can be obtained j depending on its unnormalized input B k The normalization is defined as:

[0126]

[0127]

[0128] where the numerator B j is the specific stimulus input of neuron j, generally the weighted sum of specific stimulus inputs, where w f represents the weighted value of neuron j receiving the f-th stimulus, I f represents the specific stimulus received by the neuron, which can come from external stimuli (such as vision, audition, olfaction, and gustation, etc.), or from other neurons, where the subscript f of I f represents the number of specific stimuli, and γ represents a constant; the denominator is the sum of a constant σ and all input stimuli where B h represents all stimulus inputs of neuron j, including specific stimuli and other stimuli.

[0129] In the present invention, the peripheral modulation module includes a motion direction modulation module and a color modulation module;

[0130] The motion direction modulation module is used to obtain the motion direction contrast coefficient R m ,y q ,t k ) at the pixel position (x dir (x m ,y q ,t k )

[0131]

[0132] where N dir (x m ,y q ,t k ) represents the pixel (x m ,yq , t k ), the frequency of occurrence of the motion direction at the position in the current video image represents the sum of the frequencies of occurrence of all motion directions. There are 9 groups of directions in total. d = 0 indicates no motion, and d = 1, 2, 3, 4, 5, 6, 7, 8 respectively indicate that the motion directions are right, left, down, up, upper left, lower left, upper right, and lower right;

[0133] The color modulation module is used to obtain the color contrast coefficient R at the pixel position (x m , y q , t k ); Combining the color characteristics of the smoke target, that is, the color is mainly distributed in the range from light gray to gray, and from white to blue - white. The YIQ color space has both color and brightness information and can better distinguish the smoke target in the forest background. In the YIQ color space, the Y value represents the gray - scale value of the image, and the larger the Y value, the closer it is to the gray - scale value of the smoke. I and Q represent hue information. The values of the I and Q channels of the smoke target are relatively close. Therefore, R YIQ (x m , y q , t k ) is defined as: YIQ (x m , y q , t k ) is defined as:

[0134]

[0135] Among them, I(x m , y q , t k ), Q(x m , y q , t k ) and Y(x m , y q , t k ) represent the values of the I, Q, and Y channels in the YIQ color space at the pixel position (x m , y q , t k ), and ∑Y represents the sum of the Y - channel values at all pixel positions in the current image frame.

[0136] After the above division normalization operation, the maximum values of the motion direction contrast coefficient and the color contrast coefficient are both 1. In this calculation method, if a certain feature appears frequently, then the response coefficients at all pixel points corresponding to this feature after division normalization will be relatively small. Similarly, if a certain feature appears rarely, then the response coefficients at all pixel points corresponding to this feature after division normalization will be relatively large. Therefore, this process simulates the peripheral modulation effect in the optic tectum neurons.

[0137] Finally, in order to prevent some weak signals that are hardly perceptible by biological vision from being extremely amplified during this processing, in the present invention, for features with a frequency of occurrence of the motion direction less than 10, they are set to zero after the division normalization calculation. Therefore, the final response coefficients of the peripheral modulation module, including the motion direction contrast coefficient R dir (x m ,y q ,t k ) and the color contrast coefficient R YIQ (x m ,y q ,t k ) are:

[0138] R(x m ,y q ,t k ) = w dir *R dir (x m ,y q ,t k ) + (1 - w dir )*R YIQ (x m ,y q ,t k );

[0139] Among them, R(x m ,y q ,t k ) represents the final response coefficient at the position of the pixel point (x m ,y q ,t k ) in the peripheral modulation module, and w dir represents the weight coefficient of the motion direction contrast. 1 - w dir represents the weight coefficient of the color contrast;

[0140] In this embodiment, when there is motion information, let the weight coefficient of the motion direction contrast be w dir = 0.8, then the weight coefficient of the color contrast is 1 - w dir = 1 - 0.8 = 0.2. That is

[0141]

[0142] An information fusion module, which is used to integrate the enhanced smoke motion response information output by the spatio-temporal accumulation module and the final response coefficient output by the peripheral modulation module, that is, to convolve the enhanced smoke motion response information and the final response coefficient from the same pixel position to obtain a fusion value, take the maximum value in the fusion value as the fusion comparison value, and then compare the fusion comparison value with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a smoke target in this frame of video image.

[0143] When there is a smoke target in this frame of video image, the information fusion module also uses the pixel position corresponding to the corresponding fusion comparison value as the reference pixel point, calculates the fusion comparison values corresponding to each pixel point around the reference pixel point, and determines whether it is greater than the set comparison threshold of the fusion comparison value of the reference pixel point, and then takes the positions of the pixels corresponding to all the fusion comparison values greater than the set comparison threshold and the reference pixel as the appearance position of the smoke target.

[0144] In this embodiment, the set comparison threshold is 80%;

[0145] The fusion value output by the information fusion module is expressed as:

[0146] S(x m ,y q ,t k ) = F(x m ,y q ,t k ) · R(x m ,y q ,t k );

[0147] Among them, F(x m ,y q ,t k ) represents the enhanced smoke motion response information output by the spatio-temporal accumulation module, and R(x m ,y q ,t k ) represents the final response coefficient output by the peripheral modulation module.

[0148] After this calculation, the fusion value of each frame of video image output by the present invention is a two-dimensional matrix (x m ,y q)。In the performance functional test of the computational model simulating insect vision, multiple target positions are usually detected in each frame. Although this method can achieve a high accuracy rate, it will result in a particularly high error rate. Moreover, this method does not conform to the functional characteristics of the pigeon optic tectum. Existing literature has shown that the pigeon optic tectum and the isthmic nucleus together form the midbrain saliency network, which is responsible for calculating the highest-priority stimuli and outputting them through the tectum. Here, we regard the smoke target as the highest-priority stimulus in the field of view. Therefore, only one target position is output in each frame of the image.

[0149] Therefore, in the present invention, a detection threshold is set. If the two-dimensional matrix (x m , y q ) has a maximum value greater than the set motion detection threshold, it is considered that there is a smoke target in the video image frame corresponding to this maximum value. Otherwise, it is considered that there is no smoke target in this video image frame.

[0150] When there is a smoke target in this frame of the video image, further determine the position corresponding to the maximum value in the image video frame as the reference position of the smoke target detected in the current frame, that is, the reference pixel point, and regard the area around the reference pixel point that is not lower than the set comparison threshold (80%) of the fusion comparison value at the reference pixel point as the detected smoke target area.

[0151] As Figure 1 shown, the forest smoke detection method based on the moving target detection mechanism described in the present invention successively includes the following steps:

[0152] A: Use a drone or other aircraft to carry a video acquisition device to acquire real-time videos of the forest in the monitoring area;

[0153] B: Use the brightness perception module to calculate the brightness information of each pixel point in each frame of the video image and perform smoothing processing on the brightness information;

[0154] C: Use the motion detection module to calculate the ON-OFF signal of the brightness change of each pixel point in each frame of the video image, and perform delay processing on the separated ON signal; then convolve the delay of the obtained ON signal with the OFF signal of the same pixel point in the same frame of the video image to obtain the smoke motion response information D(x m , y q , t k ) at the position of the pixel point (x m , y q , t k ), D(x m , y q , t k ) = S OFF (x m , y q , tk )·S D-ON (x m ,y q ,t k );

[0155] D: Use the spatio-temporal accumulation module to perform spatio-temporal information accumulation operations in eight motion directions: up, down, left, right, upper left, lower left, upper right, and lower right, respectively, based on the motion information of pixel points with spatio-temporal correlation in different frame video images, to obtain the motion information values in each motion direction; finally, select the maximum motion information value as the motion direction information at the position of this pixel point, and the corresponding direction is the motion direction of the smoke target at the position of this pixel point; if the motion information value in each direction is lower than the set motion information threshold E e , it is considered that there is no smoke target at the position of this pixel point; finally, obtain the enhanced smoke motion response information F(x m ,y q ,t k ) at the position of the corresponding pixel point; m ,y q ,t k ,d);

[0156] E: Use the motion direction modulation module and color modulation module in the peripheral modulation module to obtain the motion direction contrast coefficient R m (x q ,y k ) and color contrast coefficient R dir (x m ,y q ,t k ) at the position of the corresponding pixel point, and obtain the final response coefficient as R(x YIQ (x m ,y q ,t k ) at the position of the corresponding pixel point; m ,y q ,t k );

[0157] F: Use the information fusion module to perform convolution on the enhanced smoke motion response information R m (x q ,y k ) and the final response coefficient R(x dir (x m ,y q ,t k ) at the position of the corresponding pixel point to obtain the fusion value S(x m ,y q ,t k ) at the position of the corresponding pixel point; m ,y q ,tk ) Take the maximum value in the fusion value as the fusion comparison value, and then compare the fusion comparison value with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a smoke target in this frame of video image;

[0158] G: Using the information fusion module, take the pixel position corresponding to the corresponding fusion comparison value as the reference pixel point, calculate the fusion comparison values corresponding to each pixel point around the reference pixel point, and determine whether they are greater than the set comparison threshold of the fusion comparison value of the reference pixel point. Then, take the positions of the pixel points corresponding to all the fusion comparison values greater than the set comparison threshold and the position of the reference pixel as the appearance position of the smoke target, and send the appearance position of the smoke target to the display device;

[0159] H: Use the display device to identify and display the appearance position information of the smoke target, and give a reminder through the alarm module.

Claims

1. A forest smoke detection system based on a moving target detection mechanism, characterized in that: It includes a video acquisition device, a data processing device, and a display device; The video acquisition device is used to acquire the real-time video of the forest in the monitoring area and convert the acquired video into a digital signal and send it to the data processing device; The data processing device is used to calculate the brightness information of each pixel in each frame of the video image through the brightness perception module, and then obtain the smoke movement response information of each pixel in each frame of the video image under the condition of increased brightness through the motion detection module. Subsequently, the spatio-temporal accumulation module uses the smoke movement response information of the pixels with spatio-temporal correlation in different frames of the video image to obtain the enhanced smoke movement response information and the movement direction information; the data processing device also obtains the final response coefficient for adjusting the enhanced smoke movement response information through the peripheral modulation module. The final response coefficient includes the movement direction contrast coefficient and the color contrast coefficient. After the information fusion module integrates the enhanced smoke movement response information and the final response coefficient, it outputs the position information of the smoke target in the video; The display device is used to identify and display the position information of the smoke target output by the data processing device; Among them, the data processing device includes a brightness perception module, a motion detection module, a spatio-temporal accumulation module, a peripheral modulation module, and an information fusion module; The brightness perception module is used to calculate the brightness information of each pixel in each frame of the input video and smooth the brightness information; The motion detection module is used to calculate the ON-OFF signal of the brightness change of each pixel in each frame of the video image and delay the separated ON signal; then convolve the delay of the obtained ON signal with the OFF signal of the same pixel in the same frame of the video image to obtain the smoke movement response information at the corresponding pixel position; A spatio-temporal accumulation module is configured to perform spatio-temporal information accumulation operations in eight motion directions, namely up, down, left, right, upper left, lower left, upper right, and lower right, through the motion information of pixels with spatio-temporal correlation in different frame video images, so as to obtain the motion information values in each motion direction. The motion information value is the enhanced smoke motion response information. Finally, the maximum motion information value is selected as the motion direction information at the position of the pixel, and the corresponding direction is the motion direction of the moving target at the position of the pixel. If the motion information value in each direction is lower than the set motion information threshold E e , it is considered that there is no moving target at the position of the pixel; The peripheral modulation module is used to obtain the final response coefficient for adjusting the enhanced smoke movement response information through division normalization operation. The final response coefficient includes the movement direction contrast coefficient and the color contrast coefficient; The information fusion module is used to integrate the enhanced smoke movement response information output by the spatio-temporal accumulation module and the final response coefficient output by the peripheral modulation module, that is, convolve the enhanced smoke movement response information and the final response coefficient from the same pixel position to obtain a fusion value, take the maximum value in the fusion value as the fusion comparison value, and then compare the fusion comparison value with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a moving target in this frame of the video image.

2. The forest smoke detection system based on a moving target detection mechanism according to claim 1, wherein: When there is a smoke target in this frame of the video image, the information fusion module also uses the pixel position corresponding to the corresponding fusion comparison value as the reference pixel point, calculates the fusion comparison values corresponding to each pixel around the reference pixel point, and judges whether it is greater than the set comparison threshold of the fusion comparison value of the reference pixel point. Then, the positions of the pixels corresponding to all the fusion comparison values greater than the set comparison threshold and the reference pixel are used as the appearance positions of the smoke target.

3. The forest smoke detection system based on a moving target detection mechanism according to claim 1, characterized in that: When the spatio-temporal accumulation module obtains the motion information values in each motion direction, the motion information values of the pixel point position (x m , y q , t k ) in each motion direction are as follows: The motion information value in the rightward movement direction is: The motion information value in the leftward movement direction is: The motion information value in the downward movement direction is: The motion information value in the upward motion direction is: The motion information value in the upper left motion direction is: The motion information value in the lower left motion direction is: The motion information value in the upper right motion direction is: The motion information value in the lower right motion direction is: where x m , y q , t k , x m±i , y q±i , t k-i The subscript values of represent the position coordinates of the pixel points, M and Q represent the sizes of each video image frame; min(M - m, Q - q) represents the minimum value of M - m and Q - q; min(m - 1, Q - q) represents the minimum value of m - 1 and Q - q; min(M - m, q - 1) represents the minimum value of M - m and q - 1; min(m - 1, q - 1) represents the minimum value of m - 1 and q - 1; W i represents the cumulative weight value of the current video image frame; D(x, y, t) = S OFF (x, y, t) · S D-ON (x, y, t); S OFF (x, y, t) represents the negative of L(x, y, t), where L(x, y, t) is the response output of the brightness change in the motion detection module, and S D-ON (x, y, t) is the delay of the ON signal.

4. The forest smoke detection system based on a moving target detection mechanism according to claim 1, characterized in that: In the space-time accumulation module described above, the maximum motion information value is selected as the motion direction information at the position of the pixel point, and at the same time, the motion direction corresponding to the maximum motion information value is recorded as the motion direction of the moving target at the position of the pixel point; if the motion information values in all eight directions are lower than the set motion information threshold E e , it is considered that there is no moving target at the position of the pixel point; The enhanced smoke motion response information at the pixel point (x m , y q , t k ) is as follows: When dir = 0, it indicates that there is no moving target at the position of the pixel point; dir = 1, 2, 3, 4, 5, 6, 7, and 8 respectively indicate that the moving directions are right, left, down, up, upper left, lower left, upper right, and lower right; d represents the moving direction at the position of the pixel point (x m , y q , t k ).

5. The forest smoke detection system based on a moving target detection mechanism according to claim 1, wherein: The peripheral modulation module described above includes a motion direction modulation module; The motion direction modulation module is used to obtain the motion direction contrast coefficient R m , y q , t k ) at the pixel point position (x dir (x m , y q , t k ); Among them, N dir (x m , y q , t k ) represents the frequency of occurrence of the motion direction at the position of the pixel point (x m , y q , t k ) in the current video image. represents the sum of the frequencies of occurrence of all motion directions. There are a total of 9 groups of directions. d = 0 indicates no motion, and d = 1, 2, 3, 4, 5, 6, 7, 8 respectively indicate that the motion directions are right, left, down, up, upper left, lower left, upper right, and lower right.

6. The forest smoke detection system based on a moving target detection mechanism according to claim 5, characterized in that: The peripheral modulation module described above further includes a color modulation module; The color modulation module is used to obtain the color contrast coefficient R at the pixel point position (x m , y q , t k ): YIQ (x m , y q , t k ) where, I(x m , y q , t k ), Q(x m , y q , t k ), and Y(x m , y q , t k ) represent the values of the I, Q, and Y channels in the YIQ color space at the position of the pixel point (x m , y q , t k ), and ∑Y represents the sum of the Y channel values at the positions of all pixel points in the current image frame.

7. The forest smoke detection system based on the moving target detection mechanism according to claim 6, characterized in that: The final output of the peripheral modulation module includes the motion direction contrast coefficient R dir (x m ,y q ,t k ) and the color contrast coefficient R YIQ (x m ,y q ,t k ) and the final response coefficient is as follows: R(x m ,y q ,t k ) = w dir *R dir (x m ,y q ,t k )+(1 - w dir )*R YIQ (x m ,y q ,t k ); Among them, R(x m , y q , t k ) represents the final response coefficient at the position of the pixel point (x m , y q , t k ) in the peripheral modulation module, w dir represents the weight coefficient of the motion direction contrast, and 1 - w dir represents the weight coefficient of the color contrast.

8. The forest smoke detection system based on a moving target detection mechanism according to claim 1, wherein The fusion value output by the information fusion module is expressed as: S(x m ,y q ,t k ) = F(x m ,y q ,t k ) · R(x m ,y q ,t k ); Among them, F(x m , y q , t k ) represents the enhanced smoke motion response information output by the spatio-temporal accumulation module, and R(x m , y q , t k ) represents the final response coefficient output by the peripheral modulation module.

9. The detection method of the forest smoke detection system based on the moving target detection mechanism according to any one of claims 1 to 8, characterized in that: It successively includes the following steps: A: Use a video acquisition device to acquire real-time video of the forest in the monitoring area; B: Use a brightness perception module to calculate the brightness information of each pixel point in each frame of the video image and perform smoothing processing on the brightness information; C: Calculate the ON-OFF signal of the brightness change of each pixel in each frame of the video image using the motion detection module, and perform a delay process on the separated ON signal; then convolve the delay of the obtained ON signal with the OFF signal of the same pixel in the same frame of the video image to obtain the smoke motion response information D(x m ,y q ,t k ) at the position of (x m ,y q ,t k ); D: Using the spatio-temporal accumulation module, based on the motion information of pixels with spatio-temporal correlation in different frame video images, perform spatio-temporal information accumulation operations in eight motion directions: up, down, left, right, upper left, lower left, upper right, and lower right, to obtain the motion information values in each motion direction; finally, select the maximum motion information value as the motion direction information at the position of this pixel, and the corresponding direction is the motion direction of the smoke target at the position of this pixel; if the motion information value in each direction is lower than the set motion information threshold E e , it is considered that there is no smoke target at the position of this pixel; finally, obtain the enhanced smoke motion response information F(x m , y q , t k ) at the position; m , y q , t k , d); E: By using the motion direction modulation module and the color modulation module in the peripheral modulation module, respectively obtain the motion direction contrast coefficient R m , y q , t k ) at the position of the corresponding pixel point (x dir (x m , y q , t k ) and the color contrast coefficient R YIQ (x m , y q , t k ), and obtain the final response coefficient as R(x m , y q , t k ); F: Using the information fusion module, the enhanced smoke motion response information R m , y q , t k ) at the position of (x dir (x m , y q , t k ) and the final response coefficient R(x m , y q , t k ) are convolved to obtain the fusion value S(x m , y q , t k ). The maximum value in the fusion value is used as the fusion comparison value, and then the fusion comparison value is compared with the set motion detection threshold. If the fusion comparison value is greater than the motion detection threshold, it is considered that there is a smoke target in the frame of the video image; G: Use the information fusion module. Taking the pixel point position corresponding to the corresponding fusion comparison value as the reference pixel point, calculate the fusion comparison values corresponding to each pixel point around the reference pixel point, and determine whether it is greater than the set comparison threshold of the fusion comparison value of the reference pixel point. Then, take the positions of the pixel points corresponding to all the fusion comparison values greater than the set comparison threshold and the position of the reference pixel as the appearance position of the smoke target, and send the appearance position of the smoke target to the display device; H: Use the display device to identify and display the appearance position information of the smoke target, and give a reminder through the alarm module.

Images