A construction environment safety intelligent monitoring method and system

Abstract

The present invention relates to the field of intelligent fire monitoring technology, and specifically to a method and system for intelligent monitoring of construction environment safety. The present invention obtains a first fire possibility in the area to be monitored based on the texture features of the smoke area in the grayscale images of all continuous target video frames; obtains a second fire possibility in the area to be monitored based on the movement of the smoke area between the grayscale images of adjacent target video frames; obtains a third fire possibility in the area to be monitored based on the area change and texture change of the smoke area between the grayscale images of adjacent target video frames, combined with the duration of the smoke; and then evaluates the fire possibility in the area to be monitored for safety monitoring. Based on the distinguishing texture features and distinguishing motion features between fire smoke and non-fire smoke, the present invention analyzes the possibility that the smoke area in the grayscale images of continuous target video frames is fire smoke, reduces the interference of other non-fire smoke in the construction site, and improves the accuracy of intelligent fire safety monitoring of the construction site.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent fire monitoring, and in particular to a method and system for intelligent monitoring of construction environment safety. Background Art

[0002] Construction environments involve a complex mix of materials and electrical wiring, and some equipment involves open flames, which can easily cause fires. Therefore, safety supervision on construction sites is crucial for ensuring personnel safety, project quality, and maintaining a company's image. In recent years, technologies such as the Internet of Things and artificial intelligence have rapidly advanced. Industrial visual intelligent monitoring is gradually replacing manual inspections and oversight. Fire detection and prevention mechanisms based on industrial visual intelligence are being promoted and applied, significantly improving the efficiency and accuracy of construction environment safety supervision.

[0003] However, fire smoke detection in construction environments faces many challenges. There are other smoke sources in the construction site, such as dust or flying wood chips. Fire monitoring based on industrial visual intelligence will be interfered with by other smoke and prone to false alarms, which increases the difficulty of fire monitoring and early warning, and thus leads to poor safety and intelligent monitoring of the construction site. Summary of the Invention

[0004] In order to solve the technical problem that the existing technology has poor effect on safety intelligent monitoring in construction environments, the purpose of the present invention is to provide a method and system for safety intelligent monitoring of construction environments. The technical solutions adopted are as follows:

[0005] The present invention provides a construction environment safety intelligent monitoring method, the method comprising:

[0006] Obtaining continuous target video frame grayscale images in the surveillance video of the area to be monitored at the construction site, and obtaining the smoke area in each of the target video frame grayscale images;

[0007] Obtain a first fire probability of the monitored area based on the texture features of the smoke area in all the grayscale images of the target video frames; obtain a second fire probability of the monitored area based on the movement of the smoke area between adjacent grayscale images of the target video frames; obtain a third fire probability of the monitored area based on the area change and texture change of the smoke area between adjacent grayscale images of the target video frames, combined with the duration of the smoke in the monitoring video;

[0008] The first fire possibility, the second fire possibility and the third fire possibility are integrated to obtain the fire possibility of the area to be monitored; and safety monitoring is performed based on the fire possibility.

[0009] Furthermore, the method for obtaining the first fire possibility includes:

[0010] Obtaining a gray level co-occurrence matrix of the smoke region in the grayscale image of each target video frame; obtaining contrast, correlation, homogeneity and entropy of the gray level co-occurrence matrix;

[0011] Obtaining a fire smoke texture parameter of the smoke area in the grayscale image of each target video frame according to the contrast, the correlation, the homogeneity, and the entropy; wherein the contrast and the entropy are both positively correlated with the fire smoke texture parameter, and the correlation and the homogeneity are both negatively correlated with the fire smoke texture parameter;

[0012] The fire smoke texture parameters of the smoke area in all the target video frame grayscale images are combined to obtain a first fire possibility of the area to be monitored.

[0013] Furthermore, the method for obtaining the second fire possibility includes:

[0014] Obtaining a moving direction and a moving speed of the smoke region in each of the target video frame grayscale images according to a change difference of pixels in the smoke region between adjacent target video frame grayscale images;

[0015] Obtaining a smoke trajectory confusion coefficient in the area to be monitored by comprehensively analyzing the change uniformity of the movement direction and the movement speed of the smoke area between adjacent grayscale images of the target video frames;

[0016] Based on the moving speed of the smoke area in all the target video frame grayscale images, and the directional difference between the moving direction and the preset direction, combined with the smoke trajectory confusion coefficient, the second fire possibility of the monitored area is obtained; the moving speed is positively correlated with the second fire possibility, and the directional difference and the smoke trajectory confusion coefficient are both negatively correlated with the second fire possibility.

[0017] Furthermore, the method for obtaining the moving direction and the moving speed includes:

[0018] Obtaining a motion vector of each pixel in the smoke area in the grayscale image of each target video frame based on an optical flow method;

[0019] According to the motion vector of each pixel point, the moving speed and moving direction of the smoke area in each target video frame grayscale image are obtained.

[0020] Furthermore, the method for obtaining the smoke trajectory confusion coefficient includes:

[0021] Constructing a moving vector of the moving speed and the moving direction of the smoke area in each target video frame grayscale image;

[0022] Obtaining the Euclidean distance between the corresponding motion vectors between each target video frame grayscale image and the next adjacent target video frame grayscale image;

[0023] The corresponding Euclidean distances between all adjacent grayscale images of the target video frames are integrated to obtain the smoke trajectory confusion coefficient in the area to be monitored.

[0024] Furthermore, the method for obtaining the duration of the smoke includes:

[0025] The product of the appearance frequency of the smoke area in all target video frame grayscale images and the frame rate of the target video frame grayscale image is used as the smoke duration.

[0026] Furthermore, the method for obtaining the third fire possibility includes:

[0027] Between each target video frame grayscale image and the next adjacent target video frame grayscale image, the sum of the area change difference and the overall gradient change difference of the smoke region is used as the dissipation parameter of the smoke region between the adjacent target video frame grayscale images; the dissipation parameters of the smoke region between all adjacent target video frame grayscale images are combined to obtain the smoke dissipation coefficient of the area to be monitored;

[0028] The third fire possibility of the monitored area is obtained according to the smoke dissipation coefficient and the smoke duration; the smoke duration is positively correlated with the third fire possibility, and the smoke dissipation coefficient is negatively correlated with the third fire possibility.

[0029] Furthermore, the method for performing safety monitoring based on the fire possibility includes:

[0030] When the fire possibility is greater than a preset threshold, a safety warning is issued to the monitored area.

[0031] Furthermore, the preset direction is an upward direction perpendicular to the ground.

[0032] The present invention also proposes a construction environment safety intelligent monitoring system, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the steps of the construction environment safety intelligent monitoring method are implemented.

[0033] The present invention has the following beneficial effects:

[0034] The present invention obtains grayscale images of all consecutive target video frames in a surveillance video of a construction site to be monitored, and obtains the smoke area in each target video frame grayscale image for subsequent analysis of the texture characteristics and changes of the smoke area, thereby assessing the possibility of a fire in the monitored area. A first fire probability of the monitored area is obtained based on the texture characteristics of the smoke area in all target video frame grayscale images. A second fire probability of the monitored area is obtained based on the movement of the smoke area between adjacent target video frame grayscale images. A third fire probability of the monitored area is obtained based on the area and texture changes of the smoke area between adjacent target video frame grayscale images, combined with the duration of smoke in the surveillance video. The first fire probability, second fire probability, and third fire probability are combined to obtain the fire probability of the monitored area. Safety monitoring is performed based on the fire probability. The present invention analyzes the probability that the smoke area in the continuous target video frame grayscale image is fire smoke based on the distinguishing texture characteristics and distinguishing motion characteristics between fire smoke and non-fire smoke, thereby reducing the interference of other non-fire smoke in the construction site and improving the accuracy of industrial visual intelligence in intelligent fire safety monitoring of construction sites. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 A flowchart of a construction environment safety intelligent monitoring method provided by one embodiment of the present invention;

[0037] Figure 2 A flow chart of a method for obtaining a first fire probability provided by one embodiment of the present invention;

[0038] Figure 3 A flow chart of a method for obtaining a second fire probability provided by one embodiment of the present invention;

[0039] Figure 4 This is a flow chart of a method for obtaining the third fire possibility provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0040] To further illustrate the technical means and effectiveness of the present invention in achieving its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a method and system for intelligently monitoring construction environment safety proposed by the present invention. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0041] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0042] The specific scheme of the construction environment safety intelligent monitoring method and system provided by the present invention is described in detail below with reference to the accompanying drawings.

[0043] See also Figure 1 , which shows a flow chart of a construction environment safety intelligent monitoring method provided by one embodiment of the present invention, specifically including:

[0044] Step S1: Obtain all continuous target video frame grayscale images in the monitoring video of the area to be monitored at the construction site, and obtain the smoke area in each target video frame grayscale image.

[0045] In one embodiment of the present invention, high-definition cameras are installed at the construction site to monitor the flammable areas of the construction site in real time. Taking one high-definition camera monitoring one flammable area as an example, the present invention performs corresponding frame segmentation and grayscale preprocessing on the real-time acquired monitoring video, and further extracts the smoke area in the grayscale image of each video frame for subsequent analysis of the texture characteristics and changes of the smoke area, so as to evaluate the possibility of it being fire smoke, and then evaluate the possibility of fire in each monitored area; in other embodiments of the present invention, the implementer can also use multiple cameras to monitor a flammable area, and comprehensively evaluate the possibility of fire in the monitored area based on the monitoring results of multiple cameras.

[0046] As an example, surveillance video is acquired in real time, wherein the frame rate of the surveillance video is set to 30fps, and all video frame images can be obtained. Then, the video frame images are grayscale processed and conventional denoising is performed to obtain grayscale images of all video frames. The trained convolutional neural network is used to identify the smoke area in the grayscale images of all video frames, and the moment when the smoke area is first identified is used as the time for early warning analysis. Within 10 seconds after the time for early warning analysis, all video frame grayscale images in the surveillance video of the monitored area are collected in real time, that is, 300 consecutive video frame grayscale images after the smoke area is first identified are used as continuous target video frame grayscale images for analysis and monitoring, and the smoke area in each video frame grayscale image is obtained.

[0047] It should be noted that grayscale processing and denoising, and the training and application of convolutional neural networks are already existing technologies well known to those skilled in the art and will not be described in detail here. In other embodiments of the present invention, the implementer may also adopt other neural network models, or set up multiple cameras to collect surveillance videos of the area to be monitored, fuse the grayscale images of multi-view target video frames to extract the smoke area, and may also directly perform optical flow analysis on the grayscale images of all target video frames to extract the smoke area. These are all existing technologies well known to those skilled in the art and will not be described in detail here.

[0048] It should be noted that the embodiment of the present invention monitors, analyzes and issues early warning within 10 seconds after smoke is detected in the monitored area; in the 300 consecutive target video frame grayscale images within these 10 seconds, it is possible that the smoke area is not identified in some target video frame grayscale images, that is, the smoke dissipates after a period of time, and then the subsequent target video frame grayscale images will no longer be analyzed, and only the target video frame grayscale images of the continuous smoke area will be intercepted and analyzed; in other embodiments of the present invention, the implementer may also set the total number of other continuous target video frame grayscale images, such as 300 frames to 1800 frames, that is, the monitoring time from the first identification of the smoke area to the current moment is between 10 seconds and 60 seconds. The monitoring period should not be too long, otherwise the near real-time monitoring effect cannot be achieved; it should not be too short, otherwise the characteristic changes of the smoke area will not be obvious, resulting in poor analysis effect.

[0049] Step S2: Obtain the first fire possibility of the area to be monitored based on the texture features of the smoke area in the grayscale images of all target video frames; obtain the second fire possibility of the area to be monitored based on the movement of the smoke area between adjacent target video frame grayscale images; obtain the third fire possibility of the area to be monitored based on the area change and texture change of the smoke area between adjacent target video frame grayscale images, combined with the duration of smoke in the monitoring video.

[0050] Since dust or wood chips may fly around at construction sites, other non-fire smoke may appear, which may interfere with fire monitoring during visual intelligent recognition. Considering that fire smoke has a more distinct texture due to its irregular morphology and the influence of firelight refraction and shadows, while dust smoke or wood chip smoke usually presents a mist-like morphology with a more blurred smoke texture boundary, the embodiment of the present invention analyzes the texture features of the smoke area in the grayscale image of all target video frames from a static perspective based on the distinguishing texture features of fire smoke and other interfering smoke, evaluates the possibility of it being fire smoke, and obtains the first fire probability of the area to be monitored.

[0051] Preferably, in one embodiment of the present invention, the method for obtaining the first fire possibility includes:

[0052] See also Figure 2, which shows a flow chart of a method for obtaining a first fire possibility provided by an embodiment of the present invention, specifically comprising:

[0053] Step S201: Obtain the gray level co-occurrence matrix of the smoke region in the gray level image of each target video frame; obtain the contrast, correlation, homogeneity and entropy of the gray level co-occurrence matrix.

[0054] Considering that the grayscale co-occurrence matrix is ​​a statistical method for describing image texture features, in one embodiment of the present invention, the grayscale co-occurrence matrix of the smoke area in the grayscale image of each target video frame is first obtained, wherein since the smoke area may be an irregular image, after extracting the smoke area, the implementer can set the grayscale value of the non-smoke area in the grayscale image of the target video frame to 0 as the background area to obtain the processed target video frame grayscale image, and further obtain the grayscale co-occurrence matrix of the processed target video frame grayscale image. Obtaining the grayscale co-occurrence matrix of the smoke area is an existing technology well known to those skilled in the art and will not be repeated here.

[0055] Considering that the grayscale variation of fire smoke is large, the contrast is usually high. Since the correlation feature in the grayscale co-occurrence matrix reflects the similarity of matrix elements in the row and column directions, it indirectly reflects the grayscale correlation of the smoke area. The lower the correlation, the more complex and chaotic the texture features of the smoke area are, and the more consistent with the texture features of fire smoke. The entropy feature is used to measure the distribution of grayscale levels. The larger the entropy, the more uneven and chaotic the smoke texture is, and the more consistent with the texture features of fire smoke. Homogeneity also reflects the texture uniformity of the smoke area. The smaller the homogeneity, the more consistent with the texture features of fire smoke. Therefore, in one embodiment of the present invention, the contrast, correlation, homogeneity, and entropy of the grayscale co-occurrence matrix are obtained to prepare for obtaining the fire smoke texture parameters of the smoke area in the grayscale image of each target video frame in the subsequent step S202.

[0056] It should be noted that the methods for obtaining features such as contrast, correlation, homogeneity and entropy are already existing technologies well known to those skilled in the art and will not be described in detail here; in other embodiments of the present invention, the implementer may also obtain features of other gray-level co-occurrence matrices to evaluate the uniformity of the texture and further evaluate the first fire possibility.

[0057] Step S202 , obtaining fire smoke texture parameters of the smoke area in the grayscale image of each target video frame based on contrast, correlation, homogeneity, and entropy; contrast and entropy are both positively correlated with the fire smoke texture parameters, while correlation and homogeneity are both negatively correlated with the fire smoke texture parameters.

[0058] As an example, the fire smoke texture parameters are calculated as follows: Among them, JT i,kis the fire smoke texture parameter of the smoke area in the grayscale image of the kth target video frame in the i-th monitoring video; CT i,k is the contrast of the grayscale co-occurrence matrix of the smoke area in the grayscale image of the kth target video frame in the i-th surveillance video; EN i,k is the entropy of the gray-level co-occurrence matrix of the smoke region in the grayscale image of the k-th target video frame in the i-th surveillance video; CL i,k Hom is the correlation of the gray-level co-occurrence matrix of the smoke region in the grayscale image of the k-th target video frame in the i-th surveillance video; i,k is the homogeneity of the gray-level co-occurrence matrix of the smoke region in the grayscale image of the k-th target video frame in the i-th surveillance video.

[0059] In the formula, addition or ratio is specifically used to represent the corresponding positive correlation or negative correlation; in other embodiments of the present invention, the implementer may also use other basic mathematical operations or related mapping means to represent the correlation, such as addition or subtraction, which will not be repeated here.

[0060] Step S203 , combining the fire smoke texture parameters of the smoke areas in the grayscale images of all target video frames to obtain a first fire possibility of the area to be monitored.

[0061] As an example, the mean of the fire smoke texture parameters of the smoke area in the grayscale images of all target video frames is normalized to obtain the first fire possibility of the monitored area; specifically, the maximum and minimum value normalization method is adopted. The implementer may also adopt other normalization methods, which will not be repeated here.

[0062] At this point, the first fire possibility of the area to be monitored in each surveillance video is obtained.

[0063] Fire smoke typically exhibits a rapid upward and drifting motion due to the heat released by burning materials, causing the surrounding air to rise. In contrast, dust or sawdust smoke may diffuse freely and directionlessly through the air. Therefore, the present invention uses the distinguishing motion characteristics of fire smoke and other interfering smoke to dynamically analyze the changes in the movement direction and speed of smoke regions between grayscale images of adjacent target video frames to determine the potential for a secondary fire in the monitored area.

[0064] Preferably, in one embodiment of the present invention, the method for obtaining the second fire probability includes:

[0065] See also Figure 3 , which shows a flow chart of a method for obtaining a second fire probability provided by an embodiment of the present invention, specifically comprising:

[0066] Step S301 , obtaining the moving direction and moving speed of the smoke area in each target video frame grayscale image according to the change difference of the pixels in the smoke area between adjacent target video frame grayscale images.

[0067] Considering the distinguishing motion characteristics between fire smoke and other smoke, the changes in the smoke area between the grayscale images of adjacent target video frames may also be inconsistent. Therefore, one embodiment of the present invention will evaluate the motion of the smoke area based on the change differences of the pixel points in the smoke area between the grayscale images of adjacent target video frames, and obtain the moving direction and speed of the smoke area in the grayscale image of each target video frame.

[0068] In a preferred embodiment of the present invention, considering that the optical flow method can accurately analyze the motion of objects in a video frame sequence, the method for obtaining the moving direction and moving speed includes:

[0069] Obtain the motion vector of each pixel in the smoke area of ​​each target video frame grayscale image based on the optical flow method;

[0070] Based on the motion vector of each pixel, the moving speed and direction of the smoke area in the grayscale image of each target video frame are obtained.

[0071] As an example, after obtaining the motion vector of each pixel based on the optical flow method, since the motion vector includes the horizontal component and vertical component of each pixel, the Euclidean norm can be used to obtain the displacement value of each pixel, and this displacement value approximately replaces its moving speed; the inverse tangent function can be used to obtain the displacement angle, that is, the moving direction, of each pixel; in the smoke area of ​​each target video frame grayscale image, the moving speed and moving direction of all pixels are averaged respectively to obtain the overall moving speed and moving direction of the smoke area in the target video frame grayscale image.

[0072] It should be noted that the optical flow method for obtaining motion vectors and the use of motion vectors to obtain speed and direction are existing technologies well known to those skilled in the art and will not be elaborated here. In other examples, the implementer can also obtain the mode of the moving speed and moving direction of all pixels respectively instead of calculating the mean, which will not be elaborated here.

[0073] In another embodiment of the present invention, the implementer may also detect and track feature points or feature descriptors in the smoke area in the video frame, match the feature points or descriptors in the smoke area between the grayscale images of adjacent target video frames, and estimate the movement direction and speed through the matching results. This is an existing technology well known to those skilled in the art and will not be described in detail here.

[0074] Step S302 , the smoke trajectory confusion coefficient in the area to be monitored is obtained by comprehensively analyzing the change uniformity of the moving direction and moving speed of the smoke area between the grayscale images of adjacent target video frames.

[0075] Considering that the movement direction and speed of the smoke area reflect the movement trajectory of the smoke area to a certain extent, since fire smoke usually spreads rapidly upward vertically from the ground, the movement trajectory is relatively consistent; while other interfering smoke usually diffuses freely without direction, and the movement trajectory is relatively chaotic; therefore, one embodiment of the present invention comprehensively considers the uniformity of changes in the movement direction and speed of the smoke area, estimates the smoke movement trajectory, and obtains the smoke trajectory confusion coefficient in the monitored area. The larger the smoke trajectory confusion coefficient, the less it conforms to the movement trajectory characteristics of fire smoke and the smaller the possibility of fire.

[0076] In a preferred embodiment of the present invention, considering that both the moving speed and the moving angle can reflect the movement trajectory of the smoke, a comprehensive analysis of the two is performed, and a method for obtaining the smoke trajectory confusion coefficient includes:

[0077] Construct the moving vector of the moving speed and moving direction of the smoke area in the grayscale image of each target video frame;

[0078] Obtain the Euclidean distance between the corresponding motion vectors between each target video frame grayscale image and the adjacent next target video frame grayscale image;

[0079] The corresponding Euclidean distances between the grayscale images of all adjacent target video frames are combined to obtain the smoke trajectory confusion coefficient in the monitored area.

[0080] As an example, the moving vector of the smoke area in the grayscale image of the t-th target video frame in the i-th surveillance video is (V i,t ,Q i,t ), where V i,t is the moving speed of the smoke area in the grayscale image of the t-th target video frame in the i-th surveillance video; Q i,t is the moving direction of the smoke area in the grayscale image of the tth target video frame in the i-th surveillance video; the mean of the corresponding Euclidean distances between all adjacent target video frame grayscale images is taken as the smoke trajectory confusion coefficient in the monitored area.

[0081] In another embodiment of the present invention, the direction sequence and speed sequence of the moving direction and moving speed of the smoke area can be obtained respectively according to the time sequence of the grayscale image of the target video frame, and the variance of the speed sequence and the direction sequence can be further obtained respectively, and the variances are averaged to obtain the smoke trajectory confusion coefficient; the corresponding curves of the direction sequence and the speed sequence can also be fitted, and whether the motion trajectory changes are chaotic can be evaluated according to the shape of the curve, and then the smoke trajectory confusion coefficient can be comprehensively obtained, which will not be repeated here.

[0082] In step S303, the second fire possibility of the monitored area is obtained based on the movement speed of the smoke area in the grayscale images of all target video frames, the difference between the movement direction and the preset direction, and the smoke trajectory confusion coefficient. The movement speed is positively correlated with the second fire possibility, while the direction difference and the smoke trajectory confusion coefficient are both negatively correlated with the second fire possibility.

[0083] In one embodiment of the present invention, considering that fire smoke rises vertically rapidly with hot air and its movement trajectory is relatively regular and consistent, the greater the moving speed of the smoke area, the closer the moving direction is to vertically upward from the ground, and the smaller the smoke trajectory confusion coefficient, the greater the possibility that it is fire smoke; based on this, the second fire probability is obtained.

[0084] As an example, the calculation formula for the second fire probability is: Among them, JD i is the second fire probability of the monitored area in the i-th surveillance video; norm[] is the standard normalization function; MT i is the total number of grayscale images of target video frames in the i-th surveillance video; V i,t is the moving speed of the smoke area in the grayscale image of the t-th target video frame in the i-th surveillance video; Q i,t is the moving direction of the smoke area in the grayscale image of the t-th target video frame in the i-th surveillance video; Q is the preset direction; C i is the smoke trajectory confusion coefficient of the monitored area in the i-th surveillance video.

[0085] In the above formula, the preset direction is the upward direction perpendicular to the ground, that is, Q is 90°; the average moving speed of the smoke area in the grayscale image of all target video frames reflects the moving speed of the smoke, and the average deviation of the moving angle relative to the vertical direction of the ground reflects whether the smoke rises vertically; the average moving speed is used as the numerator, and the product of the smoke trajectory confusion coefficient and the mean direction deviation is used as the denominator. The greater the moving speed, the greater the possibility of a second fire, and the smaller the smoke trajectory confusion coefficient and the mean direction deviation, the greater the possibility of a second fire. In other examples, implementers can also use other basic mathematical operations such as addition, subtraction and other related mapping methods to combine the three, which will not be repeated here.

[0086] At this point, the second fire possibility of the area to be monitored in each surveillance video is obtained.

[0087] Considering that fire combustion is a continuous process, fire smoke will gradually spread and expand, the smoke area will gradually increase and the smoke concentration will become higher and higher, making the texture features more and more irregular and complex. Non-fire smoke usually settles slowly and naturally, the mist-like features will usually gradually dissipate and the smoke area will not continue to expand. The texture features of the smoke area will gradually weaken or disappear; therefore, the embodiment of the present invention will obtain the third fire possibility of the monitored area based on the area change and texture change of the smoke area between the grayscale images of adjacent target video frames, combined with the duration of the smoke in the surveillance video.

[0088] Preferably, in one embodiment of the present invention, the method for obtaining the third fire probability includes:

[0089] See also Figure 4 , which shows a flow chart of a method for obtaining the third fire possibility provided by an embodiment of the present invention, specifically comprising:

[0090] In step S401, the sum of the area change difference and the overall gradient change difference of the smoke area between each target video frame grayscale image and the adjacent next target video frame grayscale image is used as the dissipation parameter of the smoke area between the adjacent target video frame grayscale images; and the dissipation parameters of the smoke area between all adjacent target video frame grayscale images are combined to obtain the smoke dissipation coefficient of the area to be monitored.

[0091] It should be noted that, in one embodiment of the present invention, the method for obtaining the overall gradient of the smoke area is: using the Sobel operator convolution to calculate the gradient size of each pixel point in the smoke area, and averaging the gradient sizes of all pixels in the smoke area to obtain the overall gradient of the entire smoke area; the larger the overall gradient, the more complex and obvious the texture features in the smoke area; in other embodiments of the present invention, the implementer may also use other operators to obtain the pixel gradient, which is an existing technology well known to those skilled in the art and will not be described here.

[0092] As an example, since fire smoke will spread and expand over time, and the overall texture features will become more and more complex and obvious, the overall gradient will become larger and larger. Then, the smoke area of ​​each target video frame grayscale image relative to the smoke area of ​​the adjacent next target video frame grayscale image, the area change difference and the overall gradient change difference will both be less than zero; on the contrary, non-fire smoke will gradually dissipate and decrease over time, and the overall texture features will further fade and disappear, the overall gradient will become smaller and smaller, and the area change difference and the overall gradient change difference will both be greater than zero; the sum of the area change difference and the overall gradient change difference of the smoke area indirectly reflects the smoke dissipation situation between the corresponding adjacent target video frame grayscale images. The larger the sum value, the larger the corresponding dissipation parameter, and the greater the possibility of non-fire smoke; then all dissipation parameters are averaged to obtain the smoke dissipation coefficient of the monitored area; the smaller the smoke dissipation coefficient, the greater the possibility of fire smoke.

[0093] In other examples, considering that in the method for obtaining the first fire possibility, the fire smoke texture parameter is also used to evaluate the possibility of fire smoke based on the texture characteristics of the fire smoke, and the texture characteristics of the initial fire smoke are relatively lighter than the texture characteristics of the later fire smoke, then the fire smoke texture parameter should become larger and larger as time changes; the difference in the fire smoke texture parameter between the grayscale image of each target video frame and the grayscale image of the next adjacent target video frame can be added and combined with the difference in the area change of the smoke area to obtain the dissipation parameter, and then the smoke dissipation coefficient of the area to be monitored can be obtained.

[0094] Step S402 , obtaining the third fire possibility of the monitored area according to the smoke dissipation coefficient and the smoke duration; the smoke duration is positively correlated with the third fire possibility, and the smoke dissipation coefficient is negatively correlated with the third fire possibility.

[0095] As an example, the smoke dissipation coefficient is subjected to negative correlation mapping adjustment logic, and then the negative correlation mapping result is multiplied and merged with the smoke duration, and then normalized to obtain the third fire possibility of the monitored area; wherein, the smoke dissipation coefficient is used as the x in exp(-x) to perform negative correlation mapping adjustment logic, exp() is an exponential function with the natural constant e as the base, and other negative correlation mapping methods can also be used; maximum and minimum value normalization is used, and other normalization methods can also be used; the larger the smoke dissipation coefficient, the smaller the possibility of fire smoke, and the longer the smoke duration, the greater the possibility of fire smoke.

[0096] Among them, the product of the frequency of occurrence of the smoke area in the grayscale images of all target video frames and the frame rate of the grayscale images of the target video frames is used as the duration of the smoke; the implementer can also directly use the frequency of occurrence of the smoke area in the grayscale images of all target video frames to approximate the duration of the smoke.

[0097] At this point, the third fire possibility of the area to be monitored in each surveillance video is obtained.

[0098] Step S3: The first fire possibility, the second fire possibility, and the third fire possibility are integrated to obtain the fire possibility of the area to be monitored; and safety monitoring is performed based on the fire possibility.

[0099] In one embodiment of the present invention, the first fire possibility, the second fire possibility and the third fire possibility are multiplied and combined to obtain the fire possibility of the monitored area in each surveillance video; in other embodiments, the implementer may also fuse the three through basic mathematical operations such as addition or weighted correlation and related mapping means, which will not be repeated here.

[0100] The greater the possibility of fire in each monitoring video, the greater the possibility of fire. In one embodiment of the present invention, a preset threshold is specifically set. When the preset threshold is exceeded, it is considered that a fire has occurred in the monitored area in the current monitoring video, and a safety warning is required for the monitored area. The implementer can also further integrate the fire possibility of all monitoring videos and evaluate the necessity of safety warnings. When more than a certain proportion, such as 60%, of the monitoring videos consider that a fire has occurred in the monitored area, a safety alarm is issued to warn the relevant persons in charge in the construction site to take safety fire-fighting measures.

[0101] In summary, the embodiment of the present invention obtains the first fire possibility of the area to be monitored based on the texture features of the smoke area in the grayscale images of all target video frames; obtains the second fire possibility of the area to be monitored based on the movement of the smoke area between adjacent target video frame grayscale images; obtains the third fire possibility of the area to be monitored based on the area change and texture change of the smoke area between adjacent target video frame grayscale images, combined with the duration of the smoke in the monitoring video; and then fuses the first fire possibility, the second fire possibility and the third fire possibility to obtain the fire possibility of the area to be monitored for safety monitoring. Based on the distinguishing texture features and distinguishing motion features between fire smoke and non-fire smoke, the present invention analyzes the possibility that the smoke area in the grayscale image of the target video frame is fire smoke, thereby reducing the interference of other non-fire smoke in the construction site and improving the accuracy of industrial visual intelligence in intelligent fire safety monitoring of the construction site.

[0102] The present invention also proposes a construction environment safety intelligent monitoring system, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, it implements the steps of a construction environment safety intelligent monitoring method described in steps S1 to S3.

[0103] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0104] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A construction environment safety intelligent monitoring method, characterized in that: The method comprises: Obtaining continuous target video frame grayscale images in the surveillance video of the area to be monitored at the construction site, and obtaining the smoke area in each of the target video frame grayscale images; Obtaining a first fire probability of the area to be monitored based on the texture features of the smoke area in the grayscale images of all the target video frames; the first fire probability is used to reflect the texture features of the smoke area in the grayscale images of all the target video frames from a static perspective; Between adjacent target video frame grayscale images, a second fire possibility of the area to be monitored is obtained based on the movement of the smoke area; the second fire possibility is used to reflect the changes in the moving direction and moving speed of the smoke area between adjacent target video frame grayscale images from a dynamic perspective; the second fire possibility is determined based on the size of the moving speed of the smoke area in all target video frame grayscale images, the directional difference between the moving direction of the smoke area in all target video frame grayscale images and a preset direction, and the smoke trajectory confusion coefficient of the area to be monitored, the moving speed and moving direction are determined based on the difference in the change of pixels in the smoke area between adjacent target video frame grayscale images, and the smoke trajectory confusion coefficient is determined based on the uniformity of the change in the moving direction and moving speed of the smoke area between adjacent target video frame grayscale images; A third fire probability of the area to be monitored is obtained based on the area change and texture change of the smoke area between the grayscale images of adjacent target video frames, combined with the duration of smoke in the surveillance video; the third fire probability is used to reflect the smoke dissipation between the grayscale images of adjacent target video frames; the third fire probability is determined based on the duration of smoke and the smoke dissipation coefficient of the area to be monitored, and the smoke dissipation coefficient is determined by comprehensively considering the dissipation parameters of the smoke area between all adjacent target video frame grayscale images. The dissipation parameters of the smoke area between adjacent target video frame grayscale images are the sum of the difference in area change and the difference in overall gradient change of the smoke area between each target video frame grayscale image and the grayscale image of the next adjacent target video frame; The first fire possibility, the second fire possibility and the third fire possibility are integrated to obtain the fire possibility of the area to be monitored; and safety monitoring is performed based on the fire possibility.

2. A construction environment safety intelligent monitoring method according to claim 1, characterized in that: The method for obtaining the first fire probability includes: Obtaining a gray level co-occurrence matrix of the smoke region in the grayscale image of each target video frame; obtaining contrast, correlation, homogeneity and entropy of the gray level co-occurrence matrix; Obtaining a fire smoke texture parameter of the smoke area in the grayscale image of each target video frame according to the contrast, the correlation, the homogeneity, and the entropy; wherein the contrast and the entropy are both positively correlated with the fire smoke texture parameter, and the correlation and the homogeneity are both negatively correlated with the fire smoke texture parameter; The fire smoke texture parameters of the smoke area in all the target video frame grayscale images are combined to obtain a first fire possibility of the area to be monitored.

3. A construction environment safety intelligent monitoring method according to claim 1, characterized in that: The method for obtaining the moving direction and the moving speed includes: Obtaining a motion vector of each pixel in the smoke area in the grayscale image of each target video frame based on an optical flow method; According to the motion vector of each pixel point, the moving speed and moving direction of the smoke area in each target video frame grayscale image are obtained.

4. A construction environment safety intelligent monitoring method according to claim 1, characterized in that: The method for obtaining the smoke trajectory confusion coefficient includes: Constructing a moving vector of the moving speed and the moving direction of the smoke area in each target video frame grayscale image; Obtaining the Euclidean distance between the corresponding motion vectors between each target video frame grayscale image and the next adjacent target video frame grayscale image; The corresponding Euclidean distances between all adjacent grayscale images of the target video frames are integrated to obtain the smoke trajectory confusion coefficient in the area to be monitored.

5. The method for intelligent monitoring of construction environment safety according to claim 1, characterized in that: The method for obtaining the duration of the smoke includes: The product of the appearance frequency of the smoke area in all target video frame grayscale images and the frame rate of the target video frame grayscale image is used as the smoke duration.

6. A construction environment safety intelligent monitoring method according to claim 1, characterized in that: The smoke duration is positively correlated with the third fire probability, and the smoke dissipation coefficient is negatively correlated with the third fire probability.

7. A construction environment safety intelligent monitoring method according to claim 1, characterized in that: The method for performing safety monitoring according to the fire possibility comprises: When the fire possibility is greater than a preset threshold, a safety warning is issued to the monitored area.

8. The method for intelligent monitoring of construction environment safety according to claim 1, characterized in that: The preset direction is a direction perpendicular to the ground and upward.

9. A construction environment safety intelligent monitoring system, characterized in that: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the construction environment safety intelligent monitoring method as described in any one of claims 1 to 8 are implemented.

Images